LangChain Memory: Giving Your Application Context

This is the third post in our LangChain fundamentals series, building on chains and RAG applications.

We covered why an ever-growing conversation history leads to linearly-growing cost, and covered sliding windows and summarization as the two standard fixes, in our OpenAI series. LangChain's memory abstractions are essentially structured, reusable implementations of exactly those same patterns, wired directly into a chain rather than managed by hand.

Message history: the baseline

from langchain_core.chat_history import InMemoryChatMessageHistory
from langchain_core.runnables.history import RunnableWithMessageHistory
 
store = {}
 
def get_session_history(session_id: str):
    if session_id not in store:
        store[session_id] = InMemoryChatMessageHistory()
    return store[session_id]
 
chain_with_history = RunnableWithMessageHistory(
    rag_chain,  # from the previous post in this series
    get_session_history,
    input_messages_key="question",
    history_messages_key="history",
)
 
response = chain_with_history.invoke(
    {"question": "How many vacation days do I get?"},
    config={"configurable": {"session_id": "user-123"}},
)

RunnableWithMessageHistory wraps any chain (here, the RAG chain built in the previous post) with automatic conversation tracking, keyed by session_id — meaning a real application serving many concurrent users keeps each user's conversation genuinely separate, without manually threading a conversation ID through every function call by hand.

The cost problem doesn't disappear — it's just managed more consistently

This is the detail worth being explicit about: InMemoryChatMessageHistory is functionally the same "append every message forever" pattern flagged as a real cost problem in our OpenAI series — LangChain gives it a clean interface, but the underlying growth-without-bound issue is identical unless you deliberately manage it.

from langchain.memory import ConversationSummaryBufferMemory
 
memory = ConversationSummaryBufferMemory(
    llm=model,
    max_token_limit=500,
)

ConversationSummaryBufferMemory is the direct, built-in equivalent of the manual summarization function from our OpenAI cost-management post: it keeps recent messages verbatim up to a token budget, and automatically summarizes older messages using the model itself once that budget is exceeded — the same trade-off (full fidelity for recent context, compressed fidelity for older context), implemented once, reusable across every chain that needs it, instead of hand-written per application.

Persisting history beyond a single process

InMemoryChatMessageHistory disappears the moment the process restarts — genuinely fine for local development, a real problem for anything running in production across multiple server instances or surviving a redeploy.

from langchain_community.chat_message_histories import RedisChatMessageHistory
 
def get_session_history(session_id: str):
    return RedisChatMessageHistory(session_id=session_id, url="redis://localhost:6379")

Swapping the in-memory store for a Redis-backed one is, again, a one-line change to the same interface — conversation history now survives a process restart and is shared correctly across multiple server instances handling the same user's requests, which matters the moment a real application runs behind a load balancer with more than one backend instance, the exact networking pattern covered earlier in this blog for Docker and Kubernetes.

Memory in a RAG context: a subtlety worth naming

Combining memory with RAG (as the example at the top of this post does) introduces a real design question: should a follow-up question be rephrased using conversation history before retrieval runs, so retrieval itself receives full context?

condense_prompt = ChatPromptTemplate.from_messages([
    ("system", "Rephrase the follow-up question as a standalone question, using the chat history for context."),
    ("user", "Chat history:\n{history}\n\nFollow-up question: {question}"),
])

Without this step, a user asking "what about sick days?" as a follow-up to "how many vacation days do I get?" sends only "what about sick days?" to the retriever — which has no idea that question is actually about a company's paid-time-off policy, and may retrieve entirely unrelated content. Condensing the follow-up into a standalone question first ("What is the company's policy on sick days?") gives the retrieval step what it actually needs to find the right passage. This is a genuinely common, easy-to-miss gap in a first RAG-plus-memory implementation.

What to actually remember from this post

RunnableWithMessageHistory tracks conversation state per session, wrapping any existing chain without rewriting it.
The underlying cost problem from our OpenAI series doesn't disappear — ConversationSummaryBufferMemory is the built-in, reusable equivalent of the manual summarization fix covered there.
Swap the history backend (Redis, a database) for anything that needs to survive a process restart or run correctly across multiple server instances.
Condense follow-up questions into standalone ones before retrieval, in a RAG-plus-memory setup — otherwise the retriever never sees the context a follow-up question actually depends on.