Building RAG Applications with LangChain

This is the second post in our LangChain fundamentals series, building on chains, prompts, and models. It also assumes the RAG concepts from our OpenAI series.

We built a RAG pipeline from raw embeddings and manual cosine-similarity comparisons earlier in this blog — genuinely useful for understanding what's actually happening underneath, and genuinely more manual work than a real application needs to take on directly. LangChain provides purpose-built abstractions for the recurring parts of that pipeline: loading documents, splitting them into chunks, storing and searching embeddings.

Loading and splitting real documents

from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
 
loader = PyPDFLoader("employee-handbook.pdf")
documents = loader.load()
 
splitter = RecursiveCharacterTextSplitter(
    chunk_size=500,
    chunk_overlap=50,
)
chunks = splitter.split_documents(documents)
print(f"{len(documents)} pages split into {len(chunks)} chunks")

24 pages split into 87 chunks

RecursiveCharacterTextSplitter tries to split on natural boundaries first — paragraph breaks, then sentences, then words — only falling back to a hard character-count cut when nothing more natural is available nearby. This directly addresses the chunking trade-off flagged in our OpenAI RAG post: chunk_overlap gives each chunk a small amount of shared context with its neighbor, so a fact or sentence that happens to fall right at a chunk boundary isn't lost entirely from either piece.

Embedding and storing in a vector store

from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import Chroma
 
embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
vectorstore = Chroma.from_documents(chunks, embeddings, persist_directory="./chroma_db")

One call replaces the manual embedding-and-storage loop from our raw-Python RAG example — Chroma.from_documents embeds every chunk and stores the results in a local vector database, persisted to disk so it doesn't need to be rebuilt on every application restart. Swapping Chroma for Pinecone, Weaviate, or pgvector for a production deployment at real scale is a difference of a few lines, not a pipeline rewrite — the same "swap the provider behind a consistent interface" benefit covered in the previous post, now applied to vector storage specifically.

Retrieval, as a first-class LangChain object

retriever = vectorstore.as_retriever(search_kwargs={"k": 3})
relevant_chunks = retriever.invoke("How many vacation days do I get?")

as_retriever() turns the vector store into a Retriever — an object with a consistent .invoke() interface, exactly like the models and parsers from the previous post, meaning a retriever can be dropped directly into an LCEL chain alongside them.

Composing the full RAG chain

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough
 
prompt = ChatPromptTemplate.from_messages([
    ("system", "Answer using only this context. If the answer isn't here, say you don't know.\n\n{context}"),
    ("user", "{question}"),
])
 
def format_docs(docs):
    return "\n\n".join(doc.page_content for doc in docs)
 
rag_chain = (
    {"context": retriever | format_docs, "question": RunnablePassthrough()}
    | prompt
    | model
    | StrOutputParser()
)
 
answer = rag_chain.invoke("How many vacation days do I get?")
print(answer)

Based on the employee handbook, full-time employees receive 25 days
of paid vacation per year, accrued monthly.

This is the exact same four-step RAG pattern from our OpenAI series — retrieve, format context, prompt with that context, generate — expressed as one composed LCEL chain instead of a hand-written function. RunnablePassthrough() simply passes the original question straight through unchanged, so it's available to the prompt template alongside the retrieved context, both flowing into the same final prompt step.

Where this earns its keep over the hand-rolled version

The manual version from our OpenAI series is genuinely clearer for understanding the concept, and entirely reasonable for a small, fixed set of documents. LangChain's version earns its abstraction once you need: document loaders for multiple real file formats (PDF, Word, HTML, Notion exports, and dozens more, each with its own loader), a persistent, swappable vector store rather than an in-memory Python list, and composability with everything else in the ecosystem — memory, agents, and tools, covered in the remaining posts in this series, all sharing the same underlying Runnable interface as the retriever and chain shown here.

What to actually remember from this post

RecursiveCharacterTextSplitter chunks on natural boundaries first — paragraphs and sentences, before falling back to a hard character cut.
chunk_overlap prevents a fact from being lost entirely at a chunk boundary — the concrete fix for the chunking trade-off raised in our OpenAI RAG post.
A retriever is a first-class, composable object in LangChain, sharing the same .invoke() interface as models and parsers.
The full RAG chain composes retrieval, formatting, prompting, and generation into one LCEL pipeline — the same four conceptual steps as the hand-rolled version, with far less manual glue code.

Next in the series: LangChain Memory: Giving Your Application Context, where we cover how a chain remembers previous turns in an actual conversation.