Debugging and Observability in LangChain Applications

This is the fifth and final post in our LangChain fundamentals series, building on everything from chains through RAG, memory, and agents.

A chain that produced the wrong answer, or an agent that took a genuinely strange sequence of actions, is only debuggable if you can actually see every intermediate step: what prompt was sent, what the retriever returned, which tool the agent chose and why, what each step's exact input and output actually was. Without that visibility, "the agent gave a weird answer" isn't a debuggable problem statement — it's a shrug.

The simplest tool: verbose mode

executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

We already saw this in the previous post — verbose=True prints each step directly to the console as it happens. Genuinely useful for local development and quick debugging, and not something you'd want scattered through production logs at real volume, nor does it give you a way to go back and inspect a specific past run after the fact.

LangSmith: tracing built for exactly this problem

import os
 
os.environ["LANGCHAIN_TRACING_V2"] = "true"
os.environ["LANGCHAIN_API_KEY"] = "your-langsmith-api-key"
os.environ["LANGCHAIN_PROJECT"] = "novuspark-support-agent"
 
# Every chain and agent invocation is now traced automatically —
# no other code change required
result = rag_chain.invoke("How many vacation days do I get?")

With tracing enabled, every chain or agent run is captured in LangSmith (LangChain's dedicated observability platform) as a structured trace: every prompt actually sent to the model, every document a retriever returned, every tool call an agent made and the reasoning that preceded it, exact token counts and latency per step. This is the direct answer to "the agent gave a weird answer" — instead of guessing, you open the actual trace for that specific run and see, step by step, exactly what happened and where it diverged from what you expected.

What to actually look for in a trace

• Retrieval quality: did the retriever actually return the relevant chunk, or did it retrieve something plausible-sounding but genuinely unrelated to the question? A wrong final answer traced back to bad retrieval needs a fix in chunking or embedding strategy — covered in our RAG post — not a fix to the prompt.
• Prompt construction: is the actual, fully-assembled prompt — after every template variable was filled in — what you intended? A subtle bug in how context gets formatted before insertion is often invisible until you look at the literal, final prompt text actually sent, not just the template.
• Tool selection: for an agent, did it choose a reasonable tool given the question, or does the tool's description need to be clearer? We flagged in the previous post that a tool's docstring is what the model uses to decide when to reach for it — a trace showing an agent skipping the right tool is often a signal the description needs to be more specific, not that the model is behaving unpredictably for no reason.

Evaluation: testing quality, not just checking for a crash

A chain or agent can run successfully — no exception, no error — and still produce a genuinely low-quality or subtly wrong answer. Standard software tests catch crashes; they don't catch "technically ran without error, but the answer is misleading." LangChain integrates with evaluation frameworks specifically built for this gap:

from langsmith.evaluation import evaluate
 
def correctness_evaluator(run, example):
    # Compare the chain's actual output against a known-good reference answer
    predicted = run.outputs["output"]
    expected = example.outputs["expected_answer"]
    # ... scoring logic, often using another model call to judge similarity
    return {"score": 1 if predicted_matches(predicted, expected) else 0}
 
evaluate(
    rag_chain.invoke,
    data="vacation-policy-qa-dataset",
    evaluators=[correctness_evaluator],
)

Running an evaluation suite against a fixed dataset of real questions with known-good expected answers, on every meaningful change to a prompt or chain structure, is what catches a regression before it reaches production — the direct equivalent of a test suite in ordinary software, applied to LLM output quality specifically, rather than assuming "it still runs" is a sufficient bar for something this probabilistic.

Logging real production usage, not just traces from testing

import logging
 
logger = logging.getLogger("langchain_app")
 
def logged_invoke(chain, inputs, session_id):
    logger.info(f"session={session_id} input={inputs}")
    result = chain.invoke(inputs)
    logger.info(f"session={session_id} output={result}")
    return result

Tracing during development answers "why did this specific test case behave this way." Structured logging in production answers a different, equally necessary question: what are real users actually asking, how often do certain failure patterns show up in the wild, and which prompts or chain structures need the next round of improvement — the same shift from local debugging to production observability that's true of any real software system, applied here to a probabilistic one where "did it work" is a fuzzier question than a stack trace can fully answer on its own.

What to actually remember from this series

• Verbose mode is for quick, local debugging — LangSmith or an equivalent tracing tool is what makes past production runs actually inspectable after the fact.
• A wrong final answer usually traces back to either bad retrieval or a subtly malformed prompt — inspect the actual, fully-assembled request, not just the template.
• "It ran without an error" is not the same bar as "it produced a good answer" — evaluation against a real dataset with known-good answers is what actually tests output quality.
• Production logging and development tracing answer different questions — one tells you what real users are actually asking; the other tells you why one specific test case behaved the way it did.

That closes out our LangChain fundamentals series — from chains and prompts through RAG, memory, agents, and now observability. For genuinely complex, multi-step agent behavior — including branching and revisiting earlier steps — LangGraph, covered next in this blog, extends these same concepts further. If your team is building real production AI applications, this is exactly the kind of hands-on work we build our AI & Generative AI training around.