LangGraph and LangChain: Building Production-Grade LLM Applications

Introduction

Building production-grade applications with large language models in 2026 means choosing an orchestration framework. The raw API pattern — a single chat completion call — works for prototypes but collapses under the weight of multi-turn conversations, retrieval-augmented generation, tool-using agents, and human-in-the-loop workflows. You need state management, observability, error recovery, and a programming model that scales with complexity.

The LangChain ecosystem is the most mature and widely adopted answer to this problem. It comprises four products: LangChain (the core orchestration framework), LangGraph (state-graph-based agent and workflow engine), LangSmith (observability and evaluation platform), and LangServe (deployment infrastructure). Together, they form a full lifecycle platform for LLM application development — from prototyping to production monitoring.

This guide is a deep technical walkthrough of the entire ecosystem. We cover the core abstractions, the LangChain Expression Language (LCEL) for composing chains, building RAG pipelines, designing state graphs in LangGraph for agents and multi-step workflows, persistence and checkpointing, streaming, human-in-the-loop patterns, observability with LangSmith, deployment with LangServe, and production best practices. By the end, you will know not just how to use these tools but when to use them — and when to reach for something else.

The LangChain Ecosystem

The ecosystem is often referred to collectively as "LangChain," but each component serves a distinct purpose [1].

LangChain is the core framework. It provides the abstractions — models, prompts, chains, retrievers, tools, memory — that let you compose LLM operations in a declarative, composable way. Its defining contribution is the LangChain Expression Language (LCEL), a pipe-based syntax for chaining operations that supports streaming, async, batching, and parallel execution out of the box.

LangGraph extends LangChain with a graph-based execution model. Where LangChain chains are linear or DAG-shaped, LangGraph supports cycles, branching, conditional routing, and persistent state. This makes it the natural choice for agents — systems that loop, call tools, observe results, and decide whether to continue. LangGraph also introduces checkpointing, which persists execution state at every step, enabling human-in-the-loop interventions, fault recovery, and time travel debugging.

LangSmith is the observability and evaluation platform. It traces every LLM call, chain invocation, tool execution, and graph step, surfaces latency and token usage, and provides a playground for running evaluations. It is the operational layer that production applications need but that open-source frameworks historically leave as an exercise to the developer [2].

LangServe deploys LangChain chains and LangGraph graphs as REST APIs. It auto-generates OpenAPI schemas, streaming endpoints, and runnable UIs from your compiled graph or chain. It is the simplest path from a Jupyter notebook to a production endpoint.

Core Abstractions: Models, Prompts, and Chains

Every LangChain application is built on a small set of core abstractions. Understanding them is essential because they compose uniformly — a chain that returns a string can be piped into a prompt that expects a string, which can be piped into a model that expects a prompt, and so on.

Models wrap LLM providers. LangChain supports OpenAI, Anthropic, Google, Azure, AWS Bedrock, Together AI, Ollama, and dozens more through a unified interface. Every model implements the sameinvoke,stream, andbatchmethods. This means you can swap providers by changing a single import and config object, not rewriting every invocation.

Prompts are templated inputs to models.ChatPromptTemplateandPromptTemplateaccept a template string with named placeholders and format it at runtime. Prompts can be composed, serialised to JSON, and loaded from Hub. LangChain Hub is a registry of community-contributed prompts that you can pull and adapt rather than writing from scratch.

Chains are sequences of operations. In LangChain, a chain is any object that implements the Runnable interface — invoke, stream, batch. Modern chains are built with LCEL rather than the legacy Chain class. A typical chain pipes a prompt into a model into an output parser: prompt | model | parser. This expression is itself a Runnable that supports streaming, async, and batching automatically.

Retrievers and Tools

Two more abstractions connect LLMs to the outside world: retrievers for fetching relevant context, and tools for executing actions.

Retrievers implement a standard interface for fetching documents given a query. A vector store retriever wraps a similarity search. A parent-document retriever retrieves chunks and returns the full parent document. A multi-query retriever generates multiple query variations to improve recall. The retriever abstraction is what makes RAG pipelines swappable: you can change from dense retrieval (embedding-based) to sparse (BM25) to hybrid without touching the chain logic [3].

Toolsare functions that the model can invoke during execution. LangChain tools wrap any function with a name, description, and JSON schema for the arguments. The model decides when to call a tool based on the tool's description and the current conversation context. Tools are the building blocks of agents — every tool call is a structured interaction that the framework can log, trace, and recover from.

Defining a tool in LangChain is a matter of decorating a function:

@tool
def calculate_growth(previous: float, current: float) -> float:
    """Calculate the percentage growth between two values."""
    return ((current - previous) / previous) * 100

The decorator auto-generates the JSON schema using Pydantic, registers the function with the tool registry, and makes it available for model binding. The model receives the schema, decides whether to call the tool, and the framework dispatches the execution [4].

LangChain Expression Language (LCEL)

LCEL is the most important design decision in modern LangChain. It replaces the legacy Chain subclasses with a pipe operator that composes Runnables into a lazy, composable DAG.

Every Runnable — model, prompt, parser, retriever, tool — supports the same interface:

• invoke(input): synchronous single execution.
• stream(input): yields output tokens as they are produced.
• batch(inputs): executes multiple inputs concurrently.
• astream(input): async streaming variant.

The power of LCEL is that composing Runnables with the pipe operator produces a new Runnable. This means your entire chain — prompt, model, parser — is itself a Runnable with all the same methods. You can pipe a chain into another chain, add retry logic withwith_retry(), add fallbacks withwith_fallbacks(), and bind runtime configuration without touching the chain definition.

LCEL also handles runtime concerns automatically. A piped chain respects theRunnableConfigobject propagated through the entire pipeline, enabling callbacks, metadata tagging, and per-step tracing without threading configuration through every component manually.

Building RAG Chains with LCEL

Retrieval-Augmented Generation is where LCEL's composability shines. A RAG chain combines a retriever, a prompt that injects retrieved context, a model that generates the answer, and an output parser that formats the response. Each piece is independently testable, swappable, and traceable.

import { ChatOpenAI } from "@langchain/openai";
import { createRetrievalChain } from "langchain/chains/retrieval";
import { createStuffDocumentsChain } from "langchain/chains/combine_documents";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { StringOutputParser } from "@langchain/core/output_parsers";

// 1. Define the prompt template
const prompt = ChatPromptTemplate.fromTemplate(
  `Answer the question based only on the provided context.
If the context does not contain enough information,
say so — do not make up an answer.

Context: {context}
Question: {input}

Answer:`
);

// 2. Build the document-combining chain (LCEL)
const combineChain = prompt
  .pipe(model)
  .pipe(new StringOutputParser());

// 3. Wrap with retrieval
const ragChain = await createRetrievalChain({
  retriever: vectorStore.asRetriever(3),
  combineDocsChain: combineChain,
});

// 4. Invoke
const result = await ragChain.invoke({
  input: "What are the three pillars of observability?",
});
console.log(result.answer);

Several design decisions here deserve attention. First, the prompt template explicitly instructs the model to refuse answering if the context is insufficient — this is the standard defence against hallucination in RAG systems, and we discuss it in more depth in our guide on best RAG practices.

Second, the createRetrievalChain helper handles the wiring between the retriever and the document-combining chain. It retrieves documents from the vector store, formats them into the context variable, and passes the original input through to the combine chain. The result is a single Runnable that streams, batches, and traces automatically.

Third, the retriever is parameterised with topK=3, which balances relevance against context window usage. For more sophisticated retrieval — multi-query expansion, hybrid search, re-ranking — you compose additional Runnables into the pipeline before the retrieval step.

For a deeper exploration of RAG architecture patterns, see our post on agentic AI architecture.

Enter LangGraph: State Graphs

LangChain chains handle linear and DAG-shaped workflows well. But many LLM applications require cycles: an agent that calls a tool, observes the result, and decides whether to call another tool or return. This is where LangGraph comes in [5].

LangGraph models application logic as a state graph. The graph has three primitives:

• State: a shared, typed data object that persists across the entire execution. Every node reads from and writes to this state.
• Nodes: functions that receive the current state, perform some operation (model call, tool invocation, data transformation), and return an update to the state.
• Edges: connections between nodes that define control flow. Edges can be unconditional (always go from Node A to Node B) or conditional (route based on the current state).

The state is the heart of the graph. LangGraph uses Pydantic schemas (or its Annotation API) to define the state type, which gives you runtime validation, serialisation for checkpointing, and IDE autocompletion. Each node returns a partial state update, and LangGraph merges it into the shared state after every step.

A directed graph — the same abstraction LangGraph uses for agent workflows and multi-step LLM pipelines.

Nodes, Edges, and Conditional Routing

Building a graph in LangGraph is a three-step process: define the state schema, define the nodes, and wire them together with edges.

Nodes are async or sync functions that accept the state and return a partial update. A node can call an LLM, execute a tool, run arbitrary Python code, or trigger a subgraph. The only contract is that the return value is a dictionary whose keys match the state schema.

Edges come in two flavours. A direct edge (via .addEdge()) always routes from one node to another. A conditional edge (via .addConditionalEdges()) accepts a routing function that inspects the current state and returns the name of the next node. This is how agents decide whether to call a tool or return the final answer.

LangGraph includes two special nodes: START and END. Every graph must define an edge from START to at least one node, and every path through the graph must eventually reach END. Conditional edges typically return the string "__end__" (aliased as END) to terminate execution.

The routing function is the core decision-making primitive. In an agent, it checks whether the last model response contained tool calls: if yes, route to the tool execution node; if no, the agent is done and routes to END. This is exactly how LangGraph implements the agent loop described in our AI agents 2026 frameworks guide.

Building Agents with LangGraph

A LangGraph agent is a cyclic graph with three nodes: call_model, call_tool, and a conditional edge from the model to either the tool or END. This is the agent loop implemented as a state graph.

import { StateGraph, Annotation, START, END } from "@langchain/langgraph";
import { ChatOpenAI } from "@langchain/openai";
import { tool } from "@langchain/core/tools";
import { z } from "zod";

// --- State schema (Pydantic via Annotation) ---
const AgentState = Annotation.Root({
  messages: Annotation(),
  next: Annotation(),
});

// --- Tools ---
const searchTool = tool(
  async ({ query }: { query: string }) => {
    const results = await searchApi.search(query);
    return JSON.stringify(results);
  },
  {
    name: "web_search",
    description: "Search the web for recent information.",
    schema: z.object({ query: z.string() }),
  }
);

const tools = [searchTool];
const llm = new ChatOpenAI({ model: "gpt-4o" }).bindTools(tools);

// --- Nodes ---
async function callModel(state: typeof AgentState.State) {
  const response = await llm.invoke(state.messages);
  return { messages: [response] };
}

async function callTool(state: typeof AgentState.State) {
  const last = state.messages[state.messages.length - 1];
  const toolCall = last.tool_calls?.[0];
  if (!toolCall) throw new Error("No tool call found");
  const result = await searchTool.invoke(toolCall);
  return { messages: [{ role: "tool", content: result }] };
}

// --- Conditional routing ---
function shouldContinue(state: typeof AgentState.State): string {
  const last = state.messages[state.messages.length - 1];
  return last.tool_calls?.length ? "call_tool" : END;
}

// --- Build graph ---
const graph = new StateGraph(AgentState)
  .addNode("call_model", callModel)
  .addNode("call_tool", callTool)
  .addEdge(START, "call_model")
  .addConditionalEdges("call_model", shouldContinue)
  .addEdge("call_tool", "call_model")
  .compile();

// --- Run ---
const result = await graph.invoke({
  messages: [{ role: "user", content: "What was the GDP growth in Q2 2026?" }],
});

Several patterns in this implementation are worth examining. First, the model is bound to tools with .bindTools(tools), which appends the tool schema to every model call. The model responds with a message that includes tool_calls metadata when it decides to invoke a tool.

Second, the shouldContinue function is the routing decision point. It reads the last message in the state and checks for tool calls. This pattern generalises to any conditional logic: you could route based on the content of the response, the confidence score, the number of iterations, or an external signal.

Third, the tool execution node is deliberately simple. It extracts the first tool call, invokes the tool synchronously, and returns a tool message. In production, you would add error handling, retry logic, and timeouts here — the tool node is the most likely point of failure in any agent system.

LangGraph also supports parallel tool execution natively. If the model returns multiple tool calls, you can execute them concurrently in a single node and merge the results back into state. This is critical for agents that need to gather information from multiple sources before synthesising a response.

Persistence and Checkpointing

One of LangGraph's most powerful features is checkpointing. Every time a node finishes executing, LangGraph can persist the entire state — messages, intermediate results, tool outputs, control flow position — to a checkpoint store. This enables three critical capabilities [6].

Fault recovery. If a node crashes or the process is killed, you can resume execution from the last checkpoint. The graph picks up exactly where it left off, with no data loss and no duplicate side effects.

Time travel debugging. You can replay a graph execution step by step, inspecting the state at every checkpoint. This is invaluable for debugging agent failures — you can see the exact state that led to a bad tool call or a hallucinated response.

Human-in-the-loop. The combination of checkpointing and interrupt points (covered in the next section) lets you pause execution, inspect the state, inject human feedback, and resume — all without losing context.

LangGraph supports multiple checkpoint backends. The default is an in-memory store suitable for development. For production, you use the SQLite or Postgres checkpointer, which persists state to durable storage and supports concurrent access across multiple processes. The interface is identical regardless of the backend:

import { SqliteSaver } from "@langchain/langgraph-checkpoint-sqlite";

const checkpointer = SqliteSaver.fromConnString("checkpoints.db");

const graph = new StateGraph(AgentState)
  // ... define nodes and edges ...
  .compile({ checkpointer });

Each execution is scoped by a thread ID. You pass the thread ID in the runtime config when invoking the graph, and LangGraph uses it to associate checkpoints with a specific conversation or workflow instance. This is how multi-turn conversations work in LangGraph — the graph maintains state across invocations within the same thread.

Streaming and Event Handling

LLM applications in 2026 are expected to stream. Users watch tokens appear in real time, watch tool calls being made, and watch intermediate results flow in. LangGraph supports streaming at multiple levels through a unified event system [7].

Token-level streaming emits individual tokens from the LLM as they are generated. This is the familiar typewriter effect used by chatbots and copilots. LangGraph propagates token events from model nodes through the graph without buffering.

Node-level streaming emits events when nodes start and finish executing, along with the state before and after each node. This lets you build a real-time execution view: "Calling web search... Result received... Generating final answer..."

Value-level streaming emits the full state after each node execution. This is the coarsest granularity but the easiest to consume — you receive the complete conversation state at every step and render it directly.

The streaming API is uniform across LangChain and LangGraph:

const stream = await graph.stream(input, {
  streamMode: "values",
  configurable: { thread_id: "thread-1" },
});

for await (const state of stream) {
  const lastMessage = state.messages.at(-1);
  if (lastMessage.role === "assistant" && !lastMessage.tool_calls) {
    render(lastMessage.content);
  }
}

For custom event handling, LangGraph emits structured events through the LangChain callback system. You can attach listeners for model start, model end, tool start, tool end, chain start, chain end, and error events. This is how LangSmith tracing works under the hood — it registers a callback that records every event emitted by the graph [8].

Human-in-the-Loop with Interrupt/Continue

Fully autonomous agents are the goal, but in practice every production agent needs human oversight at critical decision points. LangGraph supports this through interrupt and continue primitives built on top of the checkpointing system.

An interrupt pauses graph execution at a specific node. The graph persists its state and returns control to the caller. No further execution occurs until an external signal — typically a human review — resumes the graph. This is fundamentally different from a timeout or error; the graph is alive, waiting, and fully recoverable.

To interrupt, you pass interruptBefore or interruptAfter when compiling the graph. The parameter accepts a node name or an array of node names. The graph executes normally until it reaches the specified node, then pauses before (or after) that node runs.

Resuming execution is a two-step process. First, inspect the current state with getState(). This returns the full checkpoint, including messages, tool results, and the next node to execute. Second, optionally update the state with updateState() — for example, to inject human approval or to correct a tool result. Finally, call invoke(null, config) to resume from the interruption point.

// --- Streaming ---
const stream = await graph.stream(
  { messages: [{ role: "user", content: "Analyse this report: ..." }] },
  { streamMode: "values" }
);

for await (const event of stream) {
  const last = event.messages[event.messages.length - 1];
  if (last.role !== "tool") {
    process.stdout.write(last.content);
  }
}

// --- Human-in-the-loop with interrupt ---
const graphWithInterrupt = new StateGraph(AgentState)
  // ... nodes and edges ...
  .addNode("review", async (state) => {
    // Execution pauses here — LangGraph persists state
    // and waits for an external signal
    return state;
  })
  .compile({ interruptBefore: ["review"] });

// Run and resume
const config = { configurable: { thread_id: "thread-1" } };

// First run — interrupts before "review"
await graphWithInterrupt.invoke(input, config);

// Inspect the state
const snapshot = await graphWithInterrupt.getState(config);

// Add human feedback
await graphWithInterrupt.updateState(config, {
  messages: [
    {
      role: "human",
      content: "Approved. Proceed with the original plan.",
    },
  ],
});

// Resume from the interruption point
const final = await graphWithInterrupt.invoke(null, config);

This pattern is used extensively in production for content moderation workflows (pause before publishing generated content), financial approvals (pause before executing a trade), and sensitive data handling (pause before accessing a restricted database). The checkpointing ensures that the human reviewer sees the full context that led to the decision, not just the final output.

LangSmith for Observability

Observability is the gap between working prototypes and production deployments. When a LangGraph agent returns a wrong answer in development, you debug it by running it again with print statements. When it returns a wrong answer at 3 AM in production, you need a recorded trace of exactly what happened — every LLM call, every tool invocation, every state transition.

LangSmith solves this with automatic tracing. You configure your LangChain application with a LangSmith API key, and every Runnable and Graph invocation is automatically recorded — no instrumentation code required [9]. Each trace captures:

• The full input and output of every LLM call.
• The prompt template and the rendered prompt.
• Token usage (prompt tokens, completion tokens, total).
• Latency per step and total.
• Tool call arguments and results.
• State snapshots at each graph node.
• Error stack traces when nodes fail.

LangSmith's trace viewer renders these as hierarchical trees that you can expand and inspect. For LangGraph agents, the trace shows the full loop — call model, route to tool, execute tool, route back to model, call model again — making it easy to identify where the agent made a wrong decision.

LangSmith also supports online evaluation. You define evaluators (LLM-as-a-judge, exact match, regex, custom functions) and LangSmith runs them against every production trace. If an evaluator flags a bad response, you get an alert with a link to the full trace. This is how teams move from reactive debugging to proactive quality monitoring.

Evaluation with LangSmith

Beyond observability, LangSmith provides a structured evaluation framework. You define datasets of input-output pairs (or input-only, for tasks with open-ended outputs), then run your chain or graph against the dataset and apply evaluators.

LangSmith supports several evaluator types. Correctness evaluators compare the model output against a ground truth answer using criteria like exact match, contains, or semantic similarity. LLM-as-a-judge evaluators ask a second LLM to rate the output on dimensions like helpfulness, harmlessness, and coherence. Pairwise evaluators compare two model outputs (e.g., before and after a prompt change) and determine which is better.

For agent evaluation, LangSmith supports trajectory evaluation. Instead of evaluating only the final output, you evaluate the entire sequence of tool calls and intermediate results. A trajectory evaluator can check: did the agent call the right tools in the right order? Did it retrieve useful context before answering? Did it stop in a reasonable number of steps?

The evaluation workflow integrates with CI/CD. You define a test dataset, run evaluations on every pull request, and block merges if quality metrics regress. This is the closest the LLM application world has to traditional unit testing, and it is essential for maintaining quality as your prompts, models, and graph topologies evolve.

Deployment with LangServe

LangServe converts any Runnable — a LangChain chain, a LangGraph compiled graph, a custom Runnable — into a REST API with minimal boilerplate. It auto-generates OpenAPI schemas, streaming endpoints, and a playground UI where you can test your endpoints from the browser [10].

The deployment model is straightforward. You define your chain or graph in a Python file, add a LangServe server wrapper, and run it with any ASGI server (Uvicorn, Gunicorn). LangServe automatically exposes four endpoints per Runnable:

• POST /invoke: synchronous invocation, returns the full result.
• POST /stream: server-sent events stream of output tokens.
• POST /batch: concurrent invocation with multiple inputs.
• GET /schema: OpenAPI JSON schema for the endpoint.

LangServe handles several production concerns automatically. It sets CORS headers, manages concurrent requests with configurable worker pools, integrates with LangSmith tracing (every invocation is traced), and supports authentication via API keys or custom middleware. For a detailed walkthrough of deploying LLM applications in production, see our documentation.

Best Practices: Error Handling, Rate Limiting, Testing

Production LLM applications fail in predictable ways. The patterns that follow are not optional — they are the difference between a demo and a service.

Error Handling

LLM calls fail for three reasons: rate limits (throttling by the provider), transient errors (network blips, server overload), and content moderation flags. LangChain's Runnable interface supports fallbacks and retries natively:

const model = new ChatOpenAI({ model: "gpt-4o" })
  .withFallbacks({
    fallbacks: [new ChatOpenAI({ model: "gpt-4o-mini" })],
  })
  .withRetry({
    stopAfterAttempt: 3,
    maxDelay: 30_000,
  });

The fallback chain means if GPT-4o returns a rate limit error, the framework automatically retries with a different model without changing application code. Retries use exponential backoff with jitter by default.

Rate Limiting

Every LLM provider imposes rate limits. LangChain does not have a built-in rate limiter, but the recommended approach is to use middleware that wraps the Runnable with a token bucket or sliding window. TheRunnableWithRateLimit pattern (custom, not built-in) wraps the invoke method with a semaphore that respects provider-specific limits. For LangGraph agents that make multiple rapid tool calls, you also need per-tool rate limiting to avoid overwhelming downstream APIs.

Testing

Testing LLM applications requires three levels. Unit tests validate individual components — does the prompt template render correctly? Does the tool function handle edge case inputs? Do conditional routing functions return the expected next node? Integration tests run the full chain or graph against recorded inputs and verify the output structure — not the exact content (which is non-deterministic) but the schema, the presence of expected fields, and the absence of error indicators. Evaluation tests use LangSmith datasets to measure quality metrics — correctness, helpfulness, trajectory quality — and enforce regression gates in CI.

A common mistake is testing LLM outputs for exact string matches. This produces flaky tests that break whenever the model changes phrasing. Instead, test for structural properties: the output should be valid JSON, the number of tool calls should be within bounds, the answer should contain no placeholder text, the confidence score should be above a threshold.

When to Use LangChain vs Raw API vs Other Frameworks

No framework is universally the right choice. The decision depends on the complexity profile of your application.

Raw API (OpenAI, Anthropic, etc.) is the right choice for simple, stateless applications: a single chat completion call, a summarisation endpoint, a classification task. If you have no retrieval, no tool calling, no state across invocations, and no multi-step workflows, the raw API is simpler, has fewer dependencies, and avoids framework lock-in. The threshold is roughly three to five distinct LLM interactions — if you have more than that, the orchestration framework saves you code.

LangChainis the right choice when you need composability across LLM interactions. If you are building a RAG pipeline, a multi-turn chatbot with retrieval, a content generation workflow with multiple steps, or any system where components need to be independently testable and swappable, LangChain's LCEL and abstraction layer provide concrete value. The ecosystem's integrations — 50+ vector stores, 30+ LLM providers, 20+ document loaders — mean you are unlikely to hit a compatibility wall.

LangGraph is the right choice when your application requires state, cycles, and persistence. Agents are the canonical use case, but LangGraph also excels at multi-step data processing pipelines, human review workflows, multi-turn conversations with checkpointing, and any application where control flow depends on the content of intermediate results. If your application can be modelled as a directed graph with shared state, LangGraph is the most mature framework for that paradigm.

Other frameworks are worth evaluating for specific niches. LlamaIndex has stronger data ingestion and indexing capabilities for complex RAG scenarios. CrewAI and AutoGen offer higher-level abstractions for multi-agent teams. DSPy provides automated prompt optimization. Vercel AI SDK offers a simpler, React-native streaming experience. The choice should be driven by the specific bottleneck in your application — data processing, multi-agent coordination, prompt optimization, or frontend integration.