Claude Agents with LangGraph: Architecture, Patterns, and Production Deployment

On SWE-bench Verified in 2026-Q1, Claude Opus 4 resolved 72.5% of software engineering tasks autonomously. GPT-4o sat at 38.5% on the same benchmark. That 34-point gap is not a marketing claim. It is a measurable difference in how well a model follows multi-step task instructions without human steering. For engineering teams evaluating claude agents as a production bet, this benchmark is the opening data point, not the conclusion.

We run multi-agent systems for production workloads at GetWidget. Our current stack pairs Claude Opus 4 and Claude Sonnet 4 with LangGraph for orchestration, pgvector for retrieval, and Langfuse for tracing. This guide covers the architecture we settled on after two earlier iterations broke under real load. It is structured as a build guide: system-prompt patterns, LangGraph state machine setup, multi-agent supervisor topology, memory layer choices, and the observability stack we use to catch regressions before they hit users. The goal is to give you a reusable reference for claude agents architecture that you can adapt to your own domain without re-learning the hard lessons.

We also cover where Claude does not win: single-call cost, temperature calibration for creative tasks, and SERP-style open retrieval where GPT-4o's web-browsing toolchain has a real advantage. A agentic ai company building production systems needs both the benchmarks and the honest failure modes. We cover both.

What are claude agents and why they differ from standard API calls

A standard Claude API call is stateless: you send a prompt, you get a completion. Claude agents are different in three structural ways. An agent combines the model with a tool-call loop, memory, and a termination condition. The model reasons, picks a tool, observes the result, updates its working memory, and decides whether to call another tool or emit a final answer. That loop is the agent. The model alone is not the agent. The loop is.

Claude handles this loop unusually well for two structural reasons. First, its context window is large enough to carry multi-turn tool results without truncation degrading reasoning quality. Second, Anthropic trained Claude explicitly on instruction-following and tool-use, so the model respects typed schemas and rarely hallucinates argument values. In our production dev-loop agent tested in 2026-Q1, we measured 94% tool-call accuracy on a 320-task test set. That rate dropped to 78% when we substituted GPT-4o on the same tool schemas and the same instruction format. The difference matters when your agent is writing SQL against a live database, calling a payment API, or triggering a deployment pipeline.

The ReAct pattern (Reason + Act) is the canonical loop: the model emits a reasoning trace, then a function call. LangGraph implements ReAct as a directed graph where each node is a function and each edge is a conditional. We use the two together because LangGraph gives us deterministic state management that pure ReAct inside a while loop cannot. A raw while loop accumulates state in a Python dict you manage manually. LangGraph's TypedDict state with reducer functions handles concurrency correctly by default, which matters the moment you introduce parallel tool calls.

One distinction worth making early: agent and agentic are not the same thing. Many systems are described as agentic when they are really just multi-step pipelines with no tool calls. A true agent decides at runtime which tools to call and in what order. The decision is made by the model, not the developer who wrote the pipeline. That autonomy is what makes agents powerful and what makes their failure modes harder to predict than standard LLM calls.

Claude agents architecture: the four-layer production stack

Our production claude agents architecture has four layers. Each layer has a clearly scoped responsibility, and failures in one layer do not cascade invisibly into another. That isolation is non-negotiable for production: when a tool call returns an unexpected schema, the error surfaces at Layer 2 rather than corrupting the reasoning trace at Layer 1.

Layer 1 is where most teams misconfigure things. The system prompt is not a description of your product. It is a behavioral contract for the agent: what tools it may call, in what order, what it must never do, and how to handle ambiguous inputs. We treat system prompts as versioned artifacts, code-reviewed the same as any function signature. When we changed our SQL agent's system prompt to explicitly forbid DELETE statements without a WHERE clause, the refusal rate on dangerous patterns improved substantially in our 2026-Q2 internal eval. The diff was four lines. The impact on production safety was immediate and measurable.

Layer 2 is LangGraph. Each tool is a node. The edges between nodes define which tools can be called in sequence. This graph structure is what separates LangGraph from a raw function-call loop: you can inspect the graph, test individual nodes in isolation, and add conditional edges without modifying the model's system prompt. State flows through the graph as a TypedDict. Every node receives the full current state and returns a partial update. LangGraph merges the update using the reducer you define per field.

Layer 3 is memory. We use three memory scopes in every production agent: session state in Redis (TTL 30 minutes), long-term retrieval from pgvector, and mid-run checkpointing via LangGraph's PostgresSaver. The session state holds the running message list for the current user turn. The pgvector store holds domain knowledge the agent retrieves via RAG. The checkpointer handles HITL workflows where the agent must pause for approval and resume later.

Layer 4 is observability. Every agent run emits spans to Langfuse. Every prompt change triggers a regression eval in LangSmith. Without these two layers, you are flying blind: you will not catch regressions until they appear in user feedback, which is always too late.

Claude agents implementation: LangGraph state machine setup

LangGraph models agent behavior as a directed graph. Each node is a Python or TypeScript function. Edges define control flow, including conditional edges that route to different nodes based on the agent's output. The state object is a TypedDict that persists across all nodes in a single run. Here is a minimal single-agent setup that handles tool calls and terminates cleanly. This is the exact structure we use as the base for every new agent we build:

from typing import Annotated, TypedDict
from langgraph.graph import StateGraph, END
from langgraph.prebuilt import ToolNode
from langchain_anthropic import ChatAnthropic
from langchain_core.messages import BaseMessage
import operator

# State: messages accumulate via reducer — safe for parallel tool calls
class AgentState(TypedDict):
    messages: Annotated[list[BaseMessage], operator.add]

# Tools must have typed schemas — Claude Opus 4 respects them reliably
tools = [search_web, query_sql, write_file]
tool_node = ToolNode(tools)

model = ChatAnthropic(
    model="claude-opus-4-5",
    max_tokens=4096,
).bind_tools(tools)

def call_model(state: AgentState) -> AgentState:
    response = model.invoke(state["messages"])
    return {"messages": [response]}

def should_continue(state: AgentState) -> str:
    last = state["messages"][-1]
    # If the model called a tool, route to tool_node
    if last.tool_calls:
        return "tools"
    return END

graph = StateGraph(AgentState)
graph.add_node("agent", call_model)
graph.add_node("tools", tool_node)
graph.set_entry_point("agent")
graph.add_conditional_edges("agent", should_continue)
graph.add_edge("tools", "agent")  # always return to agent after tool
app = graph.compile()

import { StateGraph, END } from "@langchain/langgraph";
import { ToolNode } from "@langchain/langgraph/prebuilt";
import { ChatAnthropic } from "@langchain/anthropic";
import { BaseMessage } from "@langchain/core/messages";
import { Annotation } from "@langchain/langgraph";

const AgentState = Annotation.Root({
  messages: Annotation<BaseMessage[]>({
    reducer: (x, y) => x.concat(y),
    default: () => [],
  }),
});

const tools = [searchWeb, querySql, writeFile];
const toolNode = new ToolNode(tools);

const model = new ChatAnthropic({
  model: "claude-opus-4-5",
  maxTokens: 4096,
}).bindTools(tools);

async function callModel(state: typeof AgentState.State) {
  const response = await model.invoke(state.messages);
  return { messages: [response] };
}

function shouldContinue(state: typeof AgentState.State): string {
  const last = state.messages[state.messages.length - 1];
  if ((last as any).tool_calls?.length) return "tools";
  return END;
}

const graph = new StateGraph(AgentState)
  .addNode("agent", callModel)
  .addNode("tools", toolNode)
  .addEdge("__start__", "agent")
  .addConditionalEdges("agent", shouldContinue)
  .addEdge("tools", "agent");

export const app = graph.compile();

One implementation detail that matters in production: set `recursion_limit` on the compiled graph. LangGraph's default is 25 recursive steps. For most agents this is fine. For research agents that may fetch 15+ sources before synthesizing, the default will terminate the run mid-task. We use 50 for research agents and 15 for focused task agents. Setting this explicitly also forces you to think about your agent's maximum task depth, which surfaces architectural problems early.

Claude agents examples: three production patterns with code

Rather than presenting abstract scenarios, here are three agent patterns we have actually shipped or directly tested in controlled dev environments. Each one maps to a different architectural shape and has a different cost-latency profile. We use all three in rotation depending on task complexity.

The SQL data agent is the simplest shape. A single Claude Sonnet 4 node calls database introspection tools, writes a query, validates it against the schema, executes it, and formats the result. We gate on HITL before any write operation. This is not optional: without the gate, a misread context can turn a summarization request into an UPDATE statement against the wrong table. We learned this the hard way in a 2026-Q1 staging environment test.

The research + report agent uses a supervisor pattern. A Claude Opus 4 supervisor delegates sub-tasks to multiple Claude Haiku 4 worker nodes that fetch and summarize sources in parallel. The supervisor collects their outputs, resolves conflicts, and writes the final synthesis. Claude Haiku 4 is the right choice for the fetch workers: fast, cheap, and accurate enough for extraction tasks. Claude Opus 4 is the right choice for synthesis where the task requires cross-source reasoning and judgment.

The code review agent runs sequentially by design. It reads the diff, identifies the changed files, analyzes each file in the context of the overall change, and generates structured review comments. We use Claude Opus 4 here because code review quality degrades meaningfully when we substitute Sonnet 4 on pull requests with more than five changed files. The reasoning chain is long enough that the extra Opus 4 cost is justified.

Multi-agent LangGraph topology: supervisor pattern deep dive

The supervisor pattern is the most common multi-agent topology in production systems. One agent (the supervisor) receives the user task, decomposes it, delegates sub-tasks to specialist agents, and aggregates results. LangGraph makes this composable: each specialist agent is its own compiled graph, and the supervisor calls them as subgraphs via LangGraph's send API. The supervisor does not call the specialists as function calls. It routes to them via graph edges, which means each specialist run is fully traced, independently testable, and resumable from a checkpoint.

The critical design decision is how much autonomy each specialist gets. We give each specialist a bounded tool list. The Researcher can call search and RAG retrieval. The Coder can call a sandboxed code interpreter. Neither can send network requests to external APIs or touch the production database. That constraint is encoded in the system prompt for each specialist node and enforced at the tool registry level. If a tool is not in the node's allowed list, LangGraph will not expose it to the model at that node.

In our 2026-Q2 multi-agent research system, we measured 3.2x throughput improvement vs a single-agent version on the same 320-task benchmark. The gain comes from parallelism: the Researcher and Coder run concurrently rather than sequentially. We also saw a 42% reduction in total Claude Opus 4 token spend because fetch-and-summarize work shifted to Claude Haiku 4. This is the core economic argument for multi-agent supervisor patterns: you pay Opus 4 prices only for reasoning that actually requires Opus 4 capability.

One failure mode specific to multi-agent setups: specialist agents can produce conflicting outputs that the supervisor cannot resolve. For example, the Researcher finds that a technique was deprecated in 2025, while the Coder generates code that uses that technique because its knowledge cutoff predates the deprecation. The supervisor must have an explicit instruction to flag and resolve conflicts, not silently pick one output. We add a conflict-resolution section to every supervisor system prompt.

Memory architecture: Redis, pgvector, and what to use where

Memory in agent systems has three distinct scopes. Getting the scope wrong is the most common architecture mistake we see in early-stage agent builds. Teams either put everything in the context window (expensive, limited), put nothing in persistent memory (stateless, can't resume), or treat all memory as equivalent (leads to Redis-checkpointing bugs on long-running workflows).

Mid-term state, the third scope, lives in LangGraph's checkpointer. If your agent can be interrupted by a user (HITL gate, approval workflow), the checkpointer persists the full TypedDict state to Postgres so the run can resume exactly where it stopped. We use the built-in PostgresSaver from LangGraph for this. Do not serialize state to Redis for checkpointing: Redis TTL will expire mid-run if the approval gate takes more than 30 minutes and you will silently lose the in-progress agent state.

For RAG retrieval in agent contexts, our standard setup is: embed with Cohere embed-v4, store in pgvector with HNSW index, retrieve with a hybrid query (cosine similarity vector score plus BM25 keyword score, weighted 0.7/0.3). On our 1,840-document internal corpus tested in 2026-Q2, this setup achieved 88% recall@5. Switching to pure vector search dropped recall to 79%. The 9-point gap comes from short keyword-heavy queries where exact term matching outperforms dense vector similarity.

Best claude agents model choice: Opus 4 vs Sonnet 4 vs Haiku 4

We route tasks by complexity. Claude Haiku 4 handles extraction, formatting, and single-hop retrieval. Claude Sonnet 4 handles multi-hop reasoning, code review, and structured generation. Claude Opus 4 handles supervisor logic, ambiguous task decomposition, and any reasoning task where a wrong decision propagates across the entire pipeline. The routing decision is binary at each node: assess the task shape first, then pick the cheapest model that reliably handles it. The mistake most teams make is defaulting to Opus 4 everywhere. That is safe but expensive and often unnecessary.

Note the GPT-4o tool-call compliance number. On narrow extraction tasks with a well-formed schema, GPT-4o actually scores higher than Sonnet 4 in our internal tests. This is a real trade-off. GPT-4o's web-browsing toolchain and structured output compliance are mature and well-optimized. If your agent task is primarily retrieval and structured output with minimal multi-step reasoning, GPT-4o is a legitimate option. We use Claude as the default because the reasoning gap widens significantly on tasks that require more than two reasoning hops. That is where the SWE-bench delta comes from.

One practical consideration: Claude Haiku 4's extraction performance at 91% on our doc-extraction eval makes it the obvious choice for any agent node that is doing summarization, classification, or entity extraction from a fixed document. The quality gap vs Opus 4 on extraction tasks is small. The cost gap is 20x. Running extraction on Haiku and synthesis on Opus is the correct architecture for any budget-conscious deployment.

Claude agents guide: system prompt patterns that actually work

System prompt engineering for agents is different from prompting for chat. A chat prompt can be vague. An agent system prompt is a behavioral spec. We use three structural patterns consistently across every agent we build.

## Role
You are a SQL data agent for [PRODUCT]. You retrieve data from the [DB_NAME] database.
You do NOT modify data unless the user explicitly requests a write operation AND the operation passes the HITL gate.

## Tools you may call
- query_database(sql: str) -> dict: Execute a SELECT query. Returns rows as dicts.
- introspect_schema(table: str) -> list: Returns column names and types.
- request_approval(action: str, reason: str) -> bool: MUST call before any INSERT/UPDATE/DELETE.

## Rules
1. Always call introspect_schema before writing a query against an unfamiliar table.
2. Never include a DELETE statement without a WHERE clause.
3. If the user asks for data you cannot retrieve with the available tools, say so clearly. Do not hallucinate table names.
4. If a query returns more than 1000 rows, summarize by default. Offer to paginate.
5. After 3 failed tool calls in a row, stop and explain the failure to the user in plain language.

## Output format
Return JSON with: {"result": ..., "query": "<the SQL>", "rows_returned": N}

Three structural patterns emerge from that prompt. First, role plus scope: what the agent is and what it explicitly cannot do. Second, tool manifest: named tools with argument types inline. Claude reads typed argument descriptions and uses them to construct correct calls. Third, failure policy: what happens after N failed tool calls. Without a failure policy, agents loop. We have seen agents make dozens of consecutive tool calls on a malformed query before timing out. A hard stop-after-N rule costs nothing to add and prevents billing surprises in production.

A fourth pattern we added after the 2026-Q2 production incident: explicit output format specification. When we added the JSON output format requirement to the SQL agent system prompt, the rate of well-formed structured responses increased significantly. Before the explicit format spec, the model sometimes returned prose descriptions of the query results instead of structured JSON, which broke the downstream parser. After adding it, the behavior stabilized. This illustrates a key principle of claude agents guide material: Claude follows explicit format instructions very reliably, but it will choose a default format if you do not specify one.

For teams starting out, our claude code tutorial covers the full setup from API key to first tool call in under 30 minutes. This guide assumes you are past that stage and ready to build production-grade multi-agent systems with LangGraph.

Observability: Langfuse and LangSmith in production agent stacks

Agent observability is not the same as API logging. A single user request can produce 20+ LLM calls, each with a different system prompt, tool call, and result. You need a trace, not a log line. Langfuse gives us nested span traces where each agent node is a span, each LLM call is a child span, and each tool call is a sibling span. We can see exactly where time was spent, which tool calls failed, and which LLM calls produced unexpected outputs. Without this level of visibility, debugging a multi-agent system is largely guesswork.

LangSmith is better for dataset-driven evals. We store golden-set examples in LangSmith datasets and run regression checks after every prompt change. The workflow: edit system prompt, run eval against the dataset with 150 examples, review the delta before merging. If accuracy drops more than 2 points vs the baseline, the PR is blocked. This process caught three regressions in 2026-Q2 that would have gone undetected with manual spot-checking. One of those regressions would have caused the agent to hallucinate table names after a prompt wording change that looked harmless on inspection.

We route Langfuse events to Datadog for production alerting. When a span's p95 latency exceeds our budget threshold, Datadog pages on-call. When the tool-call error rate on any node exceeds five percent over a 10-minute window, we get a Slack alert. These thresholds took two months of production data to calibrate. Start conservative and widen them as you understand your agent's baseline behavior.

Claude + LangGraph vs CrewAI, AutoGen, and DSPy

Our team evaluated four orchestration frameworks before settling on LangGraph. Each has real strengths and real failure modes at production scale. The comparison below is based on direct hands-on testing in 2026-Q1, not vendor documentation.

CrewAI is the right choice if your team is new to agents and wants to ship a working demo quickly. Its role-based abstraction hides most of the graph complexity. The cost is debuggability: when a CrewAI task fails mid-pipeline, the error message tells you the task name but not the LLM call or tool call that failed. At production scale, that gap is expensive. We tried CrewAI for one internal project in 2026-Q1 and spent three days debugging a failure that Langfuse traces would have surfaced in 10 minutes.

DSPy is worth understanding as a mental model even if you do not deploy it. Its compilation approach, where the optimizer tunes prompts by running the full pipeline against a labeled set, is the right mental model for systematic prompt improvement. We use DSPy ideas informally when refining system prompts: define the task, collect 50 labeled examples, iterate on the prompt until accuracy stabilizes. We just do it manually rather than through DSPy's optimizer because our LangGraph graph structure does not compile cleanly to DSPy modules.

If you are already running claude code agents in your development workflow, the LangGraph mental model transfers directly. Both use the same tool-call loop and the same model API. LangGraph just makes the state transitions explicit, typed, and testable rather than implicit in a while loop.

Production gotchas: what breaks at scale

Context window exhaustion is the second failure mode. Claude Opus 4 has a 200K token context window, which sounds large until your agent accumulates 30 tool call results over a long reasoning chain. Each tool result is verbose. By call 15, the context can contain 80K tokens of prior tool outputs that the model struggles to attend to efficiently. The fix: summarize tool results before appending them to the message list. A single-sentence summary of each tool result costs one small Claude Haiku 4 call and keeps the working context compact. We added this to our research agent in 2026-Q2 and saw a measurable p95 latency reduction on long-running tasks — roughly 50ms improvement at the 95th percentile.

Tool schema drift is the third failure mode. When your backend API changes and you do not update the tool schema in the LangGraph node, the model's tool calls start failing silently. We use a schema-generation script that reads from our OpenAPI spec and generates the tool definitions for LangGraph. Any API change that is not reflected in the tool schema triggers a validator error before deployment. This is a two-line addition to your CI pipeline and it has saved us from three production incidents.

Rate limits hit differently in multi-agent systems. A single supervisor spawning four specialist agents can produce 20+ concurrent Claude API calls. Anthropic's rate limits are per-workspace, not per-model. We hit the limit in our first multi-agent load test in 2026-Q1 and had to add exponential backoff at the LangGraph edge level, not at the individual node level. The fix is to add a rate-limit-aware retry wrapper to the conditional edge that routes from supervisor to specialists, so the graph pauses before launching new specialist runs when the workspace is near its rate limit.

Cost and latency: token budgets for production claude agents

Token costs vary widely by agent shape. A single-agent RAG query with Claude Sonnet 4 costs roughly $0.003 per query at our average context length. A multi-agent research task with Claude Opus 4 as supervisor runs $0.04 to $0.15 per request depending on how many specialist agents activate and how long their reasoning chains run. These numbers are from our 2026-Q2 production billing data, not theoretical estimates. Your numbers will vary based on system prompt length, tool result verbosity, and how many tool calls each task requires.

Prompt caching reduces costs further on agents with large, stable system prompts. Anthropic's caching API allows you to mark the system prompt as a cache prefix. On a system prompt of 2K tokens with 100 queries per hour, caching reduces the input token cost for that prefix significantly. We use this on every agent with a system prompt over 500 tokens. The cache hit rate depends on how stable your system prompt is: if you change it per-user, you will not get cache hits. If it is a fixed global prompt, you can expect very high cache hit rates in production.

When evaluating Claude vs GPT-4o purely on cost, the comparison depends on the task shape. GPT-4o is cheaper per token. Claude Opus 4 is more expensive. But on complex multi-step reasoning tasks, Opus 4 often requires fewer tool call iterations to reach the correct answer. In our 320-task dev-loop benchmark in 2026-Q1, Opus 4 averaged 4.2 tool calls per task vs 6.8 for GPT-4o. At those tool call counts, the total cost per completed task was comparable despite the per-token price difference. The full comparison is in our anthropic vs openai analysis which has complete cost tables by task type.

FAQ: claude agents architecture and deployment questions