LangGraph vs Claude Agent SDK

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round drawn
   LangGraph✓fullprobed9/10

   Probes confirm a live llms.txt at docs.langchain.com (HTTP 200) with a documentation index, and individual doc pages are available in agent-friendly .md format (e.g., overview.md) that explicitly reference the llms.txt index for further crawling — this is exactly the agent-oriented docs pattern the story asks for. Missing for 10: no independent/community confirmation of an agent actually consuming llms.txt successfully in practice.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.langchain.com/llms.txt # Docs by LangChain > Documentation for LangSmith, Fleet, and our open sour…”
   • [probe] “PROBE docs-md: HTTP 200 at https://docs.langchain.com/oss/python/langgraph/overview.md > ## Documentation Index > Fetch the complete documen…”

   proof ↗⚑ flag

   Claude Agent SDK✓fullprobed9/10

   Probes confirm the product ships an official llms.txt at code.claude.com/llms.txt (HTTP 200) and markdown-formatted agent-oriented docs (e.g., overview.md) with an explicit documentation index pointer for agents to fetch, directly enabling an AI-native user to point an agent at these resources. Missing for 10: independent third-party confirmation that agents successfully consume this llms.txt in practice.

   • [probe] “PROBE llms.txt: HTTP 200 at https://code.claude.com/llms.txt # Claude Code Docs > Official documentation for Claude Code, Anthropic's agent…”
   • [probe] “PROBE docs-md: HTTP 200 at https://code.claude.com/docs/en/agent-sdk/overview.md > ## Documentation Index > Fetch the complete documentation…”
   • [probe] “official CLI documented at https://code.claude.com/docs/en/cli-reference”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to Claude Agent SDK
   LangGraph~partialclaimed6/10

   LangGraph is a pip-installable Python library with a programmatic graph API (stream/astream, Command, checkpointers) and a CLI that builds/runs an Agent Server locally, all of which support non-interactive, scriptable execution suitable for CI/automation. However, there is no explicit documentation or example of running LangGraph in a CI pipeline or headless automation context specifically. Missing for 10: explicit CI/automation guide or example, documented headless/non-interactive invocation patterns, and independent evidence of real-world CI usage.

   • [github] “pip install -U langgraph”
   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally.”
   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally. The resulting server exposes all API endpoints for r…”
   • [claimed-docs] “LangGraph CLI** is a command-line tool for building and running the [Agent Server](/langsmith/agent-server) locally. The resulting server ex…”
   • [claimed-docs] “LangGraph graphs expose the stream (sync) and astream (async) methods to yield streamed outputs as iterators.”
   • [claimed-docs] “Checkpointers persist a thread's graph state as checkpoints. Use them for short-term, thread-scoped memory, including conversation continuit…”

   proof ↗⚑ flag

   Claude Agent SDK✓fullcommunity8/10

   Docs describe headless/non-interactive automation (subprocess-based CLI, `claude -p`, hosting guide covering Docker/Kubernetes/production deployment, quickstart for autonomous bug-fixing 'without manual intervention'), and a community report confirms a user built 'an entire headless automated workflow around claude -p', corroborating real-world CI-style use. Missing for 10: dedicated CI/CD pipeline examples (e.g. GitHub Actions template) and independent benchmarking of reliability/observability in automated pipelines, and some community friction over harness DX/observability in headless mode tempers the score.

   • [claimed-docs] “Deploy the Agent SDK in production: subprocess architecture, session persistence, scaling, observability, and multi-tenant isolation for Doc…”
   • [claimed-docs] “The Agent SDK spawns and supervises a claude CLI subprocess that owns a shell, a working directory, and session files on disk.”
   • [claimed-docs] “Use the Agent SDK to build an AI agent that reads your code, finds bugs, and fixes them, all without manual intervention.”
   • [github] “The Claude Code CLI is automatically bundled with the package - no separate installation required! The SDK will use the bundled CLI by defau…”
   • [community] “Damn. I just built an entire headless automated workflow around `claude -p`”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round to Claude Agent SDK
   LangGraph✓fullclaimed6/10

   Docs confirm LangChain/LangGraph agents can consume MCP servers via MCPAdapter, which discovers a server's tools and adapts them into LangChain tools for use inside graphs. Missing for 10: deeper first-party walkthrough/code example of wiring an MCP server into a LangGraph agent, and independent/community corroboration of this working in practice.

   • [claimed-docs] “LangChain agents call tools defined on MCP servers through MCPAdapter, which discovers a server's tools and adapts them into LangChain tools…”

   proof ↗⚑ flag

   Claude Agent SDK✓fullclaimed8/10

   Dedicated first-party MCP docs describe connecting the agent to external MCP servers to query databases, integrate with Slack/GitHub, and use other services without custom tool code, and this integrates with the SDK's permission/tool-use system. Missing for 10: independent/hands-on corroboration of MCP server plugging (community evidence covers licensing/UX complaints, not MCP integration specifically) and configuration-level detail beyond the overview.

   • [claimed-docs] “With MCP, your agent can query databases, integrate with APIs like Slack and GitHub, and connect to other services without writing custom to…”
   • [claimed-docs] “The Claude Agent SDK provides permission controls to manage how Claude uses tools. Use permission modes and rules to define what's allowed a…”
   • [claimed-docs] “Custom tools extend the Agent SDK by letting you define your own functions that Claude can call during a conversation.”

   proof ↗⚑ flag

4. ai-native userUse an official CLI

   weight 2 · round to LangGraph
   LangGraph✓fullprobed9/10

   LangGraph ships an official CLI (LangGraph CLI) documented for building and running the Agent Server locally, exposing API endpoints for runs, threads, assistants, etc., with supporting services like managed DB for checkpointing — confirmed by first-party docs and a live probe of the doc page. missing for 10: no independent/community hands-on confirmation of CLI usage beyond vendor docs.

   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally.”
   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally. The resulting server exposes all API endpoints for r…”
   • [claimed-docs] “LangGraph CLI** is a command-line tool for building and running the [Agent Server](/langsmith/agent-server) locally. The resulting server ex…”
   • [probe] “official CLI documented at https://docs.langchain.com/langsmith/cli”

   proof ↗⚑ flag

   Claude Agent SDK✓fullprobed7/10

   The SDK bundles and documents an official `claude` CLI (auto-installed subprocess, dedicated CLI reference page), and community members confirm building real headless automation with `claude -p`. However, hands-on reports cite real friction (poor observability, scroll/render bugs, and new usage-policy restrictions on CLI-based SDK apps), so it works but with notable rough edges. Missing for 10: independent quality benchmarks of the CLI experience, resolution of the harness UX complaints (flashing, scroll issues, lack of observability), and clarity on usage-policy restrictions affecting CLI workflows.

   • [github] “The Claude Code CLI is automatically bundled with the package - no separate installation required! The SDK will use the bundled CLI by defau…”
   • [claimed-docs] “The Agent SDK spawns and supervises a claude CLI subprocess that owns a shell, a working directory, and session files on disk.”
   • [probe] “official CLI documented at https://code.claude.com/docs/en/cli-reference”
   • [community] “Damn. I just built an entire headless automated workflow around `claude -p`”
   • [community] “There is no difference between claude -p and typing into their 'harness' in tmux. They want to make you pay for API which is subsidizing the…”
   • [community] “I've been using claude -p with a harness called codelayer. Their [default] harness is completely garbage when it comes at displaying full de…”

   proof ↗⚑ flag

5. ai-native userDrive the product through a documented public API

   weight 3 · round to Claude Agent SDK
   LangGraph~partialprobed7/10

   LangGraph's Python API (graph construction, streaming, persistence, interrupts) is extensively documented, and the LangGraph CLI/Agent Server exposes REST endpoints for runs, threads, and assistants (docs-30, docs-37), giving programmatic/API access beyond just an SDK. However, a probe for a formal OpenAPI/swagger spec returned 404 on all candidate paths, and community comments note documentation gaps and breaking changes, suggesting the 'public API' is real but not as formally discoverable as a REST-first product. Missing for 10: a published OpenAPI/swagger spec or API reference, and independent confirmation the Agent Server API is stable/production-documented rather than CLI-only.

   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally. The resulting server exposes all API endpoints for r…”
   • [claimed-docs] “LangGraph CLI** is a command-line tool for building and running the [Agent Server](/langsmith/agent-server) locally. The resulting server ex…”
   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.langchain.com/openapi.json, https://docs.langchain.com/swagger.json, https://docs.langc…”
   • [probe] “official CLI documented at https://docs.langchain.com/langsmith/cli”
   • [community] “I read the article but have yet to understand why someone would want to use a framework that introduces meaningless abstractions that are no…”
   • [claimed-docs] “LangGraph models agent workflows as graphs. You define the behavior of your agents using three key components: State, Nodes, Edges.”
   • [claimed-docs] “LangGraph graphs expose the stream (sync) and astream (async) methods to yield streamed outputs as iterators.”

   proof ↗⚑ flag

   Claude Agent SDK✓fullcommunity9/10

   The Agent SDK is extensively documented as a public, programmable API (Python/TypeScript) covering tools, hooks, subagents, MCP, permissions, sessions, streaming, structured outputs, and hosting, with a bundled CLI and migration guides from other agent SDKs. Community evidence corroborates real hands-on use of the API (headless workflows via `claude -p`) even amid unrelated billing-policy friction. Missing for 10: independent third-party technical review confirming API stability/versioning guarantees beyond first-party docs.

   • [claimed-docs] “The Agent SDK gives you the same tools, agent loop, and context management that power Claude Code, programmable in Python and TypeScript.”
   • [claimed-docs] “Hooks are callback functions that run your code in response to agent events, like a tool being called, a session starting, or execution stop…”
   • [claimed-docs] “Subagents are separate agent instances that your main agent can spawn to handle focused subtasks. Use them to isolate context, run multiple …”
   • [claimed-docs] “With MCP, your agent can query databases, integrate with APIs like Slack and GitHub, and connect to other services without writing custom to…”
   • [claimed-docs] “Custom tools extend the Agent SDK by letting you define your own functions that Claude can call during a conversation.”
   • [claimed-docs] “Deploy the Agent SDK in production: subprocess architecture, session persistence, scaling, observability, and multi-tenant isolation for Doc…”
   • [claimed-docs] “Use ClaudeSDKClient for interactive applications such as chat interfaces, or when the next action depends on Claude's response.”
   • [claimed-docs] “Migrating from the OpenAI Agents SDK instead? The OpenAI Agents SDK migration recipe maps each primitive onto the Claude Agent SDK through a…”
   • [github] “The Claude Code CLI is automatically bundled with the package - no separate installation required! The SDK will use the bundled CLI by defau…”
   • [community] “Damn. I just built an entire headless automated workflow around `claude -p`”

   proof ↗⚑ flag

6. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round to Claude Agent SDK
   LangGraph—none0/10

   No evidence in the pack addresses issuing scoped or least-privilege API credentials for an agent; LangGraph's docs cover orchestration, persistence, streaming, memory, and deployment but nothing about credential scoping or permission-limited API keys.

   ⚑ flag

   Claude Agent SDK~partialclaimed4/10

   The SDK documents permission controls (permission modes, rules, and a canUseTool callback) that let developers restrict which tools/actions an agent can perform, which is a form of least-privilege control over agent behavior, but there is no evidence of a mechanism for issuing scoped/least-privilege API credentials or keys (e.g., limited-scope tokens for external API access) as distinct from tool-use permissioning. Missing for 10: explicit scoped-API-key/credential issuance feature, credential rotation/expiry controls, and any documentation tying permission modes to external API credential scoping.

   • [claimed-docs] “The Claude Agent SDK provides permission controls to manage how Claude uses tools. Use permission modes and rules to define what's allowed a…”
   • [claimed-docs] “Pass a canUseTool callback in your query options. The callback fires whenever Claude needs user input, receiving the tool name and input as …”
   • [claimed-docs] “The Claude Agent SDK provides permission controls to manage how Claude uses tools.”

   proof ↗⚑ flag

7. ai-native userBuild against official SDKs

   weight 2 · round drawn
   LangGraph✓fullprobed8/10

   LangGraph itself is shipped as an official, well-documented SDK/package (pip install langgraph) with extensive first-party API docs (graph API, persistence, streaming, interrupts), an official CLI/Agent Server, and GitHub-hosted source, all confirming it is a legitimate SDK for building AI-native agent systems. Missing for 10: evidence of official SDKs beyond Python (e.g., JS/TS parity claims) and independent hands-on validation of SDK API stability (community notes mention breaking changes/documentation gaps).

   • [github] “pip install -U langgraph”
   • [github] “LangGraph is a low-level orchestration framework for building, managing, and deploying long-running, stateful agents.”
   • [claimed-docs] “At its core, LangGraph models agent workflows as graphs. You define the behavior of your agents using three key components”
   • [claimed-docs] “LangGraph models agent workflows as graphs. You define the behavior of your agents using three key components: State, Nodes, Edges.”
   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally. The resulting server exposes all API endpoints for r…”
   • [probe] “official CLI documented at https://docs.langchain.com/langsmith/cli”
   • [community] “I read the article but have yet to understand why someone would want to use a framework that introduces meaningless abstractions that are no…”

   proof ↗⚑ flag

   Claude Agent SDK✓fullcommunity8/10

   Claude Agent SDK is itself the official first-party SDK (Python and TypeScript), with extensive documented APIs for tools, hooks, subagents, MCP, permissions, sessions, streaming, structured outputs, and production hosting, plus a GitHub package that bundles the CLI. This directly satisfies 'build against official SDKs' for an AI-native developer persona. Missing for 10: independent/hands-on corroboration of SDK ergonomics beyond vendor docs, and some community reports note DX friction/observability gaps with the default harness (not outright contradicting the capability but tempering the polish).

   • [claimed-docs] “The Agent SDK gives you the same tools, agent loop, and context management that power Claude Code, programmable in Python and TypeScript.”
   • [claimed-docs] “Built-in tools | Read, write, edit files, run commands, and search the web”
   • [claimed-docs] “Custom tools extend the Agent SDK by letting you define your own functions that Claude can call during a conversation.”
   • [claimed-docs] “Use ClaudeSDKClient for interactive applications such as chat interfaces, or when the next action depends on Claude's response.”
   • [github] “The Claude Code CLI is automatically bundled with the package - no separate installation required! The SDK will use the bundled CLI by defau…”
   • [claimed-docs] “Migrating from the OpenAI Agents SDK instead? The OpenAI Agents SDK migration recipe maps each primitive onto the Claude Agent SDK through a…”
   • [community] “I've been using claude -p with a harness called codelayer. Their [default] harness is completely garbage when it comes at displaying full de…”

   proof ↗⚑ flag

8. ai-native userSubscribe to events via webhooks

   weight 2 · round drawn
   LangGraph—none0/10

   Evidence covers streaming, checkpointing, interrupts, and an Agent Server exposing API endpoints, but nowhere mentions webhook subscriptions or push-based event notifications for external systems. missing for 10: any documentation of a webhook registration/subscription mechanism, delivery guarantees, or event-push API.

   ⚑ flag

   Claude Agent SDK—none0/10

   The evidence pack describes hooks, streaming input/output, sessions, and callbacks (canUseTool) but nowhere mentions webhook subscriptions or an outbound HTTP event notification mechanism for external systems to subscribe to agent events.

   ⚑ flag

Agentic features

1. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to LangGraph
   LangGraph~partialcommunity7/10

   LangGraph explicitly supports durable, long-running agent execution that persists through failures and resumes automatically, with checkpointing, human-in-the-loop interrupts, and a CLI/Agent Server for production deployment (langgraph-gh-6, langgraph-docs-13, langgraph-docs-30). This covers the core of 'autonomous background automation' but it is a low-level orchestration framework requiring developers to build and deploy the graph themselves rather than a turnkey scheduler/trigger system, and community feedback notes rough edges in streaming/persistence implementation (langgraph-comm-13). Missing for 10: built-in scheduling/trigger mechanisms for kicking off automations, independent hands-on verification of unattended long-running runs, and clearer distinction of 'autonomous' (no human) vs human-in-the-loop operation.

   • [github] “Durable execution — Build agents that persist through failures and can run for extended periods, automatically resuming from exactly where t…”
   • [github] “Production-ready deployment — Deploy sophisticated agent systems confidently with scalable infrastructure designed to handle the unique chal…”
   • [claimed-docs] “In production, use a checkpointer backed by a database: from langgraph.checkpoint.postgres import PostgresSaver”
   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally. The resulting server exposes all API endpoints for r…”
   • [claimed-docs] “Checkpointing keeps your place: the checkpointer writes the exact graph state so you can resume later, even when in an error state.”
   • [community] “The one thing I wish was better developed is persistence and streaming - they give sample code to stream, but it's essentially a complete im…”

   proof ↗⚑ flag

   Claude Agent SDK~partialcommunity6/10

   The SDK supports headless/programmatic operation (streaming input, subprocess architecture, session persistence, hooks, custom tools) that enable building autonomous background automations, and community evidence confirms real users built 'headless automated workflows' with it. However, there's no dedicated scheduling/trigger mechanism for background automation, and community feedback raises concerns about restricted usage, observability gaps, and policy uncertainty for such non-interactive uses. Missing for 10: built-in scheduling/cron or trigger-based automation features, clear documentation of long-running unattended background execution, and resolution of the community-reported usage restrictions/observability complaints for headless workflows.

   • [claimed-docs] “Deploy the Agent SDK in production: subprocess architecture, session persistence, scaling, observability, and multi-tenant isolation for Doc…”
   • [claimed-docs] “The Agent SDK spawns and supervises a claude CLI subprocess that owns a shell, a working directory, and session files on disk.”
   • [claimed-docs] “Streaming input mode is the preferred way to use the Claude Agent SDK. It provides full access to the agent's capabilities and enables rich,…”
   • [claimed-docs] “A session is the conversation history the SDK accumulates while your agent works. ... The SDK writes it to disk automatically so you can ret…”
   • [community] “Damn. I just built an entire headless automated workflow around `claude -p`”
   • [community] “There is no difference between claude -p and typing into their 'harness' in tmux. They want to make you pay for API which is subsidizing the…”
   • [community] “My frustration with Anthropic has been around the uncertainty of all of this. What constitutes fair usage, what can I play with and not risk…”

   proof ↗⚑ flag

2. ai-native userOperate the product with natural-language commands

   weight 2 · round to Claude Agent SDK
   LangGraph—none0/10

   LangGraph is a code-first orchestration framework (graphs, nodes, edges, checkpointers, CLI for running a server) with no evidence of any natural-language command interface for operating the product itself — developers configure and run it via Python/JS APIs and CLI flags, not NL prompts. Missing for 10: any documented chat/NL interface, NL-driven graph builder, or NL-based CLI/administration capability.

   • [claimed-docs] “At its core, LangGraph models agent workflows as graphs. You define the behavior of your agents using three key components”
   • [claimed-docs] “LangGraph models agent workflows as graphs. You define the behavior of your agents using three key components: State, Nodes, Edges.”
   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally.”
   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally. The resulting server exposes all API endpoints for r…”

   proof ↗⚑ flag

   Claude Agent SDK✓fullclaimed8/10

   The SDK is fundamentally natural-language driven: it's built on Claude Code's agent loop where users issue natural-language prompts/queries and the agent autonomously invokes tools, subagents, and permission flows in response (docs-1, docs-8, docs-9, docs-13, docs-26). Streaming interactive mode and the bundled CLI (claude -p) further confirm operation via conversational natural-language input rather than rigid commands. missing for 10: no independent hands-on demonstration of a non-technical user driving it purely via natural language without code/config, and community evidence focuses on licensing/harness complaints rather than confirming NL usability quality.

   • [claimed-docs] “The Agent SDK gives you the same tools, agent loop, and context management that power Claude Code, programmable in Python and TypeScript.”
   • [claimed-docs] “Custom tools extend the Agent SDK by letting you define your own functions that Claude can call during a conversation.”
   • [claimed-docs] “Pass a canUseTool callback in your query options. The callback fires whenever Claude needs user input, receiving the tool name and input as …”
   • [claimed-docs] “Use ClaudeSDKClient for interactive applications such as chat interfaces, or when the next action depends on Claude's response.”
   • [claimed-docs] “It allows the agent to operate as a long lived process that takes in user input, handles interruptions, surfaces permission requests, and ha…”
   • [github] “The Claude Code CLI is automatically bundled with the package - no separate installation required! The SDK will use the bundled CLI by defau…”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   LangGraph—none0/10

   The evidence pack shows conventional markdown documentation and a probe confirming no OpenAPI/interactive API spec is published (all candidate paths 404). There is no mention of an interactive API reference or runnable examples/playground anywhere in the docs or GitHub materials.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.langchain.com/openapi.json, https://docs.langchain.com/swagger.json, https://docs.langc…”

   proof ↗⚑ flag

   Claude Agent SDK—none0/10

   The evidence pack shows extensive markdown-based documentation (overview, hooks, subagents, permissions, sessions, etc.) and llms.txt-style text docs, but nothing indicates an interactive API reference with runnable/embedded code examples (e.g., a browser-based sandbox or live code runner) — docs appear to be static reference pages only.

   ⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   LangGraph—none0/10

   While LangGraph's Agent Server is documented as exposing REST API endpoints for runs, threads, and assistants (langgraph-docs-30/37), no evidence shows a downloadable machine-readable spec (OpenAPI/Swagger) — a direct probe for openapi.json, swagger.json, and related paths returned 404 on all candidates (langgraph-probe-3). missing for 10: a documented OpenAPI/Swagger endpoint or downloadable spec file, any doc page referencing 'openapi' or 'swagger' for the Agent Server, confirmation from the actual running server rather than just the docs site.

   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally. The resulting server exposes all API endpoints for r…”
   • [claimed-docs] “LangGraph CLI** is a command-line tool for building and running the [Agent Server](/langsmith/agent-server) locally. The resulting server ex…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.langchain.com/openapi.json, https://docs.langchain.com/swagger.json, https://docs.langc…”

   proof ↗⚑ flag

   Claude Agent SDK—none0/10

   No evidence of an OpenAPI or other machine-readable API spec being published for the Agent SDK; docs mention llms.txt indexes and Markdown docs but not a formal machine-readable API spec download.

   ⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round to LangGraph
   LangGraph~partialclaimed4/10

   LangGraph docs show a local/dev path (LangGraph CLI running the Agent Server locally, in-memory or dev checkpointers) distinct from a production database-backed checkpointer, which implies a way to iterate locally without touching production data, but there is no explicit 'sandbox environment' or test-data-isolation feature documented. missing for 10: no dedicated sandbox/staging environment concept, no explicit guidance on isolating test data from production, no independent confirmation that local runs are safely isolated from production stores.

   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally. The resulting server exposes all API endpoints for r…”
   • [claimed-docs] “LangGraph CLI** is a command-line tool for building and running the [Agent Server](/langsmith/agent-server) locally. The resulting server ex…”
   • [claimed-docs] “In production, use a checkpointer backed by a database: from langgraph.checkpoint.postgres import PostgresSaver”
   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally.”

   proof ↗⚑ flag

   Claude Agent SDK~partialclaimed3/10

   Docs mention deploying the SDK in production with 'sandbox providers' and multi-tenant isolation (Docker/Kubernetes), plus permission/hook controls to block dangerous operations, which could support building a sandboxed test setup, but there is no explicit feature or guidance for testing against a sandbox without touching production data. Missing for 10: dedicated sandbox/test-mode documentation, explicit production-data isolation guarantees, and hands-on evidence of safe test usage.

   • [claimed-docs] “Deploy the Agent SDK in production: subprocess architecture, session persistence, scaling, observability, and multi-tenant isolation for Doc…”
   • [claimed-docs] “With hooks, you can: Block dangerous operations before they execute, like destructive shell commands or unauthorized file access”
   • [claimed-docs] “The Claude Agent SDK provides permission controls to manage how Claude uses tools. Use permission modes and rules to define what's allowed a…”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   LangGraph—none0/10

   The evidence pack contains no documentation of API versioning scheme or a formal deprecation policy for LangGraph's APIs; only general framework descriptions and a community complaint that the framework 'often introduces breaking changes' without being well documented, which is unrelated to any specific versioning/deprecation guarantee. This is an applicable axis for a developer framework/API, but no supporting evidence exists.

   • [community] “I read the article but have yet to understand why someone would want to use a framework that introduces meaningless abstractions that are no…”

   proof ↗⚑ flag

   Claude Agent SDK—none0/10

   The evidence pack documents SDK features (tools, hooks, sessions, permissions) and a migration guide from a competing SDK, but contains no mention of API versioning, version numbers, a changelog, or a documented deprecation policy for the Agent SDK itself. Community threads discuss usage/billing policy shifts, which are off-topic to API versioning and deprecation guarantees.

   ⚑ flag

Agent authoring

1. developerDefine an agent with typed custom tools in a few lines of code

   weight 3 · round to Claude Agent SDK
   LangGraph~partialcommunity4/10

   Evidence confirms LangGraph nodes/tools can be freely custom-coded (comm-10) and that tool integration exists via MCPAdapter (docs-22), implying developers can define custom tools, but the pack lacks any concrete code example showing typed tool definitions or a 'few lines of code' walkthrough for tool creation. Missing for 10: a documented tool-definition API/decorator with type hints, a minimal code snippet, and independent confirmation of ease-of-use for typed tools.

   • [community] “In langgraph nodes are just functions that can do whatever you want... you don't have to use langchain tools or ToolNode with langgraph, you…”
   • [claimed-docs] “LangChain agents call tools defined on MCP servers through MCPAdapter, which discovers a server's tools and adapts them into LangChain tools…”
   • [claimed-docs] “At its core, LangGraph models agent workflows as graphs. You define the behavior of your agents using three key components”
   • [claimed-docs] “LangGraph models agent workflows as graphs. You define the behavior of your agents using three key components: State, Nodes, Edges.”

   proof ↗⚑ flag

   Claude Agent SDK✓fullclaimed7/10

   Docs explicitly describe custom-tools support letting developers define their own functions Claude can call, with permission controls and callback hooks around tool use, indicating a lightweight typed-tool definition workflow. missing for 10: no concrete code snippet showing the 'few lines of code' typed tool definition, no independent/hands-on confirmation of ergonomics or type-safety guarantees.

   • [claimed-docs] “Custom tools extend the Agent SDK by letting you define your own functions that Claude can call during a conversation.”
   • [claimed-docs] “Pass a canUseTool callback in your query options. The callback fires whenever Claude needs user input, receiving the tool name and input as …”
   • [claimed-docs] “The Claude Agent SDK provides permission controls to manage how Claude uses tools. Use permission modes and rules to define what's allowed a…”
   • [claimed-docs] “The Claude Agent SDK provides permission controls to manage how Claude uses tools.”

   proof ↗⚑ flag

Ai buildability

1. ai-native userHave a coding agent scaffold a new agent project from an official CLI or template in one command

   weight 2 · round to LangGraph
   LangGraph~partialprobed4/10

   An official LangGraph CLI is documented (installable, used to build/run the Agent Server locally), which is the kind of official tool a scaffold command would live in, but the evidence never shows a specific one-command project/template scaffolding action (e.g., `langgraph new`) — only server build/run functionality is described. missing for 10: explicit scaffold/template command documentation, a first-command quickstart example, independent confirmation it works as a one-command project generator.

   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally.”
   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally. The resulting server exposes all API endpoints for r…”
   • [claimed-docs] “LangGraph CLI** is a command-line tool for building and running the [Agent Server](/langsmith/agent-server) locally. The resulting server ex…”
   • [probe] “official CLI documented at https://docs.langchain.com/langsmith/cli”

   proof ↗⚑ flag

   Claude Agent SDK—none0/10

   The evidence describes SDK features (tools, hooks, sessions, subagents) and a quickstart guide for building an agent, plus a bundled CLI, but nothing documents a single-command scaffold/template generator (e.g., an 'init' or 'create-agent' command) for starting a new agent project.

   ⚑ flag

2. ai-native userRun the framework's example agents headlessly from a terminal so an agent can verify what it just built

   weight 2 · round to Claude Agent SDK
   LangGraph~partialprobed4/10

   LangGraph ships a CLI that runs an Agent Server locally and graphs expose sync/async invoke and stream methods that can be called headlessly from a terminal or script, which technically enables scripted verification runs. However, there is no evidence of a curated set of 'example agents' meant for headless self-verification, nor any documented workflow where an agent inspects its own build via terminal output. Missing for 10: dedicated example-agent scripts/quickstarts, explicit headless verification/testing workflow, and any first-party or community confirmation that agents use this for self-check.

   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally.”
   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally. The resulting server exposes all API endpoints for r…”
   • [claimed-docs] “LangGraph CLI** is a command-line tool for building and running the [Agent Server](/langsmith/agent-server) locally. The resulting server ex…”
   • [claimed-docs] “LangGraph graphs expose the stream (sync) and astream (async) methods to yield streamed outputs as iterators.”
   • [probe] “official CLI documented at https://docs.langchain.com/langsmith/cli”

   proof ↗⚑ flag

   Claude Agent SDK~partialcommunity5/10

   The SDK bundles the claude CLI and supports headless/non-interactive use (claude -p, subprocess architecture, streaming vs single-shot modes) confirmed by both docs and community usage of `claude -p` in automated workflows, which supports the general capability of running agents headlessly from a terminal. However, there is no evidence of packaged 'example agents' shipped with the framework meant specifically for self-verification of what an agent just built, and community reports describe the harness as hacky with poor observability. missing for 10: documented example-agent repo/templates runnable headlessly, an explicit self-verification workflow, and hands-on confirmation the examples work smoothly out of the box.

   • [claimed-docs] “The Agent SDK spawns and supervises a claude CLI subprocess that owns a shell, a working directory, and session files on disk.”
   • [claimed-docs] “Streaming input mode is the preferred way to use the Claude Agent SDK. It provides full access to the agent's capabilities and enables rich,…”
   • [github] “The Claude Code CLI is automatically bundled with the package - no separate installation required! The SDK will use the bundled CLI by defau…”
   • [community] “Damn. I just built an entire headless automated workflow around `claude -p`”
   • [community] “I've been using claude -p with a harness called codelayer. Their [default] harness is completely garbage when it comes at displaying full de…”
   • [claimed-docs] “Use the Agent SDK to build an AI agent that reads your code, finds bugs, and fixes them, all without manual intervention.”

   proof ↗⚑ flag

3. ai-native userRely on strict typing and schema validation so a coding agent catches its own mistakes at build time

   weight 2 · round to Claude Agent SDK
   LangGraph~partialclaimed3/10

   The only relevant evidence is a passing mention that the graph builder performs 'basic checks on the structure of your graph (no orphaned nodes, etc.)' at compile time, which is a thin form of build-time validation but not strict typing or schema validation of agent outputs/tools. No evidence describes typed state schemas, Pydantic/TypedDict validation, or static type-checking catching agent mistakes. missing for 10: explicit schema/type validation for node inputs-outputs, evidence of build-time type errors being caught, independent confirmation of this behavior in practice.

   • [claimed-docs] “It provides a few basic checks on the structure of your graph (no orphaned nodes, etc). It is also where you can specify runtime args like c…”

   proof ↗⚑ flag

   Claude Agent SDK~partialclaimed4/10

   The SDK offers 'structured outputs' where you define a schema and get back validated JSON matching it, which is the closest evidence to schema validation, but this is runtime validation of agent output, not compile/build-time type checking of the agent's own reasoning or code as the story implies. There's no evidence of static type-checking integration, IDE-time error catching, or build-time validation gates for agent actions. missing for 10: explicit build-time/compile-time type-checking tooling, evidence of the agent catching its own mistakes before execution (not just output shape), IDE/linter integration for agent-authored code.

   • [claimed-docs] “Structured outputs let you define the exact shape of data you want back from an agent. ... you still get validated JSON matching your schema…”
   • [claimed-docs] “Structured outputs let you define the exact shape of data you want back from an agent... you still get validated JSON matching your schema a…”
   • [claimed-docs] “Structured outputs let you define the exact shape of data you want back from an agent.”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Claude Agent SDK
   LangGraph—none0/10

   The evidence pack describes graph orchestration, streaming, checkpointing, memory, and human-in-the-loop features, and mentions internal parallel execution within a single graph (Pregel/BSP model), but there is no documentation of a bulk/batch API for invoking the graph across many independent items or records at once (e.g., a .batch()/.abatch() method or bulk import/export tooling). Missing for 10: explicit batch invocation API, bulk data import/export tooling, or evidence of processing many independent items concurrently as a first-class feature.

   • [community] “LangGraph implements a variant of the Pregel/BSP algorithm for orchestrating workflows with cycles (ie. not DAGs) and parallelism without da…”
   • [claimed-docs] “LangGraph graphs expose the stream (sync) and astream (async) methods to yield streamed outputs as iterators.”
   • [claimed-docs] “It exposes graph execution through stream modes such as updates, values, messages, custom, checkpoints, tasks, and debug.”

   proof ↗⚑ flag

   Claude Agent SDK~partialcommunity5/10

   The SDK supports programmatic automation (custom tools, subagents for parallel focused subtasks, headless/scriptable operation via claude -p, structured outputs) which can be composed to perform bulk operations across many items, but there is no explicit documentation of a bulk-operation primitive (e.g., batch processing many files/records with progress tracking, rate limiting, or a dedicated batch API). missing for 10: explicit bulk/batch operation APIs or examples, evidence of handling large item counts reliably, and independent confirmation of bulk-scale performance.

   • [claimed-docs] “Subagents are separate agent instances that your main agent can spawn to handle focused subtasks. Use them to isolate context, run multiple …”
   • [claimed-docs] “Custom tools extend the Agent SDK by letting you define your own functions that Claude can call during a conversation.”
   • [claimed-docs] “Subagents are separate agent instances that your main agent can spawn to handle focused subtasks.”
   • [community] “Damn. I just built an entire headless automated workflow around `claude -p`”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to Claude Agent SDK
   LangGraph~partialclaimed4/10

   LangGraph's graph model (State, Nodes, Edges) and interrupts allow conditional routing and pausing at specific points, which can act like internal rules driving actions as state changes, and its Agent Server exposes API endpoints for runs/threads that could be invoked on external events. However, the evidence pack contains no explicit documentation of an event-trigger system (e.g., webhooks, schedules, external event listeners, conditional-edge rule definitions) that automatically fires actions outside of manually invoked graph runs. Missing for 10: explicit conditional-edge/rule syntax, documented external event triggers (webhook/cron), and evidence of automatic action firing without a user-initiated run.

   • [claimed-docs] “At its core, LangGraph models agent workflows as graphs. You define the behavior of your agents using three key components”
   • [claimed-docs] “By composing Nodes and Edges, you can create complex, looping workflows that evolve the state over time.”
   • [claimed-docs] “LangGraph models agent workflows as graphs. You define the behavior of your agents using three key components: State, Nodes, Edges.”
   • [claimed-docs] “Interrupts allow you to pause graph execution at specific points and wait for external input before continuing.”
   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally. The resulting server exposes all API endpoints for r…”

   proof ↗⚑ flag

   Claude Agent SDK✓fullclaimed7/10

   Hooks are explicitly documented as callback functions that run custom code in response to agent events (tool calls, session start, execution stop), enabling automatic actions like blocking dangerous operations before execution. Permission modes/rules further let users define what's allowed automatically. Missing for 10: independent/hands-on corroboration of hooks in real automation workflows, and more detail on the full range of triggerable event types/rule complexity.

   • [claimed-docs] “Hooks are callback functions that run your code in response to agent events, like a tool being called, a session starting, or execution stop…”
   • [claimed-docs] “With hooks, you can: Block dangerous operations before they execute, like destructive shell commands or unauthorized file access”
   • [claimed-docs] “The Claude Agent SDK provides permission controls to manage how Claude uses tools. Use permission modes and rules to define what's allowed a…”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round drawn
   LangGraph—none0/10

   No evidence in the pack mentions cron-style scheduling, recurring triggers, or time-based/periodic job execution; LangGraph's docs focus on persistence, checkpointing, interrupts, streaming, and durable execution but not scheduled/recurring workflow invocation.

   ⚑ flag

   Claude Agent SDK—none0/10

   The evidence covers sessions, subagents, hooks, MCP, permissions, and hosting, but nowhere mentions a scheduler, cron-like trigger, or built-in mechanism for recurring/automated job execution—developers would need to build their own external scheduling infrastructure around the SDK. missing for 10: any built-in scheduling/cron primitive, recurring-trigger API, or workflow-automation feature for periodic execution.

   • [claimed-docs] “A session is the conversation history the SDK accumulates while your agent works. ... The SDK writes it to disk automatically so you can ret…”
   • [claimed-docs] “Deploy the Agent SDK in production: subprocess architecture, session persistence, scaling, observability, and multi-tenant isolation for Doc…”
   • [claimed-docs] “It allows the agent to operate as a long lived process that takes in user input, handles interruptions, surfaces permission requests, and ha…”

   proof ↗⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round to LangGraph
   LangGraph~partialclaimed5/10

   LangGraph's checkpointer/time-travel and interrupt features let developers pause for human review and resume or roll back to prior graph states, and LangSmith tracing gives visibility into execution paths, covering 'review' and partial 'rollback'. However there's no evidence of an explicit versioning system for automations (e.g., named/versioned assistant deployments, diffing or rollback UI) beyond code-level state checkpoints. missing for 10: explicit automation/version management (e.g., versioned assistants/deployments), a UI for reviewing/rolling back workflow versions, independent hands-on confirmation of rollback working in production.

   • [claimed-docs] “Checkpointers persist a thread's graph state as checkpoints. Use them for short-term, thread-scoped memory, including conversation continuit…”
   • [claimed-docs] “Checkpointing keeps your place: the checkpointer writes the exact graph state so you can resume later, even when in an error state.”
   • [github] “Seamlessly incorporate human oversight by inspecting and modifying agent state at any point during execution.”
   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally. The resulting server exposes all API endpoints for r…”
   • [claimed-docs] “Trace and compare these workflow patterns with LangSmith... Follow the tracing quickstart to see how data flows through each step.”

   proof ↗⚑ flag

   Claude Agent SDK—none0/10

   The SDK documents session persistence, forking, and resuming conversation history (docs-7, docs-21, docs-24), but this is conversation/session state, not version control, review, or rollback of the automations/agent definitions themselves. There is no evidence of a versioning system, change review/approval workflow, or rollback mechanism for the automations a user builds with the SDK.

   ⚑ flag

Deployment

1. engineering-leadDeploy an agent to a managed runtime and call it as an API endpoint

   weight 2 · round drawn
   LangGraph~partialcommunity6/10

   Docs confirm a LangGraph CLI/Agent Server that exposes API endpoints for runs, threads, and assistants with managed checkpointing/storage (langgraph-docs-30/37/23), and GitHub claims 'production-ready deployment' with scalable infrastructure for stateful agents (langgraph-gh-9). However, a community engineer explicitly asks how to deploy LangGraph as a production API beyond 'langgraph serve' locally, suggesting the managed/production deployment path is not fully clear from hands-on experience (langgraph-comm-7). Missing for 10: independent hands-on confirmation of a hosted managed cloud runtime (vs. local CLI server), and details on production SLAs/scaling beyond marketing claims.

   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally. The resulting server exposes all API endpoints for r…”
   • [claimed-docs] “LangGraph CLI** is a command-line tool for building and running the [Agent Server](/langsmith/agent-server) locally. The resulting server ex…”
   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally.”
   • [github] “Production-ready deployment — Deploy sophisticated agent systems confidently with scalable infrastructure designed to handle the unique chal…”
   • [community] “How can one deploy LangGraph as an API (with production like features)? I have worked with langgraph serve to deploy locally, but are there …”

   proof ↗⚑ flag

   Claude Agent SDK~partialclaimed6/10

   Docs explicitly cover production deployment concerns—subprocess architecture, session persistence, scaling, observability, and multi-tenant isolation across Docker, Kubernetes, and sandbox providers—giving an engineering lead a path to run the SDK as a backend service. However, this is self-hosting guidance, not a first-party managed runtime; the SDK still spawns a local CLI subprocess and there's no evidence of Anthropic providing a hosted 'deploy as endpoint' service, and the developer must build the API wrapper themselves. Missing for 10: a true managed-runtime/PaaS offering, first-party API-endpoint scaffolding, and independent confirmation of production deployments at scale.

   • [claimed-docs] “Deploy the Agent SDK in production: subprocess architecture, session persistence, scaling, observability, and multi-tenant isolation for Doc…”
   • [claimed-docs] “The Agent SDK spawns and supervises a claude CLI subprocess that owns a shell, a working directory, and session files on disk.”
   • [claimed-docs] “A `SessionStore` adapter lets you mirror those transcripts to your own backend, such as an object store, a key-value store, or a database, s…”

   proof ↗⚑ flag

2. engineering-leadRun my agents entirely on my own infrastructure with no dependence on the vendor's platform

   weight 2 · round to LangGraph
   LangGraph✓fullcommunity7/10

   LangGraph is open-source, pip-installable, and includes a CLI to build/run the Agent Server locally with self-managed checkpointing via Postgres or other backends, meaning agents can run fully on self-hosted infra without the vendor's managed platform. However, evidence pack emphasizes LangSmith for tracing/debugging and doesn't explicitly discuss self-hosting at scale or full platform parity without LangSmith. missing for 10: independent verification of large-scale self-hosted production deployments, explicit statement that all deployment features (e.g., cron/scheduling, multi-tenant auth) work without LangSmith/LangGraph Platform, and clearer separation of open-source vs paid-platform features.

   • [github] “pip install -U langgraph”
   • [claimed-docs] “In production, use a checkpointer backed by a database: from langgraph.checkpoint.postgres import PostgresSaver”
   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally.”
   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally. The resulting server exposes all API endpoints for r…”
   • [claimed-docs] “LangGraph CLI** is a command-line tool for building and running the [Agent Server](/langsmith/agent-server) locally. The resulting server ex…”
   • [community] “How can one deploy LangGraph as an API (with production like features)? I have worked with langgraph serve to deploy locally, but are there …”

   proof ↗⚑ flag

   Claude Agent SDK!disputedcontradicted4/10

   Docs claim you can self-host the SDK's orchestration layer (Docker/Kubernetes/subprocess architecture, session persistence, multi-tenant isolation) [claude-agent-sdk-docs-11, claude-agent-sdk-docs-20], but the SDK still requires the bundled Claude CLI and Anthropic's model API to function, and community evidence documents Anthropic tightening platform-level control over how the SDK can be used—restricting subscription usage, changing what's 'allowed' month to month, and creating uncertainty about bans/quotas [claude-agent-sdk-comm-1, claude-agent-sdk-comm-4, claude-agent-sdk-comm-9, claude-agent-sdk-comm-11]. This shows that despite infra-level self-hosting options, engineering leads remain functionally dependent on Anthropic's policies and API access, directly undercutting the 'no dependence on vendor's platform' claim. Missing for 10: evidence of a fully vendor-independent model backend or offline/self-hosted inference option, and confirmation that policy changes don't affect self-hosted deployments.

   • [claimed-docs] “Deploy the Agent SDK in production: subprocess architecture, session persistence, scaling, observability, and multi-tenant isolation for Doc…”
   • [claimed-docs] “The Agent SDK spawns and supervises a claude CLI subprocess that owns a shell, a working directory, and session files on disk.”
   • [community] “You can no longer use your Claude subscription with apps that use the Claude Agent SDK, e.g. claude -p, Conductor, T3code, etc.”
   • [community] “Currently the credit doesn't apply to interactive Claude Code, web/desktop/mobile apps, Claude Cowork, etc. Next month, interactive claude c…”
   • [community] “My frustration with Anthropic has been around the uncertainty of all of this. What constitutes fair usage, what can I play with and not risk…”
   • [community] “This is big, but until we have policy clarity we can't trust it. I've always migrated all of our agents to Pi SDK, we aren't going back.”

   proof ↗⚑ flag

Portability

1. developerSwap the underlying LLM provider or model without rewriting my agent

   weight 3 · round drawn
   LangGraph—none0/10

   The evidence pack contains no documentation or examples showing that LangGraph nodes use a provider-agnostic model interface (e.g., a single call that can swap between OpenAI, Anthropic, etc. without code changes); it focuses on graph structure, checkpointing, streaming, and human-in-the-loop features, not model abstraction. A stray community comment about 'bring your own keys' apps is too thin and non-technical to establish this capability. missing for 10: any docs on a unified chat-model interface, model-swap examples, or provider abstraction demonstrating no-rewrite portability.

   • [community] “The use case where they are helpful is 'bring your own keys' apps... The abstraction is very much worth it for me. That said: I migrated fro…”

   proof ↗⚑ flag

   Claude Agent SDK—none0/10

   The evidence shows the Agent SDK is architected specifically around Anthropic's Claude models — it bundles and spawns a 'claude CLI subprocess' and is built to power Claude Code — with no mention of an abstraction layer for swapping in other LLM providers/models. The only related item is a migration guide *from* the OpenAI Agents SDK *to* this SDK (docs-33), which is a one-way onboarding path, not evidence of provider-agnostic model swapping within the SDK itself.

   • [github] “The Claude Code CLI is automatically bundled with the package - no separate installation required! The SDK will use the bundled CLI by defau…”
   • [claimed-docs] “The Agent SDK spawns and supervises a claude CLI subprocess that owns a shell, a working directory, and session files on disk.”
   • [claimed-docs] “Migrating from the OpenAI Agents SDK instead? The OpenAI Agents SDK migration recipe maps each primitive onto the Claude Agent SDK through a…”

   proof ↗⚑ flag

Evals

1. engineering-leadScore agent quality with built-in evals and run them as part of CI

   weight 2 · round drawn
   LangGraph—none0/10

   The evidence pack shows LangGraph provides tracing/visualization via LangSmith and debugging tools, but contains no mention of built-in evals, scoring agent quality, or running evals as part of CI. Evals appear to be a separate LangSmith capability not documented here.

   ⚑ flag

   Claude Agent SDK—none0/10

   No evidence pack item mentions evals, scoring frameworks, benchmarks, or CI integration for agent quality; documentation covers tools, hooks, sessions, permissions, structured outputs, and hosting but nothing about built-in evaluation or CI test harnesses.

   ⚑ flag

Testing

1. developerUnit-test agents with mocked models and tools

   weight 2 · round drawn
   LangGraph—none0/10

   No evidence pack item documents unit-testing patterns, mocking of models/tools, or a testing framework/utilities for LangGraph agents; the closest is a community remark that nodes are plain functions you can implement however you like, which only implies testability rather than demonstrating it.

   • [community] “In langgraph nodes are just functions that can do whatever you want... you don't have to use langchain tools or ToolNode with langgraph, you…”

   proof ↗⚑ flag

   Claude Agent SDK—none0/10

   The evidence pack documents custom tools, hooks, permissions, sessions, and structured outputs, but there is no mention of a testing framework, mock model/tool harness, or any guidance for unit-testing agents with mocked dependencies.

   ⚑ flag

Tracing

1. developerTrace every LLM call and tool invocation of an agent run in an observability UI

   weight 3 · round to LangGraph
   LangGraph✓fullcommunity8/10

   LangGraph integrates with LangSmith to provide tracing and debugging UI that visualizes execution paths, captures state transitions, and provides runtime metrics for agent runs, with docs explicitly directing users to trace and compare workflow patterns via the tracing quickstart. missing for 10: no independent/hands-on confirmation of trace fidelity for LLM calls and tool invocations specifically, and one community comment notes streaming/observability implementation is left partly to the client.

   • [github] “Debugging with LangSmith — Gain deep visibility into complex agent behavior with visualization tools that trace execution paths, capture sta…”
   • [github] “Gain deep visibility into complex agent behavior with visualization tools that trace execution paths, capture state transitions, and provide…”
   • [claimed-docs] “Trace and compare these workflow patterns with LangSmith... Follow the tracing quickstart to see how data flows through each step.”
   • [claimed-docs] “It exposes graph execution through stream modes such as updates, values, messages, custom, checkpoints, tasks, and debug.”
   • [community] “The one thing I wish was better developed is persistence and streaming - they give sample code to stream, but it's essentially a complete im…”

   proof ↗⚑ flag

   Claude Agent SDK!disputedcontradicted4/10

   Docs mention 'observability' as a topic covered in production hosting guidance and hooks/streaming provide raw events for tool calls and messages, but there's no dedicated tracing/observability UI documented (e.g., no trace viewer, no integration with an eval/observability platform). Community hands-on reports directly contradict any claim of built-in observability, describing the default harness as having '0 observability' and failing to display full tool-call/session details in its UI. missing for 10: a dedicated observability/tracing UI or integration, documented trace export (OpenTelemetry/etc.), and evidence resolving the community complaints about poor tool-call visibility.

   • [claimed-docs] “Deploy the Agent SDK in production: subprocess architecture, session persistence, scaling, observability, and multi-tenant isolation for Doc…”
   • [claimed-docs] “Hooks are callback functions that run your code in response to agent events, like a tool being called, a session starting, or execution stop…”
   • [community] “There is no difference between claude -p and typing into their 'harness' in tmux. They want to make you pay for API which is subsidizing the…”
   • [community] “I've been using claude -p with a harness called codelayer. Their [default] harness is completely garbage when it comes at displaying full de…”

   proof ↗⚑ flag

Guardrails

1. developerAttach input/output guardrails that validate, transform, or block unsafe content

   weight 3 · round to Claude Agent SDK
   LangGraph—none0/10

   The evidence describes LangGraph's general graph/node architecture, persistence, interrupts, and human-in-the-loop features, but nothing documents a guardrails feature (input/output validation, content moderation, or blocking unsafe content). While nodes are flexible functions (allowing a developer to hand-roll such logic), there is no first-party guardrails API, validator, or moderation integration cited.

   • [community] “In langgraph nodes are just functions that can do whatever you want... you don't have to use langchain tools or ToolNode with langgraph, you…”
   • [claimed-docs] “At its core, LangGraph models agent workflows as graphs. You define the behavior of your agents using three key components”
   • [claimed-docs] “By composing Nodes and Edges, you can create complex, looping workflows that evolve the state over time.”

   proof ↗⚑ flag

   Claude Agent SDK~partialclaimed6/10

   Hooks provide a mechanism to intercept tool calls and block dangerous operations before execution, and canUseTool/permissions callbacks allow runtime validation/blocking of tool use, which together approximate input/output guardrails. However, there is no dedicated 'guardrails' API for validating or transforming model output content itself (e.g., content moderation, output rewriting) beyond structured-output schema validation. missing for 10: explicit output-content validation/transformation guardrail API, first-party examples of blocking/altering unsafe generated text (not just tool calls), independent verification of guardrail robustness.

   • [claimed-docs] “Hooks are callback functions that run your code in response to agent events, like a tool being called, a session starting, or execution stop…”
   • [claimed-docs] “With hooks, you can: Block dangerous operations before they execute, like destructive shell commands or unauthorized file access”
   • [claimed-docs] “The Claude Agent SDK provides permission controls to manage how Claude uses tools. Use permission modes and rules to define what's allowed a…”
   • [claimed-docs] “Pass a canUseTool callback in your query options. The callback fires whenever Claude needs user input, receiving the tool name and input as …”
   • [claimed-docs] “Structured outputs let you define the exact shape of data you want back from an agent. ... you still get validated JSON matching your schema…”

   proof ↗⚑ flag

2. engineering-leadRestrict what an agent may do with fine-grained tool permissions and sandboxed execution

   weight 2 · round to Claude Agent SDK
   LangGraph—none0/10

   The evidence pack documents human-in-the-loop interrupts, checkpointing, and custom node logic (langgraph-docs-4, langgraph-gh-2), but nowhere describes fine-grained per-tool permission scoping or sandboxed/isolated execution environments for agent actions. Community notes even mention nodes/tools are 'whatever you want' custom code (langgraph-comm-10), implying no built-in permissioning or sandbox layer is provided by the framework itself.

   • [claimed-docs] “Interrupts allow you to pause graph execution at specific points and wait for external input before continuing.”
   • [github] “Seamlessly incorporate human oversight by inspecting and modifying agent state at any point during execution.”
   • [github] “Human-in-the-loop — Seamlessly incorporate human oversight by inspecting and modifying agent state at any point during execution.”
   • [community] “In langgraph nodes are just functions that can do whatever you want... you don't have to use langchain tools or ToolNode with langgraph, you…”
   • [community] “Almost none of those things are part of the LangGraph framework? LangGraph does the scheduling, checkpointing, state management, etc. All of…”

   proof ↗⚑ flag

   Claude Agent SDK✓fullclaimed8/10

   Docs describe granular permission modes, rules, and a canUseTool runtime callback (permissions.md), hooks that can block dangerous operations before execution (hooks.md), and hosting guidance covering multi-tenant isolation via Docker, Kubernetes, and sandbox providers (hosting.md) — directly matching fine-grained tool control plus sandboxed execution. Missing for 10: independent/hands-on verification that sandbox isolation holds up in production and more detail on the exact rule syntax for per-tool allow/deny policies.

   • [claimed-docs] “The Claude Agent SDK provides permission controls to manage how Claude uses tools. Use permission modes and rules to define what's allowed a…”
   • [claimed-docs] “Pass a canUseTool callback in your query options. The callback fires whenever Claude needs user input, receiving the tool name and input as …”
   • [claimed-docs] “With hooks, you can: Block dangerous operations before they execute, like destructive shell commands or unauthorized file access”
   • [claimed-docs] “Deploy the Agent SDK in production: subprocess architecture, session persistence, scaling, observability, and multi-tenant isolation for Doc…”
   • [claimed-docs] “The Agent SDK spawns and supervises a claude CLI subprocess that owns a shell, a working directory, and session files on disk.”
   • [claimed-docs] “The Claude Agent SDK provides permission controls to manage how Claude uses tools.”

   proof ↗⚑ flag

Approval flows

1. developerPause an agent mid-run for human input or approval and resume with the human's decision

   weight 3 · round to LangGraph
   LangGraph✓fullclaimed9/10

   LangGraph has a dedicated interrupts feature explicitly designed to pause graph execution and wait for external input, with resumption via re-invoking the graph with a Command object carrying the human's decision; this is backed by checkpointer-based persistence for durability across pauses, and GitHub docs explicitly list 'Human-in-the-loop' as a core capability allowing inspection/modification of agent state mid-execution. Missing for 10: independent hands-on developer account specifically validating the interrupt/resume workflow (community evidence discusses persistence/streaming generally but not this exact HITL pause-resume flow).

   • [claimed-docs] “Interrupts allow you to pause graph execution at specific points and wait for external input before continuing.”
   • [claimed-docs] “you resume execution by re-invoking the graph using Command, which then becomes the return value of the interrupt() call from inside the nod…”
   • [claimed-docs] “when you're ready to continue, you resume execution by re-invoking the graph using Command, which then becomes the return value of the inter…”
   • [claimed-docs] “Checkpointing keeps your place: the checkpointer writes the exact graph state so you can resume later, even when in an error state.”
   • [claimed-docs] “Checkpointers persist a thread's graph state as checkpoints. Use them for short-term, thread-scoped memory, including conversation continuit…”
   • [github] “Seamlessly incorporate human oversight by inspecting and modifying agent state at any point during execution.”
   • [github] “Human-in-the-loop — Seamlessly incorporate human oversight by inspecting and modifying agent state at any point during execution.”

   proof ↗⚑ flag

   Claude Agent SDK✓fullclaimed8/10

   The SDK explicitly supports pausing for human approval via the canUseTool callback and AskUserQuestion tool, which fires whenever Claude needs user input, and lets developers surface approval requests/clarifying questions and return the human's decision back to the SDK to resume execution. Permission modes/rules and hooks further allow blocking operations pending human input, and streaming mode supports long-lived interactive sessions that handle interruptions and permission requests. missing for 10: no independent/hands-on corroboration of the pause-resume UX in practice, and no explicit example showing resumption after a long delay or across process restarts specifically for approval workflows.

   • [claimed-docs] “Pass a canUseTool callback in your query options. The callback fires whenever Claude needs user input, receiving the tool name and input as …”
   • [claimed-docs] “Claude requests user input in two situations: when it needs permission to use a tool (like deleting files or running commands), and when it …”
   • [claimed-docs] “Surface Claude's approval requests and clarifying questions to users, then return their decisions to the SDK.”
   • [claimed-docs] “The Claude Agent SDK provides permission controls to manage how Claude uses tools. Use permission modes and rules to define what's allowed a…”
   • [claimed-docs] “It allows the agent to operate as a long lived process that takes in user input, handles interruptions, surfaces permission requests, and ha…”
   • [claimed-docs] “With hooks, you can: Block dangerous operations before they execute, like destructive shell commands or unauthorized file access”

   proof ↗⚑ flag

2. engineering-leadRequire human approval before specific sensitive tool calls execute

   weight 2 · round drawn
   LangGraph✓fullcommunity8/10

   LangGraph's interrupt() mechanism explicitly lets a graph pause execution at any node (e.g., a node calling a sensitive tool) and wait for external input, resuming only via Command re-invocation — this is the standard pattern for gating tool calls on human approval, and checkpointers back this with durable state. GitHub feature list and docs independently confirm 'Human-in-the-loop — seamlessly incorporate human oversight... at any point during execution,' and community commentary (HN) corroborates LangGraph as providing 'a state machine framework for human in the loop.' missing for 10: a first-party worked example specifically gating a tool-call node (vs. generic interrupt points), and independent hands-on validation of the approval-before-tool-call pattern.

   • [claimed-docs] “Interrupts allow you to pause graph execution at specific points and wait for external input before continuing.”
   • [claimed-docs] “you resume execution by re-invoking the graph using Command, which then becomes the return value of the interrupt() call from inside the nod…”
   • [claimed-docs] “when you're ready to continue, you resume execution by re-invoking the graph using Command, which then becomes the return value of the inter…”
   • [claimed-docs] “Checkpointing keeps your place: the checkpointer writes the exact graph state so you can resume later, even when in an error state.”
   • [github] “Seamlessly incorporate human oversight by inspecting and modifying agent state at any point during execution.”
   • [github] “Human-in-the-loop — Seamlessly incorporate human oversight by inspecting and modifying agent state at any point during execution.”
   • [community] “I think the main thing LangGraph adds is a state machine framework for human in the loop with time travel... you won't have to make your own…”

   proof ↗⚑ flag

   Claude Agent SDK✓fullclaimed8/10

   The SDK provides a documented canUseTool callback that fires when Claude needs permission for a sensitive tool call, allowing engineering leads to intercept and require approval before execution, plus hooks that can block operations before they run and permission modes/rules for fine-grained control. missing for 10: independent/hands-on corroboration of the approval flow in production and more detail on configuring which specific tools trigger approval vs. auto-allow.

   • [claimed-docs] “The Claude Agent SDK provides permission controls to manage how Claude uses tools. Use permission modes and rules to define what's allowed a…”
   • [claimed-docs] “Pass a canUseTool callback in your query options. The callback fires whenever Claude needs user input, receiving the tool name and input as …”
   • [claimed-docs] “Claude requests user input in two situations: when it needs permission to use a tool (like deleting files or running commands), and when it …”
   • [claimed-docs] “With hooks, you can: Block dangerous operations before they execute, like destructive shell commands or unauthorized file access”
   • [claimed-docs] “Surface Claude's approval requests and clarifying questions to users, then return their decisions to the SDK.”

   proof ↗⚑ flag

Memory

1. developerTrim, summarize, or filter conversation history to keep an agent inside its context window

   weight 2 · round to Claude Agent SDK
   LangGraph—none0/10

   Evidence describes LangGraph's persistence/checkpointing and short-term vs long-term memory model, but nothing in the pack documents specific mechanisms to trim, summarize, or filter conversation history to manage context window size. Missing for 10: any mention of message trimming utilities, summarization nodes/chains, or history-filtering APIs, and independent confirmation these features work as intended.

   • [claimed-docs] “Short-term memory (thread-level persistence) enables agents to track multi-turn conversations.”
   • [claimed-docs] “Add short-term memory as a part of your agent's state to enable multi-turn conversations.”
   • [claimed-docs] “Short-term memory (thread-level persistence) enables agents to track multi-turn conversations. To add short-term memory:”
   • [claimed-docs] “Checkpointers persist a thread's graph state as checkpoints. Use them for short-term, thread-scoped memory, including conversation continuit…”
   • [claimed-docs] “Stores persist application-defined data outside the graph state. Use them for long-term, cross-thread memory, including user preferences, fa…”

   proof ↗⚑ flag

   Claude Agent SDK~partialclaimed4/10

   Docs mention the SDK provides the same 'context management' as Claude Code and that subagents can be used to isolate context for subtasks, implying some context-window management exists, but no explicit API for trimming, summarizing, or filtering conversation history is documented. Missing for 10: explicit compaction/summarization API, documented context-window truncation controls, and independent confirmation that developers can programmatically filter history.

   • [claimed-docs] “The Agent SDK gives you the same tools, agent loop, and context management that power Claude Code, programmable in Python and TypeScript.”
   • [claimed-docs] “Subagents are separate agent instances that your main agent can spawn to handle focused subtasks. Use them to isolate context, run multiple …”
   • [claimed-docs] “A session is the conversation history the SDK accumulates while your agent works. ... The SDK writes it to disk automatically so you can ret…”
   • [claimed-docs] “Fork is different: it creates a new session that starts with a copy of the original's history. The original stays”

   proof ↗⚑ flag

2. developerGive agents long-term memory that persists across sessions and threads

   weight 2 · round to LangGraph
   LangGraph✓fullcommunity8/10

   LangGraph documents a dedicated Store abstraction explicitly for 'long-term, cross-thread memory' (user preferences, facts, shared knowledge) separate from thread-scoped checkpointers, with guidance to back it with production databases (e.g., Postgres) and docs explicitly stating 'Add long-term memory to store user-specific or application-level data across sessions.' GitHub README also markets 'long-term persistent memory across sessions' as a core feature. Missing for 10: independent/hands-on corroboration of cross-thread memory at scale — one community comment vaguely notes persistence 'could be better developed,' but this is not a concrete failure report.

   • [claimed-docs] “Stores persist application-defined data outside the graph state. Use them for long-term, cross-thread memory, including user preferences, fa…”
   • [claimed-docs] “In production, use a checkpointer backed by a database: from langgraph.checkpoint.postgres import PostgresSaver”
   • [claimed-docs] “Add long-term memory to store user-specific or application-level data across sessions.”
   • [github] “Comprehensive memory — Create truly stateful agents with both short-term working memory for ongoing reasoning and long-term persistent memor…”
   • [community] “The one thing I wish was better developed is persistence and streaming - they give sample code to stream, but it's essentially a complete im…”

   proof ↗⚑ flag

   Claude Agent SDK~partialclaimed6/10

   Sessions persist conversation history to disk and can be resumed with full prior context (docs-7, docs-16, docs-24), and SessionStore lets you mirror transcripts to external backends for cross-host resumption (docs-29). However this is session/thread-level persistence rather than true cross-session long-term memory (e.g. semantic memory, facts recalled across unrelated threads); CLAUDE.md/rules provide some persistent instructions but not dynamic memory. missing for 10: dedicated long-term/semantic memory store distinct from raw transcript replay, cross-thread memory retrieval mechanism, independent verification of persistence working reliably in production.

   • [claimed-docs] “A session is the conversation history the SDK accumulates while your agent works. ... The SDK writes it to disk automatically so you can ret…”
   • [claimed-docs] “A session is the conversation history the SDK accumulates while your agent works... The SDK writes it to disk automatically so you can retur…”
   • [claimed-docs] “Returning to a session means the agent has full context from before: files it already read, analysis it already performed, decisions it alre…”
   • [claimed-docs] “A `SessionStore` adapter lets you mirror those transcripts to your own backend, such as an object store, a key-value store, or a database, s…”
   • [claimed-docs] “The Agent SDK is built on the same foundation as Claude Code, which means your SDK agents have access to the same filesystem-based features:…”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Claude Agent SDK
   LangGraph~partialprobed4/10

   The LangGraph CLI/Agent Server exposes API endpoints for runs, threads, assistants, etc., suggesting programmatic access mirrors what LangGraph Studio UI shows, but there's no explicit documentation confirming full feature parity between the Studio UI and the API. missing for 10: explicit parity documentation, a public OpenAPI spec (probe found only 404s), and hands-on confirmation that every UI action (e.g., time-travel, breakpoints, state edits) is scriptable via API.

   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally. The resulting server exposes all API endpoints for r…”
   • [claimed-docs] “LangGraph CLI** is a command-line tool for building and running the [Agent Server](/langsmith/agent-server) locally. The resulting server ex…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.langchain.com/openapi.json, https://docs.langchain.com/swagger.json, https://docs.langc…”
   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally.”

   proof ↗⚑ flag

   Claude Agent SDK~partialcommunity7/10

   Docs explicitly claim the SDK exposes 'the same tools, agent loop, and context management that power Claude Code' and shares 'the same foundation... CLAUDE.md, skills, hooks' as the CLI/UI, suggesting strong feature parity between API and interactive UI. However, community reports describe the default programmatic harness as having 'poor DX,' 'zero observability,' and being 'garbage' for surfacing tool-call detail compared to interactive use, and note new subscription-usage restrictions on SDK-driven apps, indicating parity in practice is imperfect. Missing for 10: independent verification that every UI-only feature (e.g. full interactive session UX, all slash-commands) is reachable via API, and resolution of the observability/extensibility gaps raised by developers.

   • [claimed-docs] “The Agent SDK gives you the same tools, agent loop, and context management that power Claude Code, programmable in Python and TypeScript.”
   • [claimed-docs] “The Agent SDK is built on the same foundation as Claude Code, which means your SDK agents have access to the same filesystem-based features:…”
   • [community] “There is no difference between claude -p and typing into their 'harness' in tmux. They want to make you pay for API which is subsidizing the…”
   • [community] “I've been using claude -p with a harness called codelayer. Their [default] harness is completely garbage when it comes at displaying full de…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round to Claude Agent SDK
   LangGraph~partialclaimed4/10

   LangGraph is open-source and self-hosted, and its persistence layer explicitly supports standard, user-controlled databases (e.g., PostgresSaver) rather than a proprietary hosted store, giving users inherent access to their own state/checkpoint data. However, there is no explicit documentation of an export feature, data-format guarantees, or a supported 'leave with your data' workflow beyond the fact that storage backends are pluggable/open. Missing for 10: documented export/import tooling, explicit open-format (e.g., JSON/CSV) data dumps, and any first-party or community confirmation of a clean migration/export path.

   • [claimed-docs] “In production, use a checkpointer backed by a database: from langgraph.checkpoint.postgres import PostgresSaver”
   • [claimed-docs] “Stores persist application-defined data outside the graph state. Use them for long-term, cross-thread memory, including user preferences, fa…”
   • [claimed-docs] “Checkpointers persist a thread's graph state as checkpoints. Use them for short-term, thread-scoped memory, including conversation continuit…”
   • [github] “LangGraph is a low-level orchestration framework for building, managing, and deploying long-running, stateful agents.”

   proof ↗⚑ flag

   Claude Agent SDK~partialclaimed5/10

   Session transcripts are written to disk automatically and a SessionStore adapter lets developers mirror those transcripts to their own backend (object store, KV store, database), giving users control over their conversation data and the ability to move it elsewhere. However, there's no explicit documentation of a full 'export all data' feature, no stated open/standard file format for transcripts, and no mention of exporting other data types (configs, custom tools, permissions settings). missing for 10: documented open export format for session data, a full account/data export feature, evidence covering all data types beyond session transcripts.

   • [claimed-docs] “A session is the conversation history the SDK accumulates while your agent works. ... The SDK writes it to disk automatically so you can ret…”
   • [claimed-docs] “A `SessionStore` adapter lets you mirror those transcripts to your own backend, such as an object store, a key-value store, or a database, s…”
   • [claimed-docs] “The Agent SDK spawns and supervises a claude CLI subprocess that owns a shell, a working directory, and session files on disk.”

   proof ↗⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round to LangGraph
   LangGraph~partialclaimed5/10

   Evidence confirms LangGraph's source is publicly hosted on GitHub (langchain-ai/langgraph) with install instructions and repo links, implying the code is readable, but no evidence pack item explicitly states or cites an open-source license (e.g., MIT/Apache) for the repo. missing for 10: explicit license file/citation, confirmation of license terms, any independent verification of licensing terms.

   • [github] “pip install -U langgraph”
   • [github] “LangGraph is a low-level orchestration framework for building, managing, and deploying long-running, stateful agents.”
   • [github] “Low-level orchestration framework for building stateful agents.”

   proof ↗⚑ flag

   Claude Agent SDK—none0/10

   There is no evidence the Claude Agent SDK's source is released under an open license; the Python/TypeScript SDK wraps a proprietary bundled Claude Code CLI and no license/open-source repo details are provided beyond a GitHub package listing. Community evidence even discusses restrictive usage terms, further indicating a closed, controlled distribution model rather than open-source code.

   • [github] “The Claude Code CLI is automatically bundled with the package - no separate installation required! The SDK will use the bundled CLI by defau…”
   • [community] “You can no longer use your Claude subscription with apps that use the Claude Agent SDK, e.g. claude -p, Conductor, T3code, etc.”
   • [community] “There is no difference between claude -p and typing into their 'harness' in tmux. They want to make you pay for API which is subsidizing the…”

   proof ↗⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round to LangGraph
   LangGraph✓fullclaimed8/10

   LangGraph core is a pip-installable open-source library (langgraph-gh-5) with a CLI to build and run the Agent Server locally (langgraph-docs-23, langgraph-docs-30, langgraph-docs-37), and supports production-grade self-hosted persistence via PostgresSaver (langgraph-docs-13), confirming a fully self-hostable core product outside any managed SaaS. missing for 10: explicit license/self-hosting infra docs (scaling, containerization) and independent hands-on confirmation of self-hosting beyond CLI docs.

   • [github] “pip install -U langgraph”
   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally.”
   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally. The resulting server exposes all API endpoints for r…”
   • [claimed-docs] “LangGraph CLI** is a command-line tool for building and running the [Agent Server](/langsmith/agent-server) locally. The resulting server ex…”
   • [claimed-docs] “In production, use a checkpointer backed by a database: from langgraph.checkpoint.postgres import PostgresSaver”

   proof ↗⚑ flag

   Claude Agent SDK—none0/10

   The SDK's 'hosting' docs only describe deploying the wrapper application (Docker/Kubernetes/subprocess supervision) — the core product itself, the Claude model, remains a hosted Anthropic service accessed via subscription/API, and community evidence confirms usage is gated by Anthropic's cloud plans and subject to policy restrictions, not something a user can run independently. No evidence describes self-hosting Claude's weights or inference outside Anthropic's infrastructure.

   • [claimed-docs] “Deploy the Agent SDK in production: subprocess architecture, session persistence, scaling, observability, and multi-tenant isolation for Doc…”
   • [claimed-docs] “The Agent SDK spawns and supervises a claude CLI subprocess that owns a shell, a working directory, and session files on disk.”
   • [community] “You can no longer use your Claude subscription with apps that use the Claude Agent SDK, e.g. claude -p, Conductor, T3code, etc.”
   • [community] “Currently the credit doesn't apply to interactive Claude Code, web/desktop/mobile apps, Claude Cowork, etc. Next month, interactive claude c…”
   • [community] “My frustration with Anthropic has been around the uncertainty of all of this. What constitutes fair usage, what can I play with and not risk…”

   proof ↗⚑ flag

Multi agent

1. developerOrchestrate multiple agents — handoffs, subagents, or crews — inside one workflow

   weight 3 · round drawn
   LangGraph✓fullcommunity8/10

   LangGraph explicitly documents multi-agent orchestration patterns (handoffs, subagents/crews) via its graph-api and multi-agent docs, letting developers embed agent patterns as nodes, mix deterministic/agentic steps, and use Command/interrupts for handoffs, all within one stateful graph with persistence and streaming. Community evidence corroborates it as a legitimate stateful orchestration engine (not just a wrapper) supporting cycles/parallelism. Missing for 10: no hands-on demonstration of a specific named multi-agent 'crew' example or independent benchmark of handoff reliability at scale.

   • [claimed-docs] “Build bespoke execution flows with LangGraph, mixing deterministic logic and agentic behavior. Embed other patterns as nodes in your workflo…”
   • [claimed-docs] “Custom workflow: Build bespoke execution flows with LangGraph, mixing deterministic logic and agentic behavior. Embed other patterns as node…”
   • [claimed-docs] “Custom workflow — Build bespoke execution flows with LangGraph, mixing deterministic logic and agentic behavior. Embed other patterns as nod…”
   • [claimed-docs] “Here are the main patterns for building multi-agent systems, each suited to different use cases”
   • [claimed-docs] “At its core, LangGraph models agent workflows as graphs. You define the behavior of your agents using three key components”
   • [claimed-docs] “LangGraph models agent workflows as graphs. You define the behavior of your agents using three key components: State, Nodes, Edges.”
   • [claimed-docs] “Interrupts allow you to pause graph execution at specific points and wait for external input before continuing.”
   • [claimed-docs] “when you're ready to continue, you resume execution by re-invoking the graph using Command, which then becomes the return value of the inter…”
   • [community] “LangGraph implements a variant of the Pregel/BSP algorithm for orchestrating workflows with cycles (ie. not DAGs) and parallelism without da…”
   • [community] “LangGraph is different. It is a legitimate piece of workflow software and not a wrapper framework. Now, when it comes to workflow there are …”

   proof ↗⚑ flag

   Claude Agent SDK✓fullclaimed8/10

   Docs explicitly cover subagents spawned by a main agent for parallel/focused subtasks, hooks to coordinate/intercept events, session forking to branch workflows, and MCP integration — together supporting multi-agent orchestration within one workflow. Missing for 10: independent hands-on validation of complex multi-agent crews/handoffs beyond first-party docs, and no explicit named 'handoff' primitive comparable to other agent frameworks.

   • [claimed-docs] “Subagents are separate agent instances that your main agent can spawn to handle focused subtasks. Use them to isolate context, run multiple …”
   • [claimed-docs] “Subagents are separate agent instances that your main agent can spawn to handle focused subtasks.”
   • [claimed-docs] “Hooks are callback functions that run your code in response to agent events, like a tool being called, a session starting, or execution stop…”
   • [claimed-docs] “With hooks, you can: Block dangerous operations before they execute, like destructive shell commands or unauthorized file access”
   • [claimed-docs] “Fork is different: it creates a new session that starts with a copy of the original's history. The original stays”
   • [claimed-docs] “With MCP, your agent can query databases, integrate with APIs like Slack and GitHub, and connect to other services without writing custom to…”

   proof ↗⚑ flag

Workflow control

1. developerCompose agents into an explicit graph or workflow with branching, loops, and parallel steps

   weight 2 · round to LangGraph
   LangGraph✓fullcommunity9/10

   LangGraph's core model is explicitly graph-based (State, Nodes, Edges) with support for loops/cycles, branching, and parallelism via its Pregel/BSP execution model, confirmed both by docs and independent community technical commentary. missing for 10: no first-party hands-on benchmark of parallel-branch execution at scale, and community notes some friction with built-in parallelism complicating debugging.

   • [claimed-docs] “By composing Nodes and Edges, you can create complex, looping workflows that evolve the state over time.”
   • [claimed-docs] “LangGraph models agent workflows as graphs. You define the behavior of your agents using three key components: State, Nodes, Edges.”
   • [claimed-docs] “Workflows have predetermined code paths and are designed to operate in a certain order.”
   • [community] “LangGraph implements a variant of the Pregel/BSP algorithm for orchestrating workflows with cycles (ie. not DAGs) and parallelism without da…”
   • [community] “by predeclaring the structure, you can show debugging UI of the full graph, even if you've only executed part of it... The downside is that …”
   • [community] “Hot take #1: For experienced developers, framework abstractions can add unnecessary complexity. Hot take #2: Built-in parallelism, while pro…”
   • [claimed-docs] “At its core, LangGraph models agent workflows as graphs. You define the behavior of your agents using three key components”

   proof ↗⚑ flag

   Claude Agent SDK~partialclaimed4/10

   The SDK supports spawning subagents for parallel subtasks and gives developers full Python/TypeScript control flow (which can express branching/loops in code), but there is no documented explicit graph/workflow abstraction (nodes, edges, conditional branches, loop constructs) as a first-class SDK feature. Missing for 10: a dedicated workflow/graph API, documented branching/loop primitives, and independent examples of complex multi-step orchestration graphs beyond simple subagent spawning.

   • [claimed-docs] “Subagents are separate agent instances that your main agent can spawn to handle focused subtasks. Use them to isolate context, run multiple …”
   • [claimed-docs] “Subagents are separate agent instances that your main agent can spawn to handle focused subtasks.”
   • [claimed-docs] “The Agent SDK gives you the same tools, agent loop, and context management that power Claude Code, programmable in Python and TypeScript.”
   • [claimed-docs] “Use ClaudeSDKClient for interactive applications such as chat interfaces, or when the next action depends on Claude's response.”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round drawn
   LangGraph—none0/10

   The evidence pack describes LangGraph's checkpointing/persistence architecture (Postgres checkpointer, Agent Server with managed database) but contains no documentation of region selection, data residency controls, or geographic deployment options for stored data. Since LangGraph offers a hosted Agent Server/deployment platform, region/residency is a fair question, but nothing in the pack addresses it.

   • [claimed-docs] “In production, use a checkpointer backed by a database: from langgraph.checkpoint.postgres import PostgresSaver”
   • [claimed-docs] “LangGraph CLI is a command-line tool for building and running the Agent Server locally. The resulting server exposes all API endpoints for r…”
   • [claimed-docs] “LangGraph CLI** is a command-line tool for building and running the [Agent Server](/langsmith/agent-server) locally. The resulting server ex…”

   proof ↗⚑ flag

   Claude Agent SDK—none0/10

   No evidence in the pack addresses data residency, regional storage selection, or compliance controls for where data/sessions are stored; docs mention local disk session storage and a SessionStore adapter but nothing about choosing a geographic region.

   ⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   LangGraph—none0/10

   The evidence pack contains no documentation, policy, or statement about data usage, model training opt-outs, or privacy controls for LangGraph or its hosted offerings (LangSmith, LangGraph Platform). Since LangGraph does offer hosted/managed services where such a policy would be relevant, the axis applies but is entirely unaddressed.

   ⚑ flag

   Claude Agent SDK—none0/10

   No evidence in the pack addresses training-data opt-out, data-retention controls, or privacy settings for the Agent SDK; documentation covers tools, sessions, permissions, and hosting but nothing about preventing data use for model training.

   ⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round drawn
   LangGraph—none0/10

   The evidence describes checkpointers and stores that persist conversation state and long-term memory (e.g., via Postgres), but nothing in the pack documents any deletion API, TTL/retention policy, or user-facing control to purge stored threads/state. As a self-hosted framework the user technically owns the database, but no LangGraph-specific retention/deletion mechanism is evidenced.

   • [claimed-docs] “Checkpointers persist a thread's graph state as checkpoints. Use them for short-term, thread-scoped memory, including conversation continuit…”
   • [claimed-docs] “Stores persist application-defined data outside the graph state. Use them for long-term, cross-thread memory, including user preferences, fa…”
   • [claimed-docs] “In production, use a checkpointer backed by a database: from langgraph.checkpoint.postgres import PostgresSaver”
   • [claimed-docs] “Checkpointers persist a thread's graph state as checkpoints. Use them for short-term, thread-scoped memory... Stores persist application-def…”

   proof ↗⚑ flag

   Claude Agent SDK—none0/10

   The evidence describes session persistence and a SessionStore adapter for mirroring transcripts to a user's own backend, but nothing addresses data retention policies, deletion controls, or how long Anthropic/the SDK retains data. missing for 10: any documentation on data retention periods, explicit deletion APIs/commands, or privacy/compliance controls governing stored conversation or tool-use data.

   • [claimed-docs] “A session is the conversation history the SDK accumulates while your agent works. ... The SDK writes it to disk automatically so you can ret…”
   • [claimed-docs] “A `SessionStore` adapter lets you mirror those transcripts to your own backend, such as an object store, a key-value store, or a database, s…”

   proof ↗⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   LangGraph—none0/10

   No evidence in the pack addresses telemetry, usage tracking, or an opt-out mechanism for LangGraph itself; the docs focus on orchestration, memory, streaming, and deployment, and LangSmith tracing is presented as an opt-in observability feature rather than a telemetry opt-out control.

   ⚑ flag

   Claude Agent SDK—none0/10

   No evidence in the pack mentions telemetry, usage tracking, opt-out settings, or privacy controls for the Agent SDK; the docs cover tools, hooks, sessions, permissions, and hosting but never data-collection controls.

   ⚑ flag

Durable state

1. developerCheckpoint agent state so a run can resume exactly where it left off after a crash or restart

   weight 3 · round to LangGraph
   LangGraph✓fullclaimed9/10

   LangGraph's checkpointer system explicitly persists exact graph state per thread, enabling resume after error/crash ('checkpointing keeps your place... even when in an error state'), with production-grade backends like PostgresSaver documented and GitHub README explicitly touting 'durable execution' that resumes 'exactly where they left off' after failures. This is a well-documented, core feature with clear technical backing across multiple doc pages; missing for 10: independent hands-on verification of crash-recovery behavior beyond docs/marketing claims.

   • [claimed-docs] “Checkpointers persist a thread's graph state as checkpoints. Use them for short-term, thread-scoped memory, including conversation continuit…”
   • [claimed-docs] “In production, use a checkpointer backed by a database: from langgraph.checkpoint.postgres import PostgresSaver”
   • [claimed-docs] “Checkpointing keeps your place: the checkpointer writes the exact graph state so you can resume later, even when in an error state.”
   • [github] “Build agents that persist through failures and can run for extended periods, automatically resuming from exactly where they left off.”
   • [github] “Durable execution — Build agents that persist through failures and can run for extended periods, automatically resuming from exactly where t…”

   proof ↗⚑ flag

   Claude Agent SDK✓fullclaimed7/10

   Docs describe sessions being written to disk automatically, resumable with full prior context, forkable, and a SessionStore adapter to persist transcripts to external backends so a session can resume on a different host — directly matching checkpoint/resume-after-crash needs, and hosting docs explicitly call out 'session persistence' as a production concern. Missing for 10: independent/hands-on confirmation that resume restores in-flight tool/subprocess state after an actual crash (not just conversation history), and any benchmark showing exact-resume fidelity.

   • [claimed-docs] “A session is the conversation history the SDK accumulates while your agent works. ... The SDK writes it to disk automatically so you can ret…”
   • [claimed-docs] “A session is the conversation history the SDK accumulates while your agent works... The SDK writes it to disk automatically so you can retur…”
   • [claimed-docs] “Returning to a session means the agent has full context from before: files it already read, analysis it already performed, decisions it alre…”
   • [claimed-docs] “A `SessionStore` adapter lets you mirror those transcripts to your own backend, such as an object store, a key-value store, or a database, s…”
   • [claimed-docs] “Fork is different: it creates a new session that starts with a copy of the original's history. The original stays”
   • [claimed-docs] “Deploy the Agent SDK in production: subprocess architecture, session persistence, scaling, observability, and multi-tenant isolation for Doc…”
   • [claimed-docs] “The Agent SDK spawns and supervises a claude CLI subprocess that owns a shell, a working directory, and session files on disk.”

   proof ↗⚑ flag

2. engineering-leadRun long-lived agents durably across process restarts and deploys, natively or via durable-execution integrations

   weight 2 · round to LangGraph
   LangGraph✓fullclaimed8/10

   LangGraph explicitly advertises 'Durable execution' as a core feature, with checkpointers (including production Postgres-backed checkpointers) that persist thread state so agents 'automatically resume from exactly where they left off' after failures, and interrupts that preserve execution state even in error conditions. This directly matches the engineering-lead's requirement for durable, restart-resilient long-running agents. Missing for 10: independent hands-on verification of actual crash/restart recovery in production, and explicit coverage of third-party durable-execution integrations (e.g., Temporal) beyond LangGraph's native mechanism.

   • [github] “Durable execution — Build agents that persist through failures and can run for extended periods, automatically resuming from exactly where t…”
   • [github] “Build agents that persist through failures and can run for extended periods, automatically resuming from exactly where they left off.”
   • [claimed-docs] “Checkpointers persist a thread's graph state as checkpoints. Use them for short-term, thread-scoped memory, including conversation continuit…”
   • [claimed-docs] “In production, use a checkpointer backed by a database: from langgraph.checkpoint.postgres import PostgresSaver”
   • [claimed-docs] “Checkpointing keeps your place: the checkpointer writes the exact graph state so you can resume later, even when in an error state.”
   • [github] “Production-ready deployment — Deploy sophisticated agent systems confidently with scalable infrastructure designed to handle the unique chal…”

   proof ↗⚑ flag

   Claude Agent SDK~partialclaimed7/10

   The SDK documents native session persistence (auto-written to disk, resumable/forkable) plus a SessionStore adapter for mirroring transcripts to external stores so sessions can resume across hosts, and hosting docs cover session persistence, scaling, and multi-tenant isolation for Docker/Kubernetes deploys — directly supporting durability across restarts and redeploys. However, there is no mention of integration with dedicated durable-execution frameworks (e.g., Temporal/Restate) and no independent/hands-on evidence validating resilience across actual crash/restart scenarios. Missing for 10: durable-execution framework integrations, independent verification of crash/restart resilience.

   • [claimed-docs] “A session is the conversation history the SDK accumulates while your agent works. ... The SDK writes it to disk automatically so you can ret…”
   • [claimed-docs] “A session is the conversation history the SDK accumulates while your agent works... The SDK writes it to disk automatically so you can retur…”
   • [claimed-docs] “Returning to a session means the agent has full context from before: files it already read, analysis it already performed, decisions it alre…”
   • [claimed-docs] “A `SessionStore` adapter lets you mirror those transcripts to your own backend, such as an object store, a key-value store, or a database, s…”
   • [claimed-docs] “Deploy the Agent SDK in production: subprocess architecture, session persistence, scaling, observability, and multi-tenant isolation for Doc…”
   • [claimed-docs] “The Agent SDK spawns and supervises a claude CLI subprocess that owns a shell, a working directory, and session files on disk.”

   proof ↗⚑ flag

Streaming

1. developerStream tokens and intermediate agent events (tool calls, steps) to my UI in real time

   weight 3 · round to Claude Agent SDK
   LangGraph~partialcommunity7/10

   LangGraph docs clearly document multiple stream modes including 'messages' for token streaming and 'updates'/'debug'/'tasks' for intermediate node/tool events, plus separate iterators per projection via stream/astream, directly supporting real-time UI streaming of tokens and agent steps. However, a community report notes the streaming implementation is minimal sample code that each client must fully reimplement, indicating real-world integration effort beyond the docs. missing for 10: independent hands-on confirmation of smooth tool-call/step event streaming in a UI, and clearer first-party UI integration examples beyond sample code.

   • [claimed-docs] “It exposes graph execution through stream modes such as updates, values, messages, custom, checkpoints, tasks, and debug.”
   • [claimed-docs] “Event streaming gives you separate iterators per projection (messages, values, subgraphs, output) so you can consume them independently”
   • [claimed-docs] “LangGraph graphs expose the stream (sync) and astream (async) methods to yield streamed outputs as iterators.”
   • [claimed-docs] “It exposes graph execution through stream modes such as `updates`, `values`, `messages`, `custom`, `checkpoints`, `tasks`, and `debug`.”
   • [claimed-docs] “Event streaming gives you separate iterators per projection (messages, values, subgraphs, output) so you can consume them independently inst…”
   • [community] “The one thing I wish was better developed is persistence and streaming - they give sample code to stream, but it's essentially a complete im…”

   proof ↗⚑ flag

   Claude Agent SDK✓fullclaimed8/10

   Docs explicitly describe streaming input mode as the preferred method for real-time interactive apps, a dedicated streaming-output guide showing how to enable token-level streaming via include_partial_messages/includePartialMessages, plus hooks and canUseTool callbacks that surface tool-call and step events during execution, directly matching the story. missing for 10: independent hands-on corroboration of real-time UI streaming quality, and no explicit end-to-end UI code example beyond flag/callback docs

   • [claimed-docs] “To enable streaming, set `include_partial_messages` (Python) or `includePartialMessages` (TypeScript) to `true` in your options.”
   • [claimed-docs] “Streaming input mode is the preferred way to use the Claude Agent SDK. It provides full access to the agent's capabilities and enables rich,…”
   • [claimed-docs] “It allows the agent to operate as a long lived process that takes in user input, handles interruptions, surfaces permission requests, and ha…”
   • [claimed-docs] “Hooks are callback functions that run your code in response to agent events, like a tool being called, a session starting, or execution stop…”
   • [claimed-docs] “Pass a canUseTool callback in your query options. The callback fires whenever Claude needs user input, receiving the tool name and input as …”
   • [claimed-docs] “Use ClaudeSDKClient for interactive applications such as chat interfaces, or when the next action depends on Claude's response.”

   proof ↗⚑ flag

Structured output

1. developerGet schema-validated structured output from an agent, with automatic retries when validation fails

   weight 3 · round to Claude Agent SDK
   LangGraph—none0/10

   The evidence pack covers persistence, streaming, human-in-the-loop, checkpointing, and multi-agent workflows, but contains no mention of structured output, schema validation, or automatic retries on validation failure for LangGraph agents.

   ⚑ flag

   Claude Agent SDK~partialclaimed6/10

   Docs explicitly describe structured outputs with schema-validated JSON returned at the end of an agent run, confirming schema validation support, but no evidence describes automatic retry behavior when validation fails. missing for 10: explicit documentation of retry/re-prompt behavior on validation failure, and any independent/hands-on confirmation of retry mechanics.

   • [claimed-docs] “Structured outputs let you define the exact shape of data you want back from an agent. ... you still get validated JSON matching your schema…”
   • [claimed-docs] “Structured outputs let you define the exact shape of data you want back from an agent... you still get validated JSON matching your schema a…”
   • [claimed-docs] “Structured outputs let you define the exact shape of data you want back from an agent.”

   proof ↗⚑ flag