OpenAI Agents SDK vs Google ADK

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to OpenAI Agents SDK
   OpenAI Agents SDK✓fullprobed8/10

   OpenAI's developer platform exposes a live llms.txt (HTTP 200) and markdown-formatted docs pages explicitly designed for agent consumption, which an agent built with the Agents SDK could be pointed at directly. missing for 10: no first-party doc or example showing the Agents SDK itself fetching/parsing llms.txt as a built-in feature, and no independent hands-on report confirming an agent successfully using it end-to-end.

   • [probe] “PROBE llms.txt: HTTP 200 at https://developers.openai.com/llms.txt # OpenAI Developers > Complete documentation hub for OpenAI API, Ads, Pl…”
   • [probe] “PROBE docs-md: HTTP 200 at https://developers.openai.com/api/docs/guides/agents.md # Agents > For the complete documentation index, see [ll…”
   • [claimed-docs] “The WebSearchTool lets an agent search the web.”
   • [claimed-docs] “The Agents Python SDK understands multiple MCP transports. This lets you reuse existing MCP servers or build your own to expose filesystem, …”

   proof ↗⚑ flag

   Google ADK!disputedcontradicted3/10

   Docs claim 'AI-aware developer resources' and coding-assistant integration (google-adk-docs-9), suggesting agent-oriented documentation exists, but direct probes for llms.txt and markdown-rendered docs both return 404 (google-adk-probe-1, google-adk-probe-2), and no OpenAPI/machine-readable spec is discoverable (google-adk-probe-3), contradicting the claim that an agent can straightforwardly consume these docs. Missing for 10: a working llms.txt or agent-readable doc endpoint, confirmation that the 'AI-aware resources' are actually machine-fetchable rather than just a marketing phrase.

   • [claimed-docs] “ADK is designed to be written by both humans and AI. Connect your favorite coding assistant to our ADK developer Skills and AI-aware develop…”
   • [probe] “PROBE llms.txt: HTTP 404 at https://google.github.io/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://google.github.io/adk-docs/get-started/.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://google.github.io/openapi.json, https://google.github.io/swagger.json, https://google.github.…”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to Google ADK
   OpenAI Agents SDK✓fullclaimed7/10

   The SDK is a pure Python/JS library invoked programmatically via Runner.run(), run_sync(), or run_streamed() with no UI dependency, making it inherently usable in scripts, servers, or CI pipelines; tracing/session features work headlessly too. Missing for 10: explicit CI/automation setup docs, example GitHub Actions/pipeline configs, or independent reports of production CI usage.

   • [claimed-docs] “You can run agents via the Runner class. You have 3 options: Runner.run(), Runner.run_sync(), Runner.run_streamed()”
   • [claimed-docs] “You have 3 options: Runner.run(), Runner.run_sync(), Runner.run_streamed()”
   • [claimed-docs] “You can run agents via the `Runner` class. You have 3 options: `Runner.run()`... `Runner.run_sync()`... `Runner.run_streamed()`”
   • [claimed-docs] “The Agents SDK includes built-in tracing, collecting a comprehensive record of events during an agent run: LLM generations, tool calls, hand…”
   • [claimed-docs] “The Agents SDK provides built-in session memory to automatically maintain conversation history across multiple agent runs, eliminating the n…”

   proof ↗⚑ flag

   Google ADK✓fullclaimed8/10

   ADK provides a CLI (`adk run`, `adk web`, `adk eval`, `adk deploy docker`) that supports headless invocation and scripted evaluation, plus containerized deployment for CI/production pipelines. missing for 10: explicit CI pipeline examples (e.g. GitHub Actions), independent third-party confirmation of headless CI usage.

   • [github] “adk run path/to/my_agent”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”
   • [github] “adk eval \ samples_for_testing/hello_world \ samples_for_testing/hello_world/hello_world_eval_set_001.evalset.json”
   • [github] “adk deploy docker --with_ui <agent-folder>”
   • [claimed-docs] “You can manually package your Agent into a container image and then run it in any environment that supports container images.”
   • [claimed-docs] “This approach involves creating individual test files, each representing a single, simple agent-model interaction (a session).”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round to OpenAI Agents SDK
   OpenAI Agents SDK✓fullclaimed9/10

   Docs explicitly state the Agents SDK understands multiple MCP transports, letting users reuse existing MCP servers or build their own to expose filesystem, HTTP, or connector-backed tools to an agent, and also supports hosted MCP as a managed tool option. This directly matches the story of plugging in MCP servers to use their tools. Missing for 10: independent/hands-on corroboration beyond first-party docs.

   • [claimed-docs] “The Agents Python SDK understands multiple MCP transports. This lets you reuse existing MCP servers or build your own to expose filesystem, …”
   • [claimed-docs] “The Agents Python SDK understands multiple MCP transports. This lets you reuse existing MCP servers or build your own”
   • [claimed-docs] “Use OpenAI-managed tools (web search, file search, code interpreter, hosted MCP, image generation)”

   proof ↗⚑ flag

   Google ADK✓fullclaimed8/10

   Docs explicitly state an ADK agent can act as an MCP client and use tools provided by external MCP servers, directly matching the story. missing for 10: independent/hands-on corroboration beyond first-party docs, and more detail on multi-server configuration or auth handling.

   • [claimed-docs] “An ADK agent can act as an MCP client and use tools provided by external MCP servers.”
   • [claimed-docs] “Exposing ADK Tools via an MCP Server: How to build an MCP server that wraps ADK tools, making them accessible to any MCP client.”
   • [claimed-docs] “How to build an MCP server that wraps ADK tools, making them accessible to any MCP client.”

   proof ↗⚑ flag

4. ai-native userUse an official CLI

   weight 2 · round to Google ADK
   OpenAI Agents SDK—none0/10

   The evidence pack is entirely about the Agents SDK library (Python/JS) — agent primitives, tools, tracing, sessions, MCP, streaming — with no mention of an official CLI tool for scaffolding, running, or managing agents; OpenAPI/CLI probes returned 404s.

   • [probe] “PROBE openapi: all candidate paths 404 (https://developers.openai.com/openapi.json, https://developers.openai.com/swagger.json, https://deve…”

   proof ↗⚑ flag

   Google ADK✓fullclaimed8/10

   ADK ships an official CLI (`adk run`, `adk web`, `adk eval`, `adk deploy docker`) documented in the GitHub repo with concrete command examples, plus docs reference an "Agents CLI" for scaffolding/build/test/deploy workflows tailored to AI-native/agentic use. Missing for 10: independent third-party hands-on review of the CLI's AI-native ergonomics beyond first-party docs/repo.

   • [github] “adk run path/to/my_agent”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”
   • [github] “adk eval \ samples_for_testing/hello_world \ samples_for_testing/hello_world/hello_world_eval_set_001.evalset.json”
   • [github] “adk deploy docker --with_ui <agent-folder>”
   • [claimed-docs] “Go from idea to coded ADK agent in minutes. Use your favorite AI-enabled developer environment to scaffold, build, test, evaluate, and deplo…”
   • [claimed-docs] “Migrate existing agents and workflows to ADK with Agents CLI.”

   proof ↗⚑ flag

5. ai-native userDrive the product through a documented public API

   weight 3 · round to OpenAI Agents SDK
   OpenAI Agents SDK✓fullprobed8/10

   The SDK is extensively documented as a Python/JS library with a public, well-documented API surface (Agents, Runner, Tools, Handoffs, Guardrails, Sessions, Tracing, MCP) that AI-native developers can drive programmatically, corroborated by GitHub docs and community usage reports. Missing for 10: a formal OpenAPI/machine-readable spec (probe found openapi.json 404) and deeper independent third-party validation of full API coverage.

   • [claimed-docs] “You can run agents via the Runner class. You have 3 options: Runner.run(), Runner.run_sync(), Runner.run_streamed()”
   • [claimed-docs] “An agent is a large language model (LLM) configured with instructions, tools, and optional runtime behavior such as handoffs, guardrails, an…”
   • [claimed-docs] “FunctionTool instances: wrap any Python function as a tool.”
   • [claimed-docs] “Handoffs allow an agent to delegate tasks to another agent. This is particularly useful in scenarios where different agents specialize in di…”
   • [claimed-docs] “The Agents SDK includes built-in tracing, collecting a comprehensive record of events during an agent run: LLM generations, tool calls, hand…”
   • [claimed-docs] “The Agents SDK provides built-in session memory to automatically maintain conversation history across multiple agent runs, eliminating the n…”
   • [github] “It is provider-agnostic, supporting the OpenAI Responses and Chat Completions APIs, as well as 100+ other LLMs.”
   • [community] “OpenAI SDK is much simpler to use and understand than langchain etc. I havent deployed really complicated agents, but I have a number of sim…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://developers.openai.com/openapi.json, https://developers.openai.com/swagger.json, https://deve…”

   proof ↗⚑ flag

   Google ADK~partialprobed6/10

   ADK is a Python framework/CLI (adk run, adk web, adk eval, adk deploy) with documented programmatic APIs for building and driving agents, plus MCP client/server support, but there is no evidence of a formal public REST/OpenAPI-style API surface — probes for openapi/swagger specs and llms.txt all 404. missing for 10: a documented public HTTP/OpenAPI API spec, independent third-party confirmation of programmatic drivability beyond first-party docs.

   • [claimed-docs] “Create your first Python ADK agent in minutes.”
   • [github] “adk run path/to/my_agent”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”
   • [github] “adk eval \ samples_for_testing/hello_world \ samples_for_testing/hello_world/hello_world_eval_set_001.evalset.json”
   • [github] “adk deploy docker --with_ui <agent-folder>”
   • [probe] “PROBE openapi: all candidate paths 404 (https://google.github.io/openapi.json, https://google.github.io/swagger.json, https://google.github.…”
   • [probe] “PROBE llms.txt: HTTP 404 at https://google.github.io/llms.txt”

   proof ↗⚑ flag

6. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round drawn
   OpenAI Agents SDK—none0/10

   No evidence of scoped or least-privilege API credential issuance for agents; the docs cover tools, handoffs, guardrails, tracing, sessions, MCP, and sandboxing but nothing about credential scoping or per-agent API key permissions. This is a plausible axis for an agent SDK (credential/permission management is a reasonable capability), but no evidence supports it. missing for 10: any mention of scoped/least-privilege credential issuance, API key scoping, or permission boundaries for agents.

   ⚑ flag

   Google ADK—none0/10

   No evidence in the pack describes issuing scoped or least-privilege API credentials/tokens for agents; the docs cover tools, MCP, workflows, deployment, and evaluation but nothing about credential scoping or permission management for agent identities.

   ⚑ flag

7. ai-native userBuild against official SDKs

   weight 2 · round drawn
   OpenAI Agents SDK✓fullcommunity9/10

   OpenAI Agents SDK is itself an official first-party SDK (Python and JS) with extensive documentation covering core primitives (agents, tools, handoffs, guardrails, tracing, sessions, streaming, MCP support, voice/realtime), and is corroborated by GitHub repo docs and community usage reports confirming real-world adoption. Missing for 10: deeper independent third-party production case studies beyond a couple of HN threads.

   • [claimed-docs] “An agent is a large language model (LLM) configured with instructions, tools, and optional runtime behavior such as handoffs, guardrails, an…”
   • [claimed-docs] “You can run agents via the Runner class. You have 3 options: Runner.run(), Runner.run_sync(), Runner.run_streamed()”
   • [claimed-docs] “The Agents SDK includes built-in tracing, collecting a comprehensive record of events during an agent run: LLM generations, tool calls, hand…”
   • [claimed-docs] “The Agents Python SDK understands multiple MCP transports. This lets you reuse existing MCP servers or build your own to expose filesystem, …”
   • [github] “It is provider-agnostic, supporting the OpenAI Responses and Chat Completions APIs, as well as 100+ other LLMs.”
   • [community] “OpenAI SDK is much simpler to use and understand than langchain etc. I havent deployed really complicated agents, but I have a number of sim…”

   proof ↗⚑ flag

   Google ADK✓fullclaimed9/10

   Google ADK is itself an official Python SDK/framework with extensive first-party documentation, code examples, CLI tooling (adk run/web/eval/deploy), and a public GitHub repo, giving AI-native developers a fully documented, official SDK to build against. Minor gap — missing for 10: independent third-party corroboration beyond vendor docs/repo, and llms.txt/OpenAPI probes returned 404s suggesting some machine-readable doc surfaces are incomplete.

   • [claimed-docs] “Create your first Python ADK agent in minutes.”
   • [claimed-docs] “Building an agent with just a model, instructions, and tools is a great place to start for most developers.”
   • [claimed-docs] “agent = Agent( name="researcher", model="gemini-flash-latest", instruction="You help users research topics thoroughly.", too…”
   • [github] “Agent Config: Build agents without code.”
   • [github] “adk run path/to/my_agent”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”
   • [claimed-docs] “ADK is designed to be written by both humans and AI. Connect your favorite coding assistant to our ADK developer Skills and AI-aware develop…”

   proof ↗⚑ flag

8. ai-native userSubscribe to events via webhooks

   weight 2 · round drawn
   OpenAI Agents SDK—none0/10

   The evidence shows in-process streaming/tracing for observing agent run events, but nothing about registering webhook URLs or subscribing to events via HTTP callbacks. No mention of webhook endpoints, event subscriptions, or push notifications anywhere in the docs or GitHub pack.

   ⚑ flag

   Google ADK—none0/10

   ADK's evidence shows only in-process callbacks/hooks for agent execution lifecycle, not an external webhook subscription mechanism; no docs mention registering webhook URLs or event push notifications. Missing for 10: any webhook registration API, outbound event delivery docs, or third-party confirmation of webhook support.

   • [claimed-docs] “Callbacks: Hook into specific events during an agent's execution lifecycle to add logging, monitoring, or custom side-effects without alteri…”

   proof ↗⚑ flag

Agentic features

1. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Google ADK
   OpenAI Agents SDK—none0/10

   The docs describe how to run agents interactively (Runner.run/run_sync/run_streamed), track sessions, and trace runs, but there is no evidence of scheduling, triggers, or a hosting/orchestration layer that lets a user set up an automation to run autonomously in the background without manual invocation. missing for 10: scheduling/cron or event-trigger support, a deployment/hosting mechanism for unattended background execution, and any documentation of persistent autonomous operation outside a developer-invoked run.

   • [claimed-docs] “You can run agents via the Runner class. You have 3 options: Runner.run(), Runner.run_sync(), Runner.run_streamed()”
   • [claimed-docs] “The Agents SDK provides built-in session memory to automatically maintain conversation history across multiple agent runs, eliminating the n…”
   • [claimed-docs] “The Agents SDK includes built-in tracing, collecting a comprehensive record of events during an agent run: LLM generations, tool calls, hand…”

   proof ↗⚑ flag

   Google ADK~partialclaimed6/10

   ADK supports deployable, auto-scaling agent runtimes (Cloud Run, GKE, Agent Runtime) and workflow orchestration with retries, state, and scheduling-like execution (fan-out/fan-in, loops), enabling agents to run unattended once deployed. However, evidence does not show explicit scheduling/triggers (e.g., cron-like autonomous kick-off) or a dedicated 'background automation' mode distinct from deployment. missing for 10: explicit trigger/schedule mechanism for autonomous background runs, independent evidence of long-running unattended operation, and confirmation of persistent background execution outside a deploy/response cycle.

   • [claimed-docs] “Agent Runtime is a fully managed auto-scaling service on Google Cloud specifically designed for deploying, managing, and scaling AI agents b…”
   • [claimed-docs] “Cloud Run is a managed auto-scaling compute platform on Google Cloud that enables you to run your agent as a container-based application.”
   • [claimed-docs] “GKE is a good option if you need more control over the deployment as well as for running Open Models.”
   • [github] “Workflow Runtime: A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan…”
   • [github] “A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan-out/fan-in, loops…”
   • [claimed-docs] “In ADK, any agent application that has more than one agent or executable Node is considered a workflow.”

   proof ↗⚑ flag

2. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round to OpenAI Agents SDK
   OpenAI Agents SDK~partialclaimed6/10

   The SDK's core primitives (Agent, Runner, Handoffs, Agents-as-tools) let a developer delegate tasks to an LLM-backed agent and even chain delegation between specialized agents, which is the closest analog to 'delegating tasks to a built-in assistant.' However, this is a developer framework rather than an end-user product with a ready-made assistant UI — the 'assistant' must be built and wired up by the developer, not delegated to out-of-the-box by an end user. Missing for 10: evidence of a ready-to-use, no-code assistant interface for non-developer end users, and independent hands-on confirmation of smooth task delegation in production.

   • [claimed-docs] “Handoffs allow an agent to delegate tasks to another agent. This is particularly useful in scenarios where different agents specialize in di…”
   • [claimed-docs] “You can run agents via the Runner class. You have 3 options: Runner.run(), Runner.run_sync(), Runner.run_streamed()”
   • [claimed-docs] “Agents as tools: expose an agent as a callable tool without a full handoff.”
   • [claimed-docs] “An agent is a large language model (LLM) configured with instructions, tools, and optional runtime behavior such as handoffs, guardrails, an…”

   proof ↗⚑ flag

   Google ADK—none0/10

   ADK is a framework for building agents that developers run themselves; the docs explicitly describe connecting *external* coding assistants (e.g., 'Connect your favorite coding assistant to our ADK developer Skills') rather than shipping a built-in AI assistant that end-users delegate tasks to inside the product itself. No evidence shows ADK embedding its own persistent assistant persona for task delegation.

   • [claimed-docs] “ADK is designed to be written by both humans and AI. Connect your favorite coding assistant to our ADK developer Skills and AI-aware develop…”
   • [claimed-docs] “Go from idea to coded ADK agent in minutes. Use your favorite AI-enabled developer environment to scaffold, build, test, evaluate, and deplo…”

   proof ↗⚑ flag

3. ai-native userOperate the product with natural-language commands

   weight 2 · round to OpenAI Agents SDK
   OpenAI Agents SDK✓fullclaimed7/10

   The SDK's core primitive is an LLM agent configured with natural-language instructions that processes natural-language input via Runner.run(), with tools, handoffs, and streaming built around this NL-driven interaction model. This is the central, well-documented capability of the product across many docs pages. missing for 10: no evidence of a dedicated end-user-facing NL interface (e.g., CLI/chat UI) shipped with the SDK—interaction is at the API/code level rather than an out-of-the-box NL command surface, and no independent hands-on account of NL command usage.

   • [claimed-docs] “An agent is a large language model (LLM) configured with instructions, tools, and optional runtime behavior such as handoffs, guardrails, an…”
   • [claimed-docs] “You can run agents via the Runner class. You have 3 options: Runner.run(), Runner.run_sync(), Runner.run_streamed()”
   • [claimed-docs] “You can run agents via the `Runner` class. You have 3 options: `Runner.run()`... `Runner.run_sync()`... `Runner.run_streamed()`”
   • [claimed-docs] “Agents, which are LLMs equipped with instructions and tools”
   • [claimed-docs] “Streaming lets you subscribe to updates of the agent run as it proceeds.”

   proof ↗⚑ flag

   Google ADK~partialclaimed5/10

   ADK docs claim it is designed to be built and operated via AI coding assistants (Agent Config for no-code agent building, 'Agents CLI' for AI-enabled dev environments to scaffold/build/test/deploy) which supports some natural-language-driven operation, but the primary operating surface is a traditional CLI (adk run/web/eval/deploy) and Python code, not direct NL commands to the tool itself. Missing for 10: concrete example of natural-language command controlling ADK end-to-end, independent/hands-on confirmation that Agent Config or coding-assistant integration works as a full NL interface.

   • [claimed-docs] “ADK is designed to be written by both humans and AI. Connect your favorite coding assistant to our ADK developer Skills and AI-aware develop…”
   • [claimed-docs] “Go from idea to coded ADK agent in minutes. Use your favorite AI-enabled developer environment to scaffold, build, test, evaluate, and deplo…”
   • [github] “Agent Config: Build agents without code.”
   • [github] “Agent Config: Build agents without code. Check out the Agent Config feature.”
   • [github] “Build agents without code. Check out the Agent Config feature.”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   OpenAI Agents SDK—none0/10

   The evidence pack shows only static documentation pages describing SDK concepts (agents, tools, handoffs, tracing) with no interactive API reference, no runnable code sandbox, and the OpenAPI/swagger probe explicitly returned 404 on all candidate paths, indicating no machine-readable interactive reference exists.

   • [probe] “PROBE openapi: all candidate paths 404 (https://developers.openai.com/openapi.json, https://developers.openai.com/swagger.json, https://deve…”
   • [probe] “PROBE llms.txt: HTTP 200 at https://developers.openai.com/llms.txt # OpenAI Developers > Complete documentation hub for OpenAI API, Ads, Pl…”
   • [probe] “PROBE docs-md: HTTP 200 at https://developers.openai.com/api/docs/guides/agents.md # Agents > For the complete documentation index, see [ll…”

   proof ↗⚑ flag

   Google ADK—none0/10

   The evidence pack shows standard docs, code snippets, and CLI examples, but no interactive/runnable API reference (e.g., a Swagger/OpenAPI explorer or live code sandbox); probes for openapi.json and similar endpoints explicitly returned 404s.

   • [probe] “PROBE openapi: all candidate paths 404 (https://google.github.io/openapi.json, https://google.github.io/swagger.json, https://google.github.…”
   • [probe] “PROBE docs-md: HTTP 404 at https://google.github.io/adk-docs/get-started/.md”
   • [claimed-docs] “agent = Agent( name="researcher", model="gemini-flash-latest", instruction="You help users research topics thoroughly.", too…”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   OpenAI Agents SDK—none0/10

   The probe explicitly checked common OpenAPI spec locations and all returned 404, and no other evidence shows a machine-readable API spec being published for the Agents SDK.

   • [probe] “PROBE openapi: all candidate paths 404 (https://developers.openai.com/openapi.json, https://developers.openai.com/swagger.json, https://deve…”

   proof ↗⚑ flag

   Google ADK—none0/10

   No evidence of a downloadable OpenAPI/machine-readable spec for ADK; explicit probes for openapi.json/swagger.json and llms.txt all return 404, indicating no such spec is published.

   • [probe] “PROBE llms.txt: HTTP 404 at https://google.github.io/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://google.github.io/adk-docs/get-started/.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://google.github.io/openapi.json, https://google.github.io/swagger.json, https://google.github.…”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round to OpenAI Agents SDK
   OpenAI Agents SDK✓fullclaimed7/10

   The SDK documents 'Sandbox agents' that run specialists inside real isolated workspaces with manifest-defined files and sandbox-native capabilities, plus a CodeInterpreterTool that executes code in a sandboxed environment — directly supporting isolated, non-production testing. missing for 10: independent/hands-on verification that isolation prevents production data access, and explicit guidance on how to configure sandbox vs production environments.

   • [claimed-docs] “Sandbox agents: Run specialists inside real isolated workspaces. Sandbox agents support manifest-defined files, sandbox client selection, an…”
   • [claimed-docs] “If the agent should run inside an isolated workspace with manifest-defined files and sandbox-native capabilities, read Sandbox agent concept…”
   • [claimed-docs] “OpenAI offers a few built-in tools when using the OpenAIResponsesModel... The WebSearchTool lets an agent search the web. The FileSearchTool…”

   proof ↗⚑ flag

   Google ADK~partialclaimed5/10

   ADK supports local dev/test workflows (adk run, adk web, adk eval, local evaluation with test files and eval sets) that inherently run against a local/dev environment rather than production, and offline/disconnected deployment is mentioned. However, there's no explicit documentation of a dedicated 'sandbox' environment or data isolation guarantee distinct from production. missing for 10: explicit sandbox/staging environment docs, explicit statement that test runs are isolated from production data/state, independent confirmation of this isolation.

   • [claimed-docs] “This approach involves creating individual test files, each representing a single, simple agent-model interaction (a session).”
   • [claimed-docs] “This approach involves creating individual test files, each representing a single, simple agent-model interaction (a session). It's most eff…”
   • [claimed-docs] “Expected Intermediate Tool Use Trajectory: The tool calls we expect the agent to make in order to respond correctly to the user query.”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”
   • [github] “adk eval \ samples_for_testing/hello_world \ samples_for_testing/hello_world/hello_world_eval_set_001.evalset.json”
   • [claimed-docs] “This is a good option if you prefer to run offline or disconnected, or otherwise in a system that has no connection to Google Cloud.”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   OpenAI Agents SDK—none0/10

   The evidence pack documents SDK features (agents, tools, tracing, handoffs) but contains no mention of API versioning scheme or a documented deprecation policy; the OpenAPI probe even returned 404s, and no versioning/deprecation docs are cited.

   • [probe] “PROBE openapi: all candidate paths 404 (https://developers.openai.com/openapi.json, https://developers.openai.com/swagger.json, https://deve…”

   proof ↗⚑ flag

   Google ADK—none0/10

   No evidence in the pack mentions API versioning schemes or a documented deprecation policy for ADK; probes for OpenAPI specs and related docs all returned 404s. missing for 10: versioning scheme documentation, deprecation policy, changelog/migration guides, semantic versioning commitments.

   • [probe] “PROBE llms.txt: HTTP 404 at https://google.github.io/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://google.github.io/adk-docs/get-started/.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://google.github.io/openapi.json, https://google.github.io/swagger.json, https://google.github.…”

   proof ↗⚑ flag

Agent authoring

1. developerDefine an agent with typed custom tools in a few lines of code

   weight 3 · round to OpenAI Agents SDK
   OpenAI Agents SDK✓fullcommunity9/10

   Docs show agents defined with instructions/tools in a few lines, with FunctionTool wrapping any Python function via automatic schema generation and Pydantic-powered validation, giving typed custom tools with minimal boilerplate; community feedback corroborates the SDK's simplicity relative to alternatives. Missing for 10: no direct hands-on code snippet in evidence showing the exact few-line agent+tool definition, only docs descriptions.

   • [claimed-docs] “An agent is a large language model (LLM) configured with instructions, tools, and optional runtime behavior such as handoffs, guardrails, an…”
   • [claimed-docs] “FunctionTool instances: wrap any Python function as a tool.”
   • [claimed-docs] “Function tools: Turn any Python function into a tool with automatic schema generation and Pydantic-powered validation.”
   • [claimed-docs] “`FunctionTool` instances: wrap any Python function as a tool.”
   • [community] “OpenAI SDK is much simpler to use and understand than langchain etc. I havent deployed really complicated agents, but I have a number of sim…”

   proof ↗⚑ flag

   Google ADK~partialclaimed6/10

   The docs show a concrete few-line example of defining an Agent with a model, instructions, and a tools list (google-adk-docs-22), and other docs confirm tools are a core, optional component of agent definition (google-adk-docs-2, google-adk-docs-13). However, the evidence never shows a custom Python tool function with type hints/typed parameters being defined and passed in — only a prebuilt tool (google_search) is used in the example. Missing for 10: an explicit example of writing a custom typed tool function, and documentation of automatic schema/type inference from function signatures.

   • [claimed-docs] “agent = Agent( name="researcher", model="gemini-flash-latest", instruction="You help users research topics thoroughly.", too…”
   • [claimed-docs] “Building an agent with just a model, instructions, and tools is a great place to start for most developers.”
   • [claimed-docs] “The basic components of an Agent are an artificial intelligence (AI) model, task instructions, and optionally, a set of tools to be used by …”

   proof ↗⚑ flag

Ai buildability

1. ai-native userHave a coding agent scaffold a new agent project from an official CLI or template in one command

   weight 2 · round to Google ADK
   OpenAI Agents SDK—none0/10

   The evidence pack covers agent primitives, tools, handoffs, tracing, and MCP support, but there is no mention of an official CLI or project scaffolding/template command to bootstrap a new agent project in one step.

   ⚑ flag

   Google ADK✓fullclaimed8/10

   ADK docs explicitly advertise an official 'Agents CLI' to scaffold, build, test, evaluate, and deploy agents in minutes, and the GitHub README shows concrete one-line commands (adk run, adk web, adk deploy) plus a no-code 'Agent Config' template feature for scaffolding agents. This directly matches the ai-native scaffolding story via an official CLI/template workflow. Missing for 10: independent/hands-on confirmation of the one-command scaffold experience beyond first-party docs.

   • [claimed-docs] “Go from idea to coded ADK agent in minutes. Use your favorite AI-enabled developer environment to scaffold, build, test, evaluate, and deplo…”
   • [claimed-docs] “Migrate existing agents and workflows to ADK with Agents CLI.”
   • [github] “adk run path/to/my_agent”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”
   • [github] “Agent Config: Build agents without code. Check out the Agent Config feature.”
   • [github] “Build agents without code. Check out the Agent Config feature.”

   proof ↗⚑ flag

2. ai-native userRun the framework's example agents headlessly from a terminal so an agent can verify what it just built

   weight 2 · round to Google ADK
   OpenAI Agents SDK~partialclaimed5/10

   Runner.run_sync() and run() enable headless CLI execution of agents, and tracing lets an agent's own run be verified programmatically, but there is no documented example agent or CLI harness specifically for an agent to verify what it just built. missing for 10: a ready-made example agent script for self-verification, CLI-specific documentation/tutorial, and independent confirmation of headless terminal usage for this exact workflow.

   • [claimed-docs] “You can run agents via the Runner class. You have 3 options: Runner.run(), Runner.run_sync(), Runner.run_streamed()”
   • [claimed-docs] “You can run agents via the `Runner` class. You have 3 options: `Runner.run()`... `Runner.run_sync()`... `Runner.run_streamed()`”
   • [claimed-docs] “The Agents SDK includes built-in tracing, collecting a comprehensive record of events during an agent run: LLM generations, tool calls, hand…”
   • [claimed-docs] “Using the Traces dashboard, you can debug, visualize, and monitor your workflows during development and in production.”

   proof ↗⚑ flag

   Google ADK✓fullclaimed7/10

   ADK provides a documented CLI (`adk run path/to/my_agent`) to run agents headlessly from a terminal, plus `adk eval` for automated verification of agent behavior against eval sets, matching the 'verify what it just built' use case for an ai-native/agentic workflow. Missing for 10: explicit confirmation that shipped 'example agents' (vs. user-authored ones) work with this flow, and independent/hands-on corroboration beyond the official repo docs.

   • [github] “adk run path/to/my_agent”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”
   • [github] “adk eval \ samples_for_testing/hello_world \ samples_for_testing/hello_world/hello_world_eval_set_001.evalset.json”
   • [claimed-docs] “This approach involves creating individual test files, each representing a single, simple agent-model interaction (a session).”

   proof ↗⚑ flag

3. ai-native userRely on strict typing and schema validation so a coding agent catches its own mistakes at build time

   weight 2 · round to OpenAI Agents SDK
   OpenAI Agents SDK~partialclaimed6/10

   Docs confirm automatic schema generation and Pydantic-powered validation for function tools and structured outputs, which supports catching malformed tool calls early, but this is runtime validation rather than true build-time/compile-time type checking, and no independent evidence corroborates catching agent mistakes at build time. missing for 10: evidence of compile-time/static type checking (e.g., TypeScript strict mode enforcement), independent hands-on validation of build-time error catching, and confirmation this applies uniformly across JS/Python SDKs.

   • [claimed-docs] “Function tools: Turn any Python function into a tool with automatic schema generation and Pydantic-powered validation.”
   • [claimed-docs] “An agent is a large language model (LLM) configured with instructions, tools, and optional runtime behavior such as handoffs, guardrails, an…”
   • [claimed-docs] “FunctionTool instances: wrap any Python function as a tool.”

   proof ↗⚑ flag

   Google ADK—none0/10

   The evidence pack covers ADK's agent orchestration, deployment, and evaluation features, but contains no mention of strict typing, schema validation, or build-time error detection for tool/agent definitions — the evaluation features described (docs-20, docs-21, docs-25) are runtime test-set based, not compile/build-time type checks.

   ⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Google ADK
   OpenAI Agents SDK—none0/10

   The evidence describes agent orchestration primitives (tools, handoffs, streaming, tracing, sessions) but nothing documents built-in support for bulk/batch operations across many items at once; that would need to be custom-built by a developer using function tools.

   ⚑ flag

   Google ADK~partialclaimed3/10

   ADK's Workflow Runtime offers fan-out/fan-in and loop constructs that could be used by developers to build bulk-item processing pipelines, but there is no documented built-in 'bulk operations' feature or example for end users acting across many items at once. Missing for 10: explicit bulk-operation tooling/UI, documented examples of processing many items in one call, and evidence of end-user (not just developer-framework) bulk workflows.

   • [github] “Workflow Runtime: A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan…”
   • [claimed-docs] “you can use the ADK development framework to expand them into workflows, which allow you to combine and orchestrate multiple agents and code…”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to Google ADK
   OpenAI Agents SDK~partialclaimed4/10

   The SDK supports guardrails (validation checks that can block/alter agent behavior), human-in-the-loop interruptions when a tool call requires approval, and streaming to subscribe to run events — these act as limited event-triggered mechanisms within an agent run. However, there is no documented general-purpose rule engine or event/webhook trigger system for arbitrary external events. Missing for 10: explicit event-trigger/rule definition API (e.g., on-event listeners tied to external triggers), documentation of custom event-driven automation beyond guardrails/interruptions, and independent evidence of this pattern in practice.

   • [claimed-docs] “Guardrails enable you to do checks and validations of user input and agent output.”
   • [claimed-docs] “When a tool call requires approval, the SDK pauses the run, returns interruptions, and lets you resume later from the same RunState.”
   • [claimed-docs] “When a tool call requires approval, the SDK pauses the run, returns `interruptions`, and lets you resume later from the same `RunState`.”
   • [claimed-docs] “Streaming lets you subscribe to updates of the agent run as it proceeds. This can be useful for showing the end-user progress updates and pa…”

   proof ↗⚑ flag

   Google ADK✓fullclaimed7/10

   ADK explicitly supports event-driven automation via Callbacks ("Hook into specific events during an agent's execution lifecycle... without altering core agent logic") and a Workflow Runtime graph engine with routing, retry, fan-out/fan-in and dynamic nodes for triggering actions on execution events, matching the story of defining rules that fire on events. missing for 10: independent/hands-on evidence of callback-triggered rules in production use, and more detail on condition-based rule syntax beyond docs summaries.

   • [claimed-docs] “Callbacks: Hook into specific events during an agent's execution lifecycle to add logging, monitoring, or custom side-effects without alteri…”
   • [github] “Workflow Runtime: A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan…”
   • [github] “A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan-out/fan-in, loops…”
   • [claimed-docs] “In ADK, any agent application that has more than one agent or executable Node is considered a workflow.”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round drawn
   OpenAI Agents SDK—none0/10

   The Agents SDK provides primitives for running agents (Runner, streaming, sessions, tracing) but no evidence of a scheduler, cron-like trigger, or built-in mechanism for recurring/periodic job execution; scheduling would need to be built externally.

   ⚑ flag

   Google ADK—none0/10

   The evidence covers agent/workflow orchestration, deployment targets (Cloud Run, GKE, Agent Runtime), and evaluation, but nothing describes scheduling, cron-like triggers, or recurring execution of jobs/workflows. Absence of evidence for this applicable automation-depth capability yields 'none'.

   ⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round drawn
   OpenAI Agents SDK—none0/10

   The evidence covers agent orchestration, tracing, tools, memory, and MCP support, but there is no mention of versioning automations, review workflows, or rollback capabilities anywhere in the docs or community discussion. Tracing/debugging is observability, not version control or rollback.

   ⚑ flag

   Google ADK—none0/10

   ADK is a framework for building agents (code, workflows, tools, deployment) but the evidence pack shows no version control, review, or rollback mechanism for automations themselves — no changelog/versioning UI, no approval/review workflow for agent definitions, no rollback feature. Agent code could theoretically be tracked via external git, but ADK itself provides no such capability in the evidence. Missing for 10: any versioning system, review/approval workflow, or rollback capability for automations.

   ⚑ flag

Deployment

1. engineering-leadDeploy an agent to a managed runtime and call it as an API endpoint

   weight 2 · round to Google ADK
   OpenAI Agents SDK—none0/10

   The Agents SDK is a code framework for building and running agents locally/self-hosted (Runner class, sessions, tracing), but there is no evidence of a managed runtime/hosting service that deploys an agent and exposes it as a callable API endpoint; OpenAPI probes even 404. Sandbox agents run isolated workspaces for tool execution, not deployment-as-API.

   • [claimed-docs] “You can run agents via the Runner class. You have 3 options: Runner.run(), Runner.run_sync(), Runner.run_streamed()”
   • [claimed-docs] “Sandbox agents: Run specialists inside real isolated workspaces. Sandbox agents support manifest-defined files, sandbox client selection, an…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://developers.openai.com/openapi.json, https://developers.openai.com/swagger.json, https://deve…”

   proof ↗⚑ flag

   Google ADK✓fullclaimed8/10

   ADK docs explicitly describe deploying agents to a fully managed, auto-scaling Agent Engine/Agent Runtime on Google Cloud, plus alternative managed options like Cloud Run and GKE, with the stated purpose being to make the agent 'accessed, queried, and used in production' as an API endpoint. Missing for 10: no explicit hands-on/independent confirmation of the API contract (e.g., request/response schema) or third-party verification of endpoint behavior beyond first-party docs.

   • [claimed-docs] “Agent Runtime is a fully managed auto-scaling service on Google Cloud specifically designed for deploying, managing, and scaling AI agents b…”
   • [claimed-docs] “Cloud Run is a managed auto-scaling compute platform on Google Cloud that enables you to run your agent as a container-based application.”
   • [claimed-docs] “GKE is a good option if you need more control over the deployment as well as for running Open Models.”
   • [claimed-docs] “Once you've built and tested your agent using ADK, the next step is to deploy it so it can be accessed, queried, and used in production”
   • [claimed-docs] “You can manually package your Agent into a container image and then run it in any environment that supports container images.”

   proof ↗⚑ flag

2. engineering-leadRun my agents entirely on my own infrastructure with no dependence on the vendor's platform

   weight 2 · round to Google ADK
   OpenAI Agents SDK~partialclaimed6/10

   The SDK is an open-source Python/JS library you run yourself, and it's explicitly provider-agnostic (100+ LLMs beyond OpenAI), so an engineering lead can host the orchestration layer entirely on their own infra. However, defaults point to OpenAI models/env vars, and built-in tracing pushes data to OpenAI's hosted Traces dashboard, meaning some optional but promoted features still tie back to the vendor platform. missing for 10: explicit documentation on disabling all OpenAI-hosted dependencies (tracing, hosted tools) for a fully vendor-free deployment, and independent confirmation of successful fully self-hosted production use.

   • [github] “It is provider-agnostic, supporting the OpenAI Responses and Chat Completions APIs, as well as 100+ other LLMs.”
   • [claimed-docs] “The Agents SDK includes built-in tracing, collecting a comprehensive record of events during an agent run: LLM generations, tool calls, hand…”
   • [claimed-docs] “Using the Traces dashboard, you can debug, visualize, and monitor your workflows during development and in production.”
   • [claimed-docs] “if you want to consistently use a specific model for all agents that do not set a custom model, set the `OPENAI_DEFAULT_MODEL` environment v…”

   proof ↗⚑ flag

   Google ADK✓fullclaimed8/10

   ADK is an open-source framework (google/adk-python) that supports running agents locally via `adk run`/`adk web`, packaging into containers with `adk deploy docker`, deploying to any container-supporting environment, and explicitly documents an offline/disconnected mode with no Google Cloud connection required, alongside optional managed services like Agent Runtime/Cloud Run/GKE. missing for 10: independent/hands-on confirmation of fully vendor-free operation (e.g., third-party report of running ADK completely offline with non-Google models) and clarity on whether any telemetry/model calls still phone home by default.

   • [claimed-docs] “This is a good option if you prefer to run offline or disconnected, or otherwise in a system that has no connection to Google Cloud.”
   • [claimed-docs] “You can manually package your Agent into a container image and then run it in any environment that supports container images.”
   • [github] “adk deploy docker --with_ui <agent-folder>”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”
   • [claimed-docs] “ADK can work with almost any generative AI model. The framework provides easy access to Gemini as well as other leading models, and we provi…”
   • [claimed-docs] “GKE is a good option if you need more control over the deployment as well as for running Open Models.”

   proof ↗⚑ flag

Portability

1. developerSwap the underlying LLM provider or model without rewriting my agent

   weight 3 · round drawn
   OpenAI Agents SDK✓fullclaimed8/10

   Docs confirm provider-agnostic design supporting OpenAI Responses/Chat Completions APIs plus 100+ other LLMs, with built-in OpenAI model flavors and an environment-variable/config mechanism (OPENAI_DEFAULT_MODEL) to swap default models without code rewrites, implying a model-abstraction layer for swapping providers. Missing for 10: independent hands-on verification of swapping to a non-OpenAI provider and clearer first-party documentation of the LiteLLM/custom-provider integration mechanism.

   • [github] “It is provider-agnostic, supporting the OpenAI Responses and Chat Completions APIs, as well as 100+ other LLMs.”
   • [claimed-docs] “The Agents SDK comes with out-of-the-box support for OpenAI models in two flavors”
   • [claimed-docs] “if you want to consistently use a specific model for all agents that do not set a custom model, set the `OPENAI_DEFAULT_MODEL` environment v…”

   proof ↗⚑ flag

   Google ADK✓fullclaimed8/10

   Docs explicitly state ADK works with 'almost any generative AI model,' providing adapters for Gemini and many other model providers including locally running models, and the agent definition just takes a `model` string parameter (e.g., 'gemini-flash-latest'), implying swapping providers is a config change rather than a rewrite. Missing for 10: independent/hands-on confirmation that switching providers requires no code changes to agent logic, and no explicit list of supported non-Gemini providers with concrete migration examples.

   • [claimed-docs] “ADK can work with almost any generative AI model. The framework provides easy access to Gemini as well as other leading models, and we provi…”
   • [claimed-docs] “agent = Agent( name="researcher", model="gemini-flash-latest", instruction="You help users research topics thoroughly.", too…”
   • [claimed-docs] “The basic components of an Agent are an artificial intelligence (AI) model, task instructions, and optionally, a set of tools to be used by …”

   proof ↗⚑ flag

Evals

1. engineering-leadScore agent quality with built-in evals and run them as part of CI

   weight 2 · round to Google ADK
   OpenAI Agents SDK—none0/10

   The evidence pack shows tracing/observability (traces dashboard, tool/handoff logs) but no mention of built-in evals, scoring/grading of agent outputs, or CI integration for automated quality checks.

   ⚑ flag

   Google ADK✓fullclaimed8/10

   ADK ships a first-party evaluation framework with groundtruth and rubric-based metrics, expected tool-use trajectories, evalset.json test files, and a documented CLI command (`adk eval <agent> <evalset>`) that can be scripted/invoked headlessly, which is exactly the shape needed for CI integration. Missing for 10: explicit CI/CD pipeline documentation (e.g., a GitHub Actions example) and independent/third-party corroboration of running adk eval in CI.

   • [claimed-docs] “This approach involves creating individual test files, each representing a single, simple agent-model interaction (a session).”
   • [claimed-docs] “ADK provides both groundtruth based and rubric based tool use evaluation metrics.”
   • [claimed-docs] “This approach involves creating individual test files, each representing a single, simple agent-model interaction (a session). It's most eff…”
   • [claimed-docs] “Expected Intermediate Tool Use Trajectory: The tool calls we expect the agent to make in order to respond correctly to the user query.”
   • [github] “adk eval \ samples_for_testing/hello_world \ samples_for_testing/hello_world/hello_world_eval_set_001.evalset.json”

   proof ↗⚑ flag

Testing

1. developerUnit-test agents with mocked models and tools

   weight 2 · round to Google ADK
   OpenAI Agents SDK—none0/10

   The evidence pack covers agents, tools, handoffs, tracing, sessions, and runners, but nothing mentions unit testing, mocking models/tools, or any test framework support for the SDK.

   ⚑ flag

   Google ADK~partialclaimed5/10

   ADK docs describe a test-file based evaluation approach explicitly described as 'a form of unit testing' for single agent-model interactions, with expected tool-use trajectories and groundtruth/rubric metrics plus an `adk eval` CLI — but none of this evidence explicitly describes mocking models or tools (e.g., swapping in fake LLM responses or stub tool implementations) for isolated unit tests. Missing for 10: explicit mocked-model/mocked-tool test fixtures or APIs, independent/hands-on confirmation of mocking support, and unit-test framework integration examples (e.g., pytest with mock objects).

   • [claimed-docs] “This approach involves creating individual test files, each representing a single, simple agent-model interaction (a session).”
   • [claimed-docs] “This approach involves creating individual test files, each representing a single, simple agent-model interaction (a session). It's most eff…”
   • [claimed-docs] “Expected Intermediate Tool Use Trajectory: The tool calls we expect the agent to make in order to respond correctly to the user query.”
   • [claimed-docs] “ADK provides both groundtruth based and rubric based tool use evaluation metrics.”
   • [github] “adk eval \ samples_for_testing/hello_world \ samples_for_testing/hello_world/hello_world_eval_set_001.evalset.json”

   proof ↗⚑ flag

Tracing

1. developerTrace every LLM call and tool invocation of an agent run in an observability UI

   weight 3 · round to OpenAI Agents SDK
   OpenAI Agents SDK✓fullclaimed9/10

   Docs explicitly describe built-in tracing that records LLM generations, tool calls, handoffs, guardrails, and custom events, viewable/debuggable in the Traces dashboard for development and production monitoring. This directly matches the story of tracing every LLM call and tool invocation in an observability UI. Missing for 10: independent/hands-on corroboration of the Traces UI beyond vendor docs.

   • [claimed-docs] “The Agents SDK includes built-in tracing, collecting a comprehensive record of events during an agent run: LLM generations, tool calls, hand…”
   • [claimed-docs] “Using the Traces dashboard, you can debug, visualize, and monitor your workflows during development and in production.”
   • [claimed-docs] “The Agents SDK includes built-in tracing, collecting a comprehensive record of events during an agent run: LLM generations, tool calls, hand…”
   • [claimed-docs] “The Agents SDK includes built-in tracing, collecting a comprehensive record of events during an agent run: LLM generations, tool calls, hand…”
   • [claimed-docs] “The Agents SDK includes built-in tracing, collecting a comprehensive record of events during an agent run”

   proof ↗⚑ flag

   Google ADK~partialclaimed5/10

   ADK ships a built-in development Web UI explicitly for testing, evaluating, and debugging agents, and provides callbacks to hook into execution lifecycle events for logging/monitoring, which together imply some run-level visibility into tool and model calls. However, the evidence never explicitly describes a trace view showing each LLM call and tool invocation of a run, nor mentions integration with tracing standards (e.g., OpenTelemetry) or a dedicated observability dashboard beyond the dev/eval UI. Missing for 10: explicit documentation of per-call tracing UI, tool-invocation-level trace inspection, and any third-party/hands-on confirmation of this granularity.

   • [github] “A built-in development UI to help you test, evaluate, debug, and showcase your agent(s).”
   • [github] “Web UI (supports multi-agent directories or pointing directly to a single agent folder)”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”
   • [claimed-docs] “Callbacks: Hook into specific events during an agent's execution lifecycle to add logging, monitoring, or custom side-effects without alteri…”

   proof ↗⚑ flag

Guardrails

1. developerAttach input/output guardrails that validate, transform, or block unsafe content

   weight 3 · round to OpenAI Agents SDK
   OpenAI Agents SDK✓fullclaimed8/10

   Docs directly state guardrails enable checks/validations of user input and agent output, and guardrails are a first-class runtime concept alongside handoffs/tools with tracing integration. Missing for 10: no independent/hands-on corroboration of guardrail blocking/transforming behavior in practice, and no detail on transform capability beyond validation/blocking.

   • [claimed-docs] “Guardrails enable you to do checks and validations of user input and agent output.”
   • [claimed-docs] “An agent is a large language model (LLM) configured with instructions, tools, and optional runtime behavior such as handoffs, guardrails, an…”
   • [claimed-docs] “The Agents SDK includes built-in tracing, collecting a comprehensive record of events during an agent run: LLM generations, tool calls, hand…”

   proof ↗⚑ flag

   Google ADK~partialclaimed5/10

   ADK exposes general extensibility hooks—Callbacks to intercept execution events for custom logic/side-effects, Plugins for pre-packaged behaviors, and a Tool Confirmation (HITL) flow that can guard tool execution—which developers could use to build input/output guardrails, but there is no dedicated 'guardrails' feature, built-in content-safety/validation API, or example showing blocking/transforming unsafe content end-to-end. Missing for 10: explicit guardrail/validation API or moderation integration, documented examples of blocking/transforming unsafe input or output, and any third-party/community confirmation of this pattern in practice.

   • [claimed-docs] “Callbacks: Hook into specific events during an agent's execution lifecycle to add logging, monitoring, or custom side-effects without alteri…”
   • [claimed-docs] “Plugins: Integrate complex, pre-packaged behaviors and third-party services directly into your agent's workflow.”
   • [github] “Tool Confirmation: A tool confirmation flow (HITL) that can guard tool execution with explicit confirmation and custom input.”
   • [github] “A tool confirmation flow (HITL) that can guard tool execution with explicit confirmation and custom input.”

   proof ↗⚑ flag

2. engineering-leadRestrict what an agent may do with fine-grained tool permissions and sandboxed execution

   weight 2 · round to OpenAI Agents SDK
   OpenAI Agents SDK~partialclaimed7/10

   Docs show real guardrail mechanisms: input/output validation guardrails, human-in-the-loop tool-call approval that pauses runs pending explicit permission, and dedicated 'Sandbox agents' that execute in isolated, manifest-defined workspaces, plus a sandboxed CodeInterpreterTool. Together these give an engineering lead meaningful control over tool use and execution isolation, though there's no first-party doc on granular per-tool ACL/permission policies beyond approval gating, and no independent/hands-on security audit corroborating sandbox robustness. Missing for 10: explicit fine-grained per-tool permission/policy configuration docs, independent verification of sandbox isolation guarantees.

   • [claimed-docs] “Guardrails enable you to do checks and validations of user input and agent output.”
   • [claimed-docs] “When a tool call requires approval, the SDK pauses the run, returns interruptions, and lets you resume later from the same RunState.”
   • [claimed-docs] “When a tool call requires approval, the SDK pauses the run, returns `interruptions`, and lets you resume later from the same `RunState`.”
   • [claimed-docs] “Sandbox agents: Run specialists inside real isolated workspaces. Sandbox agents support manifest-defined files, sandbox client selection, an…”
   • [claimed-docs] “If the agent should run inside an isolated workspace with manifest-defined files and sandbox-native capabilities, read Sandbox agent concept…”
   • [claimed-docs] “OpenAI offers a few built-in tools when using the OpenAIResponsesModel... The WebSearchTool lets an agent search the web. The FileSearchTool…”

   proof ↗⚑ flag

   Google ADK~partialclaimed4/10

   ADK provides a Tool Confirmation (HITL) flow that can gate tool execution with explicit confirmation/custom input, plus callbacks/plugins hooks to intercept agent actions, giving some control over agent behavior. However there is no evidence of fine-grained per-tool permission policies or an actual sandboxed execution environment for code/tool runs. Missing for 10: explicit sandboxing of tool/code execution, a permissions/ACL system scoping tool access, and independent verification of these guardrails in practice.

   • [github] “Tool Confirmation: A tool confirmation flow (HITL) that can guard tool execution with explicit confirmation and custom input.”
   • [github] “A tool confirmation flow (HITL) that can guard tool execution with explicit confirmation and custom input.”
   • [claimed-docs] “Callbacks: Hook into specific events during an agent's execution lifecycle to add logging, monitoring, or custom side-effects without alteri…”
   • [claimed-docs] “Plugins: Integrate complex, pre-packaged behaviors and third-party services directly into your agent's workflow.”

   proof ↗⚑ flag

Approval flows

1. developerPause an agent mid-run for human input or approval and resume with the human's decision

   weight 3 · round drawn
   OpenAI Agents SDK✓fullclaimed8/10

   Docs explicitly describe a human-in-the-loop flow where a tool call requiring approval pauses the run, returns interruptions, and lets the developer resume later from the same RunState with the human's decision. This directly matches the story's pause/resume-for-approval pattern. Missing for 10: independent/hands-on corroboration beyond first-party docs, and Python-specific (vs JS) documentation detail on the same mechanism.

   • [claimed-docs] “When a tool call requires approval, the SDK pauses the run, returns interruptions, and lets you resume later from the same RunState.”
   • [claimed-docs] “When a tool call requires approval, the SDK pauses the run, returns `interruptions`, and lets you resume later from the same `RunState`.”

   proof ↗⚑ flag

   Google ADK✓fullclaimed8/10

   ADK explicitly documents a Tool Confirmation flow described as HITL that can 'guard tool execution with explicit confirmation and custom input,' plus a Workflow Runtime and Task API both explicitly listing human-in-the-loop support with state management for pausing and resuming execution. This directly matches pausing mid-run for human approval and resuming with the decision, though missing for 10: a concrete end-to-end code example showing pause/resume state persistence and independent third-party corroboration beyond vendor GitHub README claims.

   • [github] “Tool Confirmation: A tool confirmation flow (HITL) that can guard tool execution with explicit confirmation and custom input.”
   • [github] “A tool confirmation flow (HITL) that can guard tool execution with explicit confirmation and custom input.”
   • [github] “Workflow Runtime: A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan…”
   • [github] “A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan-out/fan-in, loops…”
   • [github] “Task API: Structured agent-to-agent delegation with multi-turn task mode, single-turn controlled output, mixed delegation patterns, human-in…”
   • [github] “Structured agent-to-agent delegation with multi-turn task mode, single-turn controlled output, mixed delegation patterns, human-in-the-loop,…”

   proof ↗⚑ flag

2. engineering-leadRequire human approval before specific sensitive tool calls execute

   weight 2 · round drawn
   OpenAI Agents SDK✓fullclaimed8/10

   Docs explicitly describe a needsApproval/interruption mechanism: when a tool call requires approval, the SDK pauses the run, returns interruptions, and lets you resume later from the same RunState, enabling human-in-the-loop gating for specific sensitive tools. Missing for 10: independent/hands-on corroboration of this workflow and Python-side (vs JS) doc citation for the same feature.

   • [claimed-docs] “When a tool call requires approval, the SDK pauses the run, returns interruptions, and lets you resume later from the same RunState.”
   • [claimed-docs] “When a tool call requires approval, the SDK pauses the run, returns `interruptions`, and lets you resume later from the same `RunState`.”

   proof ↗⚑ flag

   Google ADK✓fullclaimed8/10

   ADK explicitly documents a 'Tool Confirmation' HITL flow that guards tool execution with explicit confirmation and custom input, plus broader human-in-the-loop support in its workflow/task orchestration engines, directly matching the story of requiring approval before sensitive tool calls execute. Missing for 10: no independent/hands-on validation or detailed walkthrough of configuring per-tool approval policies beyond the feature summary.

   • [github] “Tool Confirmation: A tool confirmation flow (HITL) that can guard tool execution with explicit confirmation and custom input.”
   • [github] “A tool confirmation flow (HITL) that can guard tool execution with explicit confirmation and custom input.”
   • [github] “Workflow Runtime: A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan…”
   • [github] “A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan-out/fan-in, loops…”
   • [github] “Task API: Structured agent-to-agent delegation with multi-turn task mode, single-turn controlled output, mixed delegation patterns, human-in…”
   • [github] “Structured agent-to-agent delegation with multi-turn task mode, single-turn controlled output, mixed delegation patterns, human-in-the-loop,…”

   proof ↗⚑ flag

Memory

1. developerTrim, summarize, or filter conversation history to keep an agent inside its context window

   weight 2 · round to Google ADK
   OpenAI Agents SDK—none0/10

   Evidence shows built-in session memory that automatically persists conversation history across runs (docs-8, docs-33), but nothing describes trimming, summarizing, or filtering that history to manage context window size. No documentation mentions truncation, summarization tools, or history-pruning APIs.

   • [claimed-docs] “The Agents SDK provides built-in session memory to automatically maintain conversation history across multiple agent runs, eliminating the n…”
   • [claimed-docs] “The Agents SDK provides built-in session memory to automatically maintain conversation history across multiple agent runs”

   proof ↗⚑ flag

   Google ADK✓fullclaimed7/10

   ADK docs explicitly state it "automatically filters irrelevant events, summarizes older conversational turns, lazy-loads artifacts, and tracks token usage," directly addressing trimming/summarizing/filtering to manage context window, reinforced by mention of designing for AI context window limits. Missing for 10: no code example/API reference showing how a developer configures or customizes this summarization/filtering behavior, and no independent/hands-on corroboration beyond first-party docs.

   • [claimed-docs] “ADK automatically filters irrelevant events, summarizes older conversational turns, lazy-loads artifacts, and tracks token usage.”
   • [claimed-docs] “Use prebuilt or custom Agent Skills to extend agent capabilities in a way that works efficiently inside AI context window limits.”

   proof ↗⚑ flag

2. developerGive agents long-term memory that persists across sessions and threads

   weight 2 · round to OpenAI Agents SDK
   OpenAI Agents SDK~partialclaimed5/10

   Docs describe built-in 'session memory' that automatically maintains conversation history across multiple agent runs, removing manual history management (openai-agents-docs-8/33), which covers persistence within a session. However, there's no evidence of explicit long-term memory that persists across different threads/sessions (e.g., a durable memory store, vector-based long-term recall, or cross-thread continuity beyond a single session object) — FileSearchTool/Vector Stores are mentioned only as retrieval tools, not as an automatic long-term memory mechanism tied to sessions. Missing for 10: documented cross-session/cross-thread persistent memory store, examples of custom long-term memory backends, and independent confirmation that memory survives beyond a single Session object.

   • [claimed-docs] “The Agents SDK provides built-in session memory to automatically maintain conversation history across multiple agent runs, eliminating the n…”
   • [claimed-docs] “The Agents SDK provides built-in session memory to automatically maintain conversation history across multiple agent runs”
   • [claimed-docs] “OpenAI offers a few built-in tools when using the OpenAIResponsesModel... The WebSearchTool lets an agent search the web. The FileSearchTool…”

   proof ↗⚑ flag

   Google ADK~partialclaimed3/10

   The docs mention session-based interactions and automatic context management (filtering irrelevant events, summarizing older turns, tracking token usage) but there is no explicit evidence of a dedicated long-term memory service or store that persists agent knowledge across separate sessions/threads. missing for 10: explicit memory/session-store API docs, cross-session persistence guarantees, first-party examples of retrieving memory in a new thread.

   • [claimed-docs] “ADK automatically filters irrelevant events, summarizes older conversational turns, lazy-loads artifacts, and tracks token usage.”
   • [claimed-docs] “This approach involves creating individual test files, each representing a single, simple agent-model interaction (a session).”

   proof ↗⚑ flag

1. ai-native userRead the product's source under an open license

   weight 2 · round to Google ADK
   OpenAI Agents SDK~partialcommunity5/10

   The evidence confirms the SDK's source code is hosted publicly on GitHub (openai/openai-agents-python) and is actively discussed by the community, implying open access to the code, but no evidence pack item explicitly states or cites a license (e.g., MIT/Apache) or license file. Missing for 10: explicit license text/confirmation, documentation citing the specific open-source license terms.

   • [github] “It is provider-agnostic, supporting the OpenAI Responses and Chat Completions APIs, as well as 100+ other LLMs.”
   • [github] “Voice agents: Build voice pipelines that combine speech-to-text, an agent workflow, and text-to-speech”
   • [github] “Realtime agents: Build powerful voice agents with `gpt-realtime-2.1` and full agent features”
   • [community] “OpenAI SDK is much simpler to use and understand than langchain etc. I havent deployed really complicated agents, but I have a number of sim…”

   proof ↗⚑ flag

   Google ADK✓fullclaimed7/10

   The evidence repeatedly links to the public GitHub repository https://github.com/google/adk-python, which hosts the full source code and CLI (adk run, adk web, adk eval, adk deploy) that AI-native users can read and inspect directly. Missing for 10: an explicit citation of the license file/type (e.g., Apache-2.0) confirming the open-license terms, and independent third-party confirmation of licensing.

   • [github] “Agent Config: Build agents without code.”
   • [github] “A built-in development UI to help you test, evaluate, debug, and showcase your agent(s).”
   • [github] “adk run path/to/my_agent”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”

   proof ↗⚑ flag

2. ai-native userSelf-host the core product

   weight 3 · round drawn
   OpenAI Agents SDK✓fullclaimed8/10

   The Agents SDK is an open-source Python/JS library (github.com/openai/openai-agents-python) that you install and run yourself via the Runner class (Runner.run/run_sync/run_streamed) with no required hosted backend, and it's provider-agnostic (100+ LLMs), meaning the core execution loop runs entirely in your own infrastructure by design. Missing for 10: no explicit self-hosting guide/deployment docs or independent report confirming production self-hosted deployments beyond code examples.

   • [github] “It is provider-agnostic, supporting the OpenAI Responses and Chat Completions APIs, as well as 100+ other LLMs.”
   • [claimed-docs] “You can run agents via the Runner class. You have 3 options: Runner.run(), Runner.run_sync(), Runner.run_streamed()”
   • [claimed-docs] “You can run agents via the `Runner` class. You have 3 options: `Runner.run()`... `Runner.run_sync()`... `Runner.run_streamed()`”
   • [claimed-docs] “if you want to consistently use a specific model for all agents that do not set a custom model, set the `OPENAI_DEFAULT_MODEL` environment v…”

   proof ↗⚑ flag

   Google ADK✓fullclaimed8/10

   ADK is an open-source framework (github.com/google/adk-python) that can be run entirely locally via `adk run`/`adk web`, packaged into containers, and deployed offline/disconnected from Google Cloud, evidencing full self-hosting capability without requiring the vendor's managed service. Missing for 10: no independent third-party report confirming a full self-hosted production deployment, and no explicit self-hosted infra requirements/scaling guidance beyond container packaging.

   • [claimed-docs] “This is a good option if you prefer to run offline or disconnected, or otherwise in a system that has no connection to Google Cloud.”
   • [claimed-docs] “You can manually package your Agent into a container image and then run it in any environment that supports container images.”
   • [github] “adk run path/to/my_agent”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”
   • [github] “adk deploy docker --with_ui <agent-folder>”
   • [claimed-docs] “GKE is a good option if you need more control over the deployment as well as for running Open Models.”

   proof ↗⚑ flag

Multi agent

1. developerOrchestrate multiple agents — handoffs, subagents, or crews — inside one workflow

   weight 3 · round drawn
   OpenAI Agents SDK✓fullcommunity9/10

   Docs explicitly describe handoffs for delegating tasks between specialized agents, agents-as-tools for subagent-style composition without full handoff, and a Runner to orchestrate multi-agent workflows with tracing, streaming, and session memory across runs. Community feedback confirms real-world use for orchestration, though notes complexity concerns for advanced cases. Missing for 10: independent/hands-on benchmark of complex multi-agent crews at scale beyond simple handoff examples.

   • [claimed-docs] “Handoffs allow an agent to delegate tasks to another agent. This is particularly useful in scenarios where different agents specialize in di…”
   • [claimed-docs] “Agents as tools: expose an agent as a callable tool without a full handoff.”
   • [claimed-docs] “You can run agents via the Runner class. You have 3 options: Runner.run(), Runner.run_sync(), Runner.run_streamed()”
   • [claimed-docs] “The Agents SDK includes built-in tracing, collecting a comprehensive record of events during an agent run: LLM generations, tool calls, hand…”
   • [community] “OpenAI SDK is much simpler to use and understand than langchain etc. I havent deployed really complicated agents, but I have a number of sim…”

   proof ↗⚑ flag

   Google ADK✓fullclaimed9/10

   ADK explicitly supports multi-agent orchestration: workflows are defined as any application with more than one agent/node, with a graph-based Workflow Runtime supporting routing, fan-out/fan-in, loops, nested workflows, and a Task API for structured agent-to-agent delegation including multi-turn task mode and mixed delegation patterns; the CLI/Web UI explicitly supports multi-agent directories. missing for 10: independent third-party hands-on validation of complex multi-agent orchestration at scale.

   • [claimed-docs] “you can use the ADK development framework to expand them into workflows, which allow you to combine and orchestrate multiple agents and code…”
   • [claimed-docs] “In ADK, any agent application that has more than one agent or executable Node is considered a workflow.”
   • [github] “Workflow Runtime: A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan…”
   • [github] “Task API: Structured agent-to-agent delegation with multi-turn task mode, single-turn controlled output, mixed delegation patterns, human-in…”
   • [github] “Structured agent-to-agent delegation with multi-turn task mode, single-turn controlled output, mixed delegation patterns, human-in-the-loop,…”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”

   proof ↗⚑ flag

Workflow control

1. developerCompose agents into an explicit graph or workflow with branching, loops, and parallel steps

   weight 2 · round to Google ADK
   OpenAI Agents SDK~partialcommunity5/10

   The SDK provides composable primitives—handoffs for delegation/branching, agents-as-tools for parallel-style composition, and Runner.run for orchestrated execution—but these are implemented via plain Python control flow rather than an explicit graph/workflow DSL with declared branching, loops, and parallel steps as first-class constructs (unlike graph-based orchestrators). Community commentary notes users often reimplement custom logic for more complex flows rather than relying on a built-in graph abstraction. missing for 10: explicit graph/DSL construct for defining workflows, first-class loop/parallel-step primitives, independent hands-on evidence of complex branching/looping workflows.

   • [claimed-docs] “Handoffs allow an agent to delegate tasks to another agent. This is particularly useful in scenarios where different agents specialize in di…”
   • [claimed-docs] “Agents as tools: expose an agent as a callable tool without a full handoff.”
   • [claimed-docs] “You can run agents via the Runner class. You have 3 options: Runner.run(), Runner.run_sync(), Runner.run_streamed()”
   • [community] “you're better off just implementing the logic yourself as it is more flexible.”

   proof ↗⚑ flag

   Google ADK✓fullclaimed9/10

   ADK provides a dedicated graph-based Workflow Runtime with explicit support for routing, fan-out/fan-in (parallel), loops, retry, nested workflows, and dynamic nodes, plus structured Task API for agent delegation and workflow nodes—directly matching branching/loops/parallel composition; docs also describe 'graph-based architectures with explicit execution paths.' Missing for 10: independent/hands-on third-party validation beyond vendor docs and GitHub README.

   • [github] “Workflow Runtime: A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan…”
   • [github] “A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan-out/fan-in, loops…”
   • [github] “Task API: Structured agent-to-agent delegation with multi-turn task mode, single-turn controlled output, mixed delegation patterns, human-in…”
   • [github] “Structured agent-to-agent delegation with multi-turn task mode, single-turn controlled output, mixed delegation patterns, human-in-the-loop,…”
   • [claimed-docs] “Weave deterministic code with adaptive AI reasoning. Orchestrate complex tasks through structured, graph-based architectures, with explicit …”
   • [claimed-docs] “In ADK, any agent application that has more than one agent or executable Node is considered a workflow.”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round drawn
   OpenAI Agents SDK—none0/10

   No evidence in the pack mentions data residency, regional storage, or data location controls for the Agents SDK; the SDK is a developer framework that relies on underlying model providers/APIs for storage, and no such configuration is documented here.

   ⚑ flag

   Google ADK—none0/10

   ADK is a framework that can be deployed via Cloud Run, GKE, or self-hosted/offline (google-adk-docs-7, google-adk-docs-14, google-adk-docs-19), which implies developers control infrastructure location, but there is no explicit documentation about data residency, region selection, or storage location controls for agent data.

   • [claimed-docs] “This is a good option if you prefer to run offline or disconnected, or otherwise in a system that has no connection to Google Cloud.”
   • [claimed-docs] “Cloud Run is a managed auto-scaling compute platform on Google Cloud that enables you to run your agent as a container-based application.”
   • [claimed-docs] “GKE is a good option if you need more control over the deployment as well as for running Open Models.”

   proof ↗⚑ flag

2. ai-native userControl data retention and deletion

   weight 2 · round drawn
   OpenAI Agents SDK—none0/10

   The evidence pack covers agents, tools, tracing, sessions, handoffs, etc., but contains no documentation of data retention policies, data deletion controls, or privacy configuration options for the SDK. This is an applicable axis (an AI SDK could plausibly document retention/deletion controls or link to API data-usage policies) but no such evidence is present.

   ⚑ flag

   Google ADK—none0/10

   The evidence describes ADK as a self-hosted/deployable agent framework (Cloud Run, GKE, offline/disconnected deployment) but contains no documentation of explicit data retention policies, session/state deletion APIs, or user-facing controls for purging stored data. missing for 10: explicit retention/deletion controls, session data lifecycle docs, any privacy/compliance statements about stored artifacts or memory.

   • [claimed-docs] “This is a good option if you prefer to run offline or disconnected, or otherwise in a system that has no connection to Google Cloud.”
   • [claimed-docs] “ADK automatically filters irrelevant events, summarizes older conversational turns, lazy-loads artifacts, and tracks token usage.”

   proof ↗⚑ flag

3. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   OpenAI Agents SDK—none0/10

   The SDK ships built-in tracing that records LLM generations, tool calls, handoffs, and guardrails to a Traces dashboard, making telemetry opt-out a fair and relevant question, but no evidence pack item documents any environment variable, config flag, or API to disable tracing/telemetry.

   • [claimed-docs] “The Agents SDK includes built-in tracing, collecting a comprehensive record of events during an agent run: LLM generations, tool calls, hand…”
   • [claimed-docs] “Using the Traces dashboard, you can debug, visualize, and monitor your workflows during development and in production.”
   • [claimed-docs] “The Agents SDK includes built-in tracing, collecting a comprehensive record of events during an agent run: LLM generations, tool calls, hand…”

   proof ↗⚑ flag

   Google ADK—none0/10

   No evidence in the pack addresses telemetry collection or an opt-out mechanism for ADK; the docs cover agent building, deployment, evaluation, and workflows but never mention usage tracking or privacy controls. This is a fair axis for a developer framework/SDK, but absence of evidence means it counts as none. missing for 10: any mention of telemetry collection, an opt-out flag/env var, or a privacy policy describing data tracking.

   ⚑ flag

Durable state

1. developerCheckpoint agent state so a run can resume exactly where it left off after a crash or restart

   weight 3 · round to OpenAI Agents SDK
   OpenAI Agents SDK~partialcommunity5/10

   The SDK provides built-in session memory to persist conversation history across turns and a RunState/interruptions mechanism that lets a paused (e.g., approval-pending) run resume from where it left off, which covers part of the checkpoint/resume story. However, there is no documented mechanism for durably persisting full agent run state to survive a process crash or restart—community evidence explicitly notes that true durable/crash-resilient execution requires wrapping the SDK with an external engine like Temporal, implying it isn't a native capability. Missing for 10: first-party docs on serializing/restoring full run state after an unexpected crash, and any built-in persistence layer beyond conversation history/session memory.

   • [claimed-docs] “The Agents SDK provides built-in session memory to automatically maintain conversation history across multiple agent runs, eliminating the n…”
   • [claimed-docs] “When a tool call requires approval, the SDK pauses the run, returns interruptions, and lets you resume later from the same RunState.”
   • [claimed-docs] “When a tool call requires approval, the SDK pauses the run, returns `interruptions`, and lets you resume later from the same `RunState`.”
   • [community] “Comparing the Temporal-based durable version to the original OpenAI Agents SDK repo's examples, it's really not all that different - some me…”

   proof ↗⚑ flag

   Google ADK—none0/10

   Evidence only mentions generic 'state management' as one feature in the workflow runtime engine, with no documentation of session/state persistence, checkpointing, or resuming an agent run after a crash or restart. Missing for 10: explicit checkpoint/save-state API, resume-from-crash mechanism, persistence backend documentation, and any hands-on confirmation of durable resumption.

   • [github] “Workflow Runtime: A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan…”
   • [github] “A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan-out/fan-in, loops…”
   • [claimed-docs] “ADK automatically filters irrelevant events, summarizes older conversational turns, lazy-loads artifacts, and tracks token usage.”

   proof ↗⚑ flag

2. engineering-leadRun long-lived agents durably across process restarts and deploys, natively or via durable-execution integrations

   weight 2 · round to OpenAI Agents SDK
   OpenAI Agents SDK~partialcommunity4/10

   The SDK has built-in session memory across turns and human-in-the-loop pause/resume via RunState, plus a community reference to a Temporal-based durable-execution integration confirming feasibility, but there's no native, documented mechanism for surviving process restarts/deploys out of the box — durability requires an external integration like Temporal. missing for 10: first-party durable-execution/persistence docs, official Temporal (or similar) integration guide, evidence of native crash/restart recovery for long-lived agent runs.

   • [claimed-docs] “The Agents SDK provides built-in session memory to automatically maintain conversation history across multiple agent runs, eliminating the n…”
   • [claimed-docs] “When a tool call requires approval, the SDK pauses the run, returns interruptions, and lets you resume later from the same RunState.”
   • [claimed-docs] “When a tool call requires approval, the SDK pauses the run, returns `interruptions`, and lets you resume later from the same `RunState`.”
   • [community] “Comparing the Temporal-based durable version to the original OpenAI Agents SDK repo's examples, it's really not all that different - some me…”

   proof ↗⚑ flag

   Google ADK~partialclaimed3/10

   ADK's Workflow Runtime mentions 'state management' and 'retry' in its graph-based execution engine, and deployment docs describe scalable hosting (Agent Runtime, Cloud Run), but there is no explicit evidence of session/state persistence surviving process restarts or redeploys, nor any named durable-execution integration (e.g., Temporal, Cloud Workflows checkpointing). Missing for 10: documented durable state store or checkpoint/resume mechanism, explicit claim of surviving restarts/redeploys, and any third-party durable-execution integration.

   • [github] “Workflow Runtime: A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan…”
   • [claimed-docs] “Agent Runtime is a fully managed auto-scaling service on Google Cloud specifically designed for deploying, managing, and scaling AI agents b…”
   • [claimed-docs] “Cloud Run is a managed auto-scaling compute platform on Google Cloud that enables you to run your agent as a container-based application.”
   • [claimed-docs] “ADK automatically filters irrelevant events, summarizes older conversational turns, lazy-loads artifacts, and tracks token usage.”

   proof ↗⚑ flag

Streaming

1. developerStream tokens and intermediate agent events (tool calls, steps) to my UI in real time

   weight 3 · round to OpenAI Agents SDK
   OpenAI Agents SDK✓fullclaimed9/10

   Docs explicitly describe Runner.run_streamed() returning a RunResultStreaming, and a dedicated Streaming guide describing subscribing to updates of the agent run including partial responses and progress, plus built-in tracing capturing tool calls/handoffs/steps that can be surfaced. This directly supports streaming tokens and intermediate events to a UI. Missing for 10: independent hands-on developer confirmation of real-time UI integration beyond official docs.

   • [claimed-docs] “Streaming lets you subscribe to updates of the agent run as it proceeds. This can be useful for showing the end-user progress updates and pa…”
   • [claimed-docs] “You can run agents via the Runner class. You have 3 options: Runner.run(), Runner.run_sync(), Runner.run_streamed()”
   • [claimed-docs] “Runner.run_streamed(), which runs async and returns a RunResultStreaming”
   • [claimed-docs] “The Agents SDK includes built-in tracing, collecting a comprehensive record of events during an agent run: LLM generations, tool calls, hand…”

   proof ↗⚑ flag

   Google ADK~partialclaimed4/10

   The evidence shows a built-in Web/dev UI (`adk web`) for testing/debugging agents and a Callbacks mechanism to hook into execution-lifecycle events (tool calls, steps), which implies some visibility into intermediate agent activity, but nothing explicitly documents token-level streaming to a custom UI (no mention of SSE/websocket/streaming API). missing for 10: explicit documentation of real-time token streaming API/protocol, evidence of streaming tool-call/step events to an arbitrary UI beyond the built-in dev UI, independent confirmation of streaming behavior.

   • [github] “A built-in development UI to help you test, evaluate, debug, and showcase your agent(s).”
   • [github] “Web UI (supports multi-agent directories or pointing directly to a single agent folder)”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”
   • [claimed-docs] “Callbacks: Hook into specific events during an agent's execution lifecycle to add logging, monitoring, or custom side-effects without alteri…”

   proof ↗⚑ flag

Structured output

1. developerGet schema-validated structured output from an agent, with automatic retries when validation fails

   weight 3 · round to OpenAI Agents SDK
   OpenAI Agents SDK~partialclaimed5/10

   Docs confirm Pydantic-powered schema validation for function tools/outputs and guardrails for validating agent output, implying structured-output support, but there is no explicit documentation of an 'output_type' structured output feature with automatic retry-on-validation-failure behavior tied to streaming. Missing for 10: explicit documentation of structured output schema enforcement on agent final output, explicit automatic retry-on-validation-failure mechanism, and independent/hands-on confirmation that retries occur when validation fails.

   • [claimed-docs] “Function tools: Turn any Python function into a tool with automatic schema generation and Pydantic-powered validation.”
   • [claimed-docs] “Guardrails enable you to do checks and validations of user input and agent output.”
   • [claimed-docs] “An agent is a large language model (LLM) configured with instructions, tools, and optional runtime behavior such as handoffs, guardrails, an…”

   proof ↗⚑ flag

   Google ADK—none0/10

   No evidence in the pack mentions schema-validated structured output (e.g., Pydantic output_schema) or automatic retry-on-validation-failure behavior for ADK agents; the evidence covers agent setup, tools, workflows, deployment, and evaluation but not structured output validation. Missing for 10: any mention of output schema enforcement, structured output configuration, or validation-retry mechanism.

   ⚑ flag