CrewAI vs Google ADK

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to CrewAI
   CrewAI✓fullprobed9/10

   CrewAI's docs site serves a live llms.txt (probe confirms HTTP 200 with a full page index) and per-page .md variants, plus explicit doc instructions to copy a setup prompt that 'points the agent at the right docs' for Claude Code, Codex, Cursor, etc. missing for 10: no independent/community confirmation of an agent actually consuming llms.txt successfully in practice.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.crewai.com/llms.txt # CrewAI - [Edge (199 pages)](https://docs.crewai.com/_llms/en/edge.md): Docum…”
   • [probe] “PROBE docs-md: HTTP 200 at https://docs.crewai.com/en/installation.md > ## Documentation Index > Fetch the complete documentation index at: …”
   • [claimed-docs] “Copy a ready-to-paste setup prompt for Claude Code, Codex, Cursor, or any coding agent. It installs the official CrewAI skills, checks the C…”

   proof ↗⚑ flag

   Google ADK!disputedcontradicted3/10

   Docs claim 'AI-aware developer resources' and coding-assistant integration (google-adk-docs-9), suggesting agent-oriented documentation exists, but direct probes for llms.txt and markdown-rendered docs both return 404 (google-adk-probe-1, google-adk-probe-2), and no OpenAPI/machine-readable spec is discoverable (google-adk-probe-3), contradicting the claim that an agent can straightforwardly consume these docs. Missing for 10: a working llms.txt or agent-readable doc endpoint, confirmation that the 'AI-aware resources' are actually machine-fetchable rather than just a marketing phrase.

   • [claimed-docs] “ADK is designed to be written by both humans and AI. Connect your favorite coding assistant to our ADK developer Skills and AI-aware develop…”
   • [probe] “PROBE llms.txt: HTTP 404 at https://google.github.io/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://google.github.io/adk-docs/get-started/.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://google.github.io/openapi.json, https://google.github.io/swagger.json, https://google.github.…”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round drawn
   CrewAI✓fullprobed8/10

   CrewAI ships an official CLI (create/train/run/test/deploy) that is scriptable for CI, plus a testing subcommand (`crewai test`) for automated performance checks, and a REST API (kickoff/status/resume endpoints) enabling fully headless invocation from external automation systems. Missing for 10: no explicit CI pipeline example (e.g. GitHub Actions) or independent report confirming headless CI usage in the wild.

   • [claimed-docs] “The CrewAI CLI provides a set of commands to interact with CrewAI, allowing you to create, train, run, and manage crews & flows”
   • [claimed-docs] “We added the CLI command crewai test to make it easy to test your crew. This command will run your crew for a specified number of iterations…”
   • [claimed-docs] “We added the CLI command `crewai test` to make it easy to test your crew.”
   • [claimed-docs] “POST /resume](https://docs.crewai.com/edge/en/api-reference/resume.md): Resume crew execution with human feedback”
   • [claimed-docs] “POST /kickoff](https://docs.crewai.com/edge/en/api-reference/kickoff.md): Start a crew execution - [GET /status/{kickoff_id}](https://docs.c…”
   • [probe] “official CLI documented at https://docs.crewai.com/en/concepts/cli”

   proof ↗⚑ flag

   Google ADK✓fullclaimed8/10

   ADK provides a CLI (`adk run`, `adk web`, `adk eval`, `adk deploy docker`) that supports headless invocation and scripted evaluation, plus containerized deployment for CI/production pipelines. missing for 10: explicit CI pipeline examples (e.g. GitHub Actions), independent third-party confirmation of headless CI usage.

   • [github] “adk run path/to/my_agent”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”
   • [github] “adk eval \ samples_for_testing/hello_world \ samples_for_testing/hello_world/hello_world_eval_set_001.evalset.json”
   • [github] “adk deploy docker --with_ui <agent-folder>”
   • [claimed-docs] “You can manually package your Agent into a container image and then run it in any environment that supports container images.”
   • [claimed-docs] “This approach involves creating individual test files, each representing a single, simple agent-model interaction (a session).”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round drawn
   CrewAI✓fullclaimed8/10

   CrewAI documents a first-party MCP integration via the `mcps` field on agents, supporting both string references and structured configs for seamless MCP tool integration, directly matching the story. Missing for 10: independent/hands-on community corroboration of MCP usage and more detail on connection/auth mechanics beyond the DSL description.

   • [claimed-docs] “Use the mcps field directly on agents for seamless MCP tool integration. The DSL supports both string references... and structured configura…”
   • [claimed-docs] “Use the `mcps` field directly on agents for seamless MCP tool integration.”
   • [claimed-docs] “Use the `mcps` field directly on agents for seamless MCP tool integration. The DSL supports both **string references** (for quick setup) and…”
   • [claimed-docs] “Use the `mcps` field directly on agents for seamless MCP tool integration. The DSL supports both **string references** (for quick setup) and…”

   proof ↗⚑ flag

   Google ADK✓fullclaimed8/10

   Docs explicitly state an ADK agent can act as an MCP client and use tools provided by external MCP servers, directly matching the story. missing for 10: independent/hands-on corroboration beyond first-party docs, and more detail on multi-server configuration or auth handling.

   • [claimed-docs] “An ADK agent can act as an MCP client and use tools provided by external MCP servers.”
   • [claimed-docs] “Exposing ADK Tools via an MCP Server: How to build an MCP server that wraps ADK tools, making them accessible to any MCP client.”
   • [claimed-docs] “How to build an MCP server that wraps ADK tools, making them accessible to any MCP client.”

   proof ↗⚑ flag

4. ai-native userConnect an agent via an official MCP server

   weight 3 · round to Google ADK
   CrewAI—none0/10

   CrewAI documents only client-side MCP integration (an `mcps` field letting CrewAI agents call out to external MCP servers), but there is no evidence of CrewAI itself exposing an official MCP server that other agents could connect to. As a framework/platform, this axis applies, but absence of a documented server-mode offering means it is not delivered.

   • [claimed-docs] “Use the mcps field directly on agents for seamless MCP tool integration. The DSL supports both string references... and structured configura…”
   • [claimed-docs] “Use the `mcps` field directly on agents for seamless MCP tool integration.”
   • [claimed-docs] “Use the `mcps` field directly on agents for seamless MCP tool integration. The DSL supports both **string references** (for quick setup) and…”
   • [claimed-docs] “Use the `mcps` field directly on agents for seamless MCP tool integration. The DSL supports both **string references** (for quick setup) and…”

   proof ↗⚑ flag

   Google ADK~partialclaimed6/10

   ADK's official docs explicitly document how to expose ADK tools via an MCP server ('build an MCP server that wraps ADK tools, making them accessible to any MCP client'), showing the framework supports the server side of MCP, not just being an MCP client. However, this is a build-your-own-server guide rather than a turnkey, pre-hosted official MCP endpoint, so it requires developer setup work. Missing for 10: a ready-made hosted/official MCP server endpoint, independent hands-on confirmation that the generated server works reliably with third-party MCP clients.

   • [claimed-docs] “Exposing ADK Tools via an MCP Server: How to build an MCP server that wraps ADK tools, making them accessible to any MCP client.”
   • [claimed-docs] “How to build an MCP server that wraps ADK tools, making them accessible to any MCP client.”
   • [claimed-docs] “An ADK agent can act as an MCP client and use tools provided by external MCP servers.”

   proof ↗⚑ flag

5. ai-native userUse an official CLI

   weight 2 · round drawn
   CrewAI✓fullprobed8/10

   CrewAI ships an official CLI (installed via `uv tool install crewai`) documented to create, train, test, and run crews & flows, with dedicated commands like `crewai test`, confirmed by both docs and a live probe of the docs page. missing for 10: independent/hands-on third-party validation of the CLI itself (community evidence covers the framework broadly, not CLI usage specifically) and no changelog/version history showing CLI maturity over time.

   • [claimed-docs] “The CrewAI CLI provides a set of commands to interact with CrewAI, allowing you to create, train, run, and manage crews & flows”
   • [claimed-docs] “We added the CLI command crewai test to make it easy to test your crew. This command will run your crew for a specified number of iterations…”
   • [claimed-docs] “Run the following command to install crewai CLI: uv tool install crewai”
   • [claimed-docs] “Create a new crew, flow, tool, skill, or template project.”
   • [claimed-docs] “These older commands still work but print a yellow deprecation warning.”
   • [probe] “official CLI documented at https://docs.crewai.com/en/concepts/cli”

   proof ↗⚑ flag

   Google ADK✓fullclaimed8/10

   ADK ships an official CLI (`adk run`, `adk web`, `adk eval`, `adk deploy docker`) documented in the GitHub repo with concrete command examples, plus docs reference an "Agents CLI" for scaffolding/build/test/deploy workflows tailored to AI-native/agentic use. Missing for 10: independent third-party hands-on review of the CLI's AI-native ergonomics beyond first-party docs/repo.

   • [github] “adk run path/to/my_agent”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”
   • [github] “adk eval \ samples_for_testing/hello_world \ samples_for_testing/hello_world/hello_world_eval_set_001.evalset.json”
   • [github] “adk deploy docker --with_ui <agent-folder>”
   • [claimed-docs] “Go from idea to coded ADK agent in minutes. Use your favorite AI-enabled developer environment to scaffold, build, test, evaluate, and deplo…”
   • [claimed-docs] “Migrate existing agents and workflows to ADK with Agents CLI.”

   proof ↗⚑ flag

6. ai-native userDrive the product through a documented public API

   weight 3 · round drawn
   CrewAI~partialprobed6/10

   CrewAI documents a REST API for driving crew execution (POST /kickoff, GET /status, POST /resume) under its Enterprise/Edge docs, and the whole framework is Python-native and scriptable via CLI/SDK, giving AI-native users multiple documented entry points. However, this API surface is only exposed for Enterprise (AMP) deployments rather than a universally public API, and a direct probe for a discoverable OpenAPI/Swagger spec returned 404s, indicating no formal machine-readable API contract. Missing for 10: a public OpenAPI/swagger spec, confirmation the REST API is available outside the paid Enterprise tier, and independent hands-on confirmation of API usage.

   • [claimed-docs] “POST /resume](https://docs.crewai.com/edge/en/api-reference/resume.md): Resume crew execution with human feedback”
   • [claimed-docs] “POST /kickoff](https://docs.crewai.com/edge/en/api-reference/kickoff.md): Start a crew execution - [GET /status/{kickoff_id}](https://docs.c…”
   • [claimed-docs] “Deploy your crews to a managed infrastructure with a few clicks... Access your deployed crews via REST API for integration with existing sys…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.crewai.com/openapi.json, https://docs.crewai.com/swagger.json, https://docs.crewai.com/…”
   • [claimed-docs] “The CrewAI CLI provides a set of commands to interact with CrewAI, allowing you to create, train, run, and manage crews & flows”

   proof ↗⚑ flag

   Google ADK~partialprobed6/10

   ADK is a Python framework/CLI (adk run, adk web, adk eval, adk deploy) with documented programmatic APIs for building and driving agents, plus MCP client/server support, but there is no evidence of a formal public REST/OpenAPI-style API surface — probes for openapi/swagger specs and llms.txt all 404. missing for 10: a documented public HTTP/OpenAPI API spec, independent third-party confirmation of programmatic drivability beyond first-party docs.

   • [claimed-docs] “Create your first Python ADK agent in minutes.”
   • [github] “adk run path/to/my_agent”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”
   • [github] “adk eval \ samples_for_testing/hello_world \ samples_for_testing/hello_world/hello_world_eval_set_001.evalset.json”
   • [github] “adk deploy docker --with_ui <agent-folder>”
   • [probe] “PROBE openapi: all candidate paths 404 (https://google.github.io/openapi.json, https://google.github.io/swagger.json, https://google.github.…”
   • [probe] “PROBE llms.txt: HTTP 404 at https://google.github.io/llms.txt”

   proof ↗⚑ flag

7. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round drawn
   CrewAI—none0/10

   No documentation or evidence shows CrewAI issuing scoped/least-privilege API credentials per agent; tools/LLM/MCP integration docs describe capability wiring but not credential scoping. A community comment even notes agents typically run with 'full credentials' access, reinforcing the absence of this feature.

   • [community] “On a CrewAI-based personal email agent: 'Local personal agents are going to be a major app type... They run locally on my computer with my f…”
   • [claimed-docs] “CrewAI tools empower agents with capabilities ranging from web searching and data analysis to collaboration and delegating tasks among cowor…”
   • [claimed-docs] “A tool in CrewAI is a skill or function that agents can utilize to perform various actions. This includes tools from the [CrewAI Toolkit] an…”

   proof ↗⚑ flag

   Google ADK—none0/10

   No evidence in the pack describes issuing scoped or least-privilege API credentials/tokens for agents; the docs cover tools, MCP, workflows, deployment, and evaluation but nothing about credential scoping or permission management for agent identities.

   ⚑ flag

8. ai-native userBuild against official SDKs

   weight 2 · round to Google ADK
   CrewAI~partialprobed6/10

   CrewAI is itself a Python SDK/framework for building agents (Agent, Task, Crew, Flow classes) and integrates with LLM provider native SDKs, plus a REST API for kickoff/status/resume, which supports SDK-style programmatic access. However, there is no evidence of official SDKs in multiple languages (e.g., JS/TS, Go) beyond Python, no versioned SDK release notes, and no independent corroboration of SDK stability/quality from third-party developers. missing for 10: multi-language official SDKs, independent developer corroboration of SDK reliability, formal SDK versioning/changelog beyond docs.

   • [claimed-docs] “CrewAI integrates with multiple LLM providers through providers native sdks, giving you the flexibility to choose the right model for your s…”
   • [claimed-docs] “CrewAI integrates with multiple LLM providers through providers native sdks, giving you the flexibility to choose the right model for your s…”
   • [claimed-docs] “POST /resume](https://docs.crewai.com/edge/en/api-reference/resume.md): Resume crew execution with human feedback”
   • [claimed-docs] “POST /kickoff](https://docs.crewai.com/edge/en/api-reference/kickoff.md): Start a crew execution - [GET /status/{kickoff_id}](https://docs.c…”
   • [claimed-docs] “Build collaborative AI agents, crews, and flows — production ready from day one.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.crewai.com/openapi.json, https://docs.crewai.com/swagger.json, https://docs.crewai.com/…”

   proof ↗⚑ flag

   Google ADK✓fullclaimed9/10

   Google ADK is itself an official Python SDK/framework with extensive first-party documentation, code examples, CLI tooling (adk run/web/eval/deploy), and a public GitHub repo, giving AI-native developers a fully documented, official SDK to build against. Minor gap — missing for 10: independent third-party corroboration beyond vendor docs/repo, and llms.txt/OpenAPI probes returned 404s suggesting some machine-readable doc surfaces are incomplete.

   • [claimed-docs] “Create your first Python ADK agent in minutes.”
   • [claimed-docs] “Building an agent with just a model, instructions, and tools is a great place to start for most developers.”
   • [claimed-docs] “agent = Agent( name="researcher", model="gemini-flash-latest", instruction="You help users research topics thoroughly.", too…”
   • [github] “Agent Config: Build agents without code.”
   • [github] “adk run path/to/my_agent”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”
   • [claimed-docs] “ADK is designed to be written by both humans and AI. Connect your favorite coding assistant to our ADK developer Skills and AI-aware develop…”

   proof ↗⚑ flag

9. ai-native userSubscribe to events via webhooks

   weight 2 · round to CrewAI
   CrewAI~partialclaimed5/10

   CrewAI's enterprise tier (AMP) explicitly advertises 'Webhook Streaming: Stream real-time events and updates to your systems' alongside a REST API for kickoff/status/resume, indicating a webhook subscription mechanism exists. However, this is gated behind the paid CrewAI AMP/Enterprise product rather than the open-source core, and no detail is given on event types, subscription setup, or payload schema. Missing for 10: documentation of specific webhook event types/payloads, open-source (non-enterprise) support, and independent/hands-on confirmation the webhook feature works as described.

   • [claimed-docs] “Webhook Streaming Stream real-time events and updates to your systems”
   • [claimed-docs] “Stream real-time events and updates to your systems”
   • [claimed-docs] “Deploy your crews to a managed infrastructure with a few clicks... Access your deployed crews via REST API for integration with existing sys…”
   • [claimed-docs] “POST /resume](https://docs.crewai.com/edge/en/api-reference/resume.md): Resume crew execution with human feedback”
   • [claimed-docs] “POST /kickoff](https://docs.crewai.com/edge/en/api-reference/kickoff.md): Start a crew execution - [GET /status/{kickoff_id}](https://docs.c…”

   proof ↗⚑ flag

   Google ADK—none0/10

   ADK's evidence shows only in-process callbacks/hooks for agent execution lifecycle, not an external webhook subscription mechanism; no docs mention registering webhook URLs or event push notifications. Missing for 10: any webhook registration API, outbound event delivery docs, or third-party confirmation of webhook support.

   • [claimed-docs] “Callbacks: Hook into specific events during an agent's execution lifecycle to add logging, monitoring, or custom side-effects without alteri…”

   proof ↗⚑ flag

Agentic features

1. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to CrewAI
   CrewAI✓fullcommunity7/10

   CrewAI agents/crews/flows are explicitly designed as autonomous units that make decisions, run tasks sequentially or hierarchically, and can be deployed to managed infrastructure with monitoring, webhooks, and REST API kickoff/resume endpoints—enabling background autonomous execution. Community evidence (HN Gmail automation example) confirms real-world use of CrewAI for unattended background automations. missing for 10: independent verification of long-running/scheduled background execution reliability, and no first-party docs on scheduling/triggers beyond kickoff API and webhooks.

   • [claimed-docs] “In the CrewAI framework, an `Agent` is an autonomous unit that can: * Perform specific tasks * Make decisions based on its role and goal * U…”
   • [claimed-docs] “A crew in crewAI represents a collaborative group of agents working together to achieve a set of tasks.”
   • [claimed-docs] “Flows allow developers to combine and coordinate coding tasks and Crews efficiently, providing a robust framework for building sophisticated…”
   • [claimed-docs] “Deploy your crews to a managed infrastructure with a few clicks... Access your deployed crews via REST API for integration with existing sys…”
   • [claimed-docs] “Webhook Streaming Stream real-time events and updates to your systems”
   • [claimed-docs] “POST /resume](https://docs.crewai.com/edge/en/api-reference/resume.md): Resume crew execution with human feedback”
   • [claimed-docs] “POST /kickoff](https://docs.crewai.com/edge/en/api-reference/kickoff.md): Start a crew execution - [GET /status/{kickoff_id}](https://docs.c…”
   • [community] “Show HN: built AI agents with CrewAI to automate an old Gmail account—categorizing emails, drafting replies, Slack alerts for high-priority …”

   proof ↗⚑ flag

   Google ADK~partialclaimed6/10

   ADK supports deployable, auto-scaling agent runtimes (Cloud Run, GKE, Agent Runtime) and workflow orchestration with retries, state, and scheduling-like execution (fan-out/fan-in, loops), enabling agents to run unattended once deployed. However, evidence does not show explicit scheduling/triggers (e.g., cron-like autonomous kick-off) or a dedicated 'background automation' mode distinct from deployment. missing for 10: explicit trigger/schedule mechanism for autonomous background runs, independent evidence of long-running unattended operation, and confirmation of persistent background execution outside a deploy/response cycle.

   • [claimed-docs] “Agent Runtime is a fully managed auto-scaling service on Google Cloud specifically designed for deploying, managing, and scaling AI agents b…”
   • [claimed-docs] “Cloud Run is a managed auto-scaling compute platform on Google Cloud that enables you to run your agent as a container-based application.”
   • [claimed-docs] “GKE is a good option if you need more control over the deployment as well as for running Open Models.”
   • [github] “Workflow Runtime: A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan…”
   • [github] “A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan-out/fan-in, loops…”
   • [claimed-docs] “In ADK, any agent application that has more than one agent or executable Node is considered a workflow.”

   proof ↗⚑ flag

2. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round to CrewAI
   CrewAI✓fullclaimed8/10

   CrewAI's core framework is built around delegating tasks to AI agents: Tasks are assigned to Agents (docs-2/34), agents can 'Delegate tasks when allowed' and 'collaborate with other agents' (docs-46), and tools support 'delegating tasks among coworkers' (docs-6/23), with hierarchical process management explicitly supporting delegation chains (docs-3/48). Missing for 10: no independent hands-on account specifically validating delegation-quality/reliability in production use (community threads focus on other aspects like email agent risk, not delegation mechanics), and delegation requires developer configuration rather than being a ready-made single assistant.

   • [claimed-docs] “a "Task" is a specific assignment completed by an "Agent". Tasks provide all necessary details for execution, such as a description, the age…”
   • [claimed-docs] “a `Task` is a specific assignment completed by an `Agent`. Tasks provide all necessary details for execution, such as a description, the age…”
   • [claimed-docs] “Perform specific tasks * Make decisions based on its role and goal * Use tools to accomplish objectives * Communicate and collaborate with o…”
   • [claimed-docs] “CrewAI tools empower agents with capabilities ranging from web searching and data analysis to collaboration and delegating tasks among cowor…”
   • [claimed-docs] “CrewAI tools empower agents with capabilities ranging from web searching and data analysis to collaboration and delegating tasks among cowor…”
   • [claimed-docs] “process = Process.sequential # or Process.hierarchical”
   • [claimed-docs] “Sequential: Tasks are executed in the order they are defined * Hierarchical: Tasks are as”
   • [claimed-docs] “A crew in crewAI represents a collaborative group of agents working together to achieve a set of tasks.”

   proof ↗⚑ flag

   Google ADK—none0/10

   ADK is a framework for building agents that developers run themselves; the docs explicitly describe connecting *external* coding assistants (e.g., 'Connect your favorite coding assistant to our ADK developer Skills') rather than shipping a built-in AI assistant that end-users delegate tasks to inside the product itself. No evidence shows ADK embedding its own persistent assistant persona for task delegation.

   • [claimed-docs] “ADK is designed to be written by both humans and AI. Connect your favorite coding assistant to our ADK developer Skills and AI-aware develop…”
   • [claimed-docs] “Go from idea to coded ADK agent in minutes. Use your favorite AI-enabled developer environment to scaffold, build, test, evaluate, and deplo…”

   proof ↗⚑ flag

3. ai-native userOperate the product with natural-language commands

   weight 2 · round to Google ADK
   CrewAI—none0/10

   CrewAI is operated via Python code, YAML config, and a traditional CLI (create/train/run/test) or a drag-and-drop Visual Builder — none of which constitute natural-language command operation of the product itself. Evidence shows agents can be given natural-language goals within a crew, but that's the framework's internal task model, not the user's interface to the product.

   • [claimed-docs] “The CrewAI CLI provides a set of commands to interact with CrewAI, allowing you to create, train, run, and manage crews & flows”
   • [claimed-docs] “Create a new crew, flow, tool, skill, or template project.”
   • [claimed-docs] “CrewAI AMP includes a Visual Agent Builder that simplifies agent creation and configuration without writing code. Design your agents visuall…”
   • [claimed-docs] “The Visual Task Builder enables: Drag-and-drop task creation, Visual task dependencies and flow, Real-time testing and validation”
   • [claimed-docs] “Crew Studio Create and customize crews using a no-code/low-code interface”

   proof ↗⚑ flag

   Google ADK~partialclaimed5/10

   ADK docs claim it is designed to be built and operated via AI coding assistants (Agent Config for no-code agent building, 'Agents CLI' for AI-enabled dev environments to scaffold/build/test/deploy) which supports some natural-language-driven operation, but the primary operating surface is a traditional CLI (adk run/web/eval/deploy) and Python code, not direct NL commands to the tool itself. Missing for 10: concrete example of natural-language command controlling ADK end-to-end, independent/hands-on confirmation that Agent Config or coding-assistant integration works as a full NL interface.

   • [claimed-docs] “ADK is designed to be written by both humans and AI. Connect your favorite coding assistant to our ADK developer Skills and AI-aware develop…”
   • [claimed-docs] “Go from idea to coded ADK agent in minutes. Use your favorite AI-enabled developer environment to scaffold, build, test, evaluate, and deplo…”
   • [github] “Agent Config: Build agents without code.”
   • [github] “Agent Config: Build agents without code. Check out the Agent Config feature.”
   • [github] “Build agents without code. Check out the Agent Config feature.”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   CrewAI—none0/10

   Evidence shows only static API reference pages (e.g., kickoff/status/resume endpoints) and markdown-based docs, not an interactive, runnable API explorer. A probe explicitly found no OpenAPI/Swagger spec at standard locations, indicating no interactive console for testing API calls exists.

   • [claimed-docs] “POST /resume](https://docs.crewai.com/edge/en/api-reference/resume.md): Resume crew execution with human feedback”
   • [claimed-docs] “POST /kickoff](https://docs.crewai.com/edge/en/api-reference/kickoff.md): Start a crew execution - [GET /status/{kickoff_id}](https://docs.c…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.crewai.com/openapi.json, https://docs.crewai.com/swagger.json, https://docs.crewai.com/…”

   proof ↗⚑ flag

   Google ADK—none0/10

   The evidence pack shows standard docs, code snippets, and CLI examples, but no interactive/runnable API reference (e.g., a Swagger/OpenAPI explorer or live code sandbox); probes for openapi.json and similar endpoints explicitly returned 404s.

   • [probe] “PROBE openapi: all candidate paths 404 (https://google.github.io/openapi.json, https://google.github.io/swagger.json, https://google.github.…”
   • [probe] “PROBE docs-md: HTTP 404 at https://google.github.io/adk-docs/get-started/.md”
   • [claimed-docs] “agent = Agent( name="researcher", model="gemini-flash-latest", instruction="You help users research topics thoroughly.", too…”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   CrewAI—none0/10

   CrewAI documents REST-style API endpoints (kickoff, status, resume) for its Enterprise/Edge offering, suggesting an API surface exists, but a direct probe for machine-readable spec files (openapi.json, swagger.json, etc.) returned 404 on all candidate paths, and no documentation links to a downloadable OpenAPI/Swagger spec.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.crewai.com/openapi.json, https://docs.crewai.com/swagger.json, https://docs.crewai.com/…”
   • [claimed-docs] “POST /resume](https://docs.crewai.com/edge/en/api-reference/resume.md): Resume crew execution with human feedback”
   • [claimed-docs] “POST /kickoff](https://docs.crewai.com/edge/en/api-reference/kickoff.md): Start a crew execution - [GET /status/{kickoff_id}](https://docs.c…”

   proof ↗⚑ flag

   Google ADK—none0/10

   No evidence of a downloadable OpenAPI/machine-readable spec for ADK; explicit probes for openapi.json/swagger.json and llms.txt all return 404, indicating no such spec is published.

   • [probe] “PROBE llms.txt: HTTP 404 at https://google.github.io/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://google.github.io/adk-docs/get-started/.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://google.github.io/openapi.json, https://google.github.io/swagger.json, https://google.github.…”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round to Google ADK
   CrewAI—none0/10

   CrewAI's `crewai test` command (crewai-docs-9/27/40) runs performance-metric iterations on a crew, but no evidence describes a distinct sandbox environment or mechanism to isolate test runs from production data/systems. Enterprise deployment docs mention managed infra and APIs but nothing about sandbox vs. production data separation.

   ⚑ flag

   Google ADK~partialclaimed5/10

   ADK supports local dev/test workflows (adk run, adk web, adk eval, local evaluation with test files and eval sets) that inherently run against a local/dev environment rather than production, and offline/disconnected deployment is mentioned. However, there's no explicit documentation of a dedicated 'sandbox' environment or data isolation guarantee distinct from production. missing for 10: explicit sandbox/staging environment docs, explicit statement that test runs are isolated from production data/state, independent confirmation of this isolation.

   • [claimed-docs] “This approach involves creating individual test files, each representing a single, simple agent-model interaction (a session).”
   • [claimed-docs] “This approach involves creating individual test files, each representing a single, simple agent-model interaction (a session). It's most eff…”
   • [claimed-docs] “Expected Intermediate Tool Use Trajectory: The tool calls we expect the agent to make in order to respond correctly to the user query.”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”
   • [github] “adk eval \ samples_for_testing/hello_world \ samples_for_testing/hello_world/hello_world_eval_set_001.evalset.json”
   • [claimed-docs] “This is a good option if you prefer to run offline or disconnected, or otherwise in a system that has no connection to Google Cloud.”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round to CrewAI
   CrewAI~partialprobed3/10

   There is only glancing evidence: CLI commands show deprecation warnings when replaced (crewai-docs-26), and the docs site exposes multiple version snapshots (Edge, v1.15.21) suggesting some versioning of releases (crewai-probe-1). However, no formal documented deprecation policy for the REST/enterprise API or SDK, no version-numbering scheme (e.g., v1/v2 REST endpoints), and no OpenAPI spec was found (crewai-probe-3 shows 404s for openapi.json), meaning API contracts aren't clearly versioned or governed by a public deprecation timeline. Missing for 10: explicit API versioning scheme (e.g., /v1/, /v2/ endpoints), a published deprecation/sunset policy document, and a machine-readable API spec (OpenAPI) confirming version support windows.

   • [claimed-docs] “These older commands still work but print a yellow deprecation warning.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.crewai.com/llms.txt # CrewAI - [Edge (199 pages)](https://docs.crewai.com/_llms/en/edge.md): Docum…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.crewai.com/openapi.json, https://docs.crewai.com/swagger.json, https://docs.crewai.com/…”

   proof ↗⚑ flag

   Google ADK—none0/10

   No evidence in the pack mentions API versioning schemes or a documented deprecation policy for ADK; probes for OpenAPI specs and related docs all returned 404s. missing for 10: versioning scheme documentation, deprecation policy, changelog/migration guides, semantic versioning commitments.

   • [probe] “PROBE llms.txt: HTTP 404 at https://google.github.io/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://google.github.io/adk-docs/get-started/.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://google.github.io/openapi.json, https://google.github.io/swagger.json, https://google.github.…”

   proof ↗⚑ flag

Agent authoring

1. developerDefine an agent with typed custom tools in a few lines of code

   weight 3 · round drawn
   CrewAI~partialclaimed6/10

   CrewAI docs show agents can be defined with tools and integrate with CrewAI Toolkit/LangChain tools (crewai-docs-6, crewai-docs-52), and tools are described as skills/functions agents can use, implying a simple decorator/class-based definition, but the evidence pack lacks a concrete code example showing typed tool schemas (e.g., pydantic-based args_schema) or a minimal few-line snippet demonstrating this end-to-end. missing for 10: an explicit few-line code sample defining a typed custom tool (with input schema) and attaching it to an agent, plus independent/hands-on confirmation of ease-of-use for typed tools.

   • [claimed-docs] “CrewAI tools empower agents with capabilities ranging from web searching and data analysis to collaboration and delegating tasks among cowor…”
   • [claimed-docs] “A tool in CrewAI is a skill or function that agents can utilize to perform various actions. This includes tools from the [CrewAI Toolkit] an…”
   • [claimed-docs] “In the CrewAI framework, an `Agent` is an autonomous unit that can: * Perform specific tasks * Make decisions based on its role and goal * U…”
   • [claimed-docs] “Perform specific tasks * Make decisions based on its role and goal * Use tools to accomplish objectives * Communicate and collaborate with o…”

   proof ↗⚑ flag

   Google ADK~partialclaimed6/10

   The docs show a concrete few-line example of defining an Agent with a model, instructions, and a tools list (google-adk-docs-22), and other docs confirm tools are a core, optional component of agent definition (google-adk-docs-2, google-adk-docs-13). However, the evidence never shows a custom Python tool function with type hints/typed parameters being defined and passed in — only a prebuilt tool (google_search) is used in the example. Missing for 10: an explicit example of writing a custom typed tool function, and documentation of automatic schema/type inference from function signatures.

   • [claimed-docs] “agent = Agent( name="researcher", model="gemini-flash-latest", instruction="You help users research topics thoroughly.", too…”
   • [claimed-docs] “Building an agent with just a model, instructions, and tools is a great place to start for most developers.”
   • [claimed-docs] “The basic components of an Agent are an artificial intelligence (AI) model, task instructions, and optionally, a set of tools to be used by …”

   proof ↗⚑ flag

Ai buildability

1. ai-native userHave a coding agent scaffold a new agent project from an official CLI or template in one command

   weight 2 · round drawn
   CrewAI✓fullprobed8/10

   CrewAI ships an official CLI (`crewai create`) to scaffold new crew/flow/tool/template projects in one command, installed via `uv tool install crewai`, and docs even provide a ready-to-paste setup prompt for coding agents like Claude Code/Cursor/Codex to bootstrap and drive the CLI. This directly matches the story of an ai-native user having a coding agent scaffold a new agent project from an official CLI. missing for 10: independent/hands-on confirmation of the scaffold command's actual output structure and no community verification of the agent-driven setup-prompt workflow.

   • [claimed-docs] “The CrewAI CLI provides a set of commands to interact with CrewAI, allowing you to create, train, run, and manage crews & flows”
   • [claimed-docs] “Create a new crew, flow, tool, skill, or template project.”
   • [claimed-docs] “Run the following command to install crewai CLI: uv tool install crewai”
   • [claimed-docs] “Copy a ready-to-paste setup prompt for Claude Code, Codex, Cursor, or any coding agent. It installs the official CrewAI skills, checks the C…”
   • [probe] “official CLI documented at https://docs.crewai.com/en/concepts/cli”

   proof ↗⚑ flag

   Google ADK✓fullclaimed8/10

   ADK docs explicitly advertise an official 'Agents CLI' to scaffold, build, test, evaluate, and deploy agents in minutes, and the GitHub README shows concrete one-line commands (adk run, adk web, adk deploy) plus a no-code 'Agent Config' template feature for scaffolding agents. This directly matches the ai-native scaffolding story via an official CLI/template workflow. Missing for 10: independent/hands-on confirmation of the one-command scaffold experience beyond first-party docs.

   • [claimed-docs] “Go from idea to coded ADK agent in minutes. Use your favorite AI-enabled developer environment to scaffold, build, test, evaluate, and deplo…”
   • [claimed-docs] “Migrate existing agents and workflows to ADK with Agents CLI.”
   • [github] “adk run path/to/my_agent”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”
   • [github] “Agent Config: Build agents without code. Check out the Agent Config feature.”
   • [github] “Build agents without code. Check out the Agent Config feature.”

   proof ↗⚑ flag

2. ai-native userRun the framework's example agents headlessly from a terminal so an agent can verify what it just built

   weight 2 · round to Google ADK
   CrewAI~partialprobed5/10

   CrewAI ships a CLI (`crewai run`, `crewai test`) that can run crews/agents headlessly from a terminal and provide performance metrics, which an agent could invoke to verify its own build, but there is no documented 'example agents' bundled specifically for self-verification nor evidence of an agent-oriented headless verification workflow. missing for 10: a documented example/template agent meant for headless self-check, explicit guidance on running examples non-interactively for verification purposes, and independent confirmation of this exact workflow.

   • [claimed-docs] “The CrewAI CLI provides a set of commands to interact with CrewAI, allowing you to create, train, run, and manage crews & flows”
   • [claimed-docs] “We added the CLI command crewai test to make it easy to test your crew. This command will run your crew for a specified number of iterations…”
   • [claimed-docs] “Create a new crew, flow, tool, skill, or template project.”
   • [claimed-docs] “The CrewAI CLI provides a set of commands to interact with CrewAI, allowing you to create, train, run, and manage crews & flows.”
   • [claimed-docs] “We added the CLI command `crewai test` to make it easy to test your crew. This command will run your crew for a specified number of iteratio…”
   • [probe] “official CLI documented at https://docs.crewai.com/en/concepts/cli”

   proof ↗⚑ flag

   Google ADK✓fullclaimed7/10

   ADK provides a documented CLI (`adk run path/to/my_agent`) to run agents headlessly from a terminal, plus `adk eval` for automated verification of agent behavior against eval sets, matching the 'verify what it just built' use case for an ai-native/agentic workflow. Missing for 10: explicit confirmation that shipped 'example agents' (vs. user-authored ones) work with this flow, and independent/hands-on corroboration beyond the official repo docs.

   • [github] “adk run path/to/my_agent”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”
   • [github] “adk eval \ samples_for_testing/hello_world \ samples_for_testing/hello_world/hello_world_eval_set_001.evalset.json”
   • [claimed-docs] “This approach involves creating individual test files, each representing a single, simple agent-model interaction (a session).”

   proof ↗⚑ flag

3. ai-native userRely on strict typing and schema validation so a coding agent catches its own mistakes at build time

   weight 2 · round drawn
   CrewAI—none0/10

   The evidence pack covers CrewAI's agents, tasks, flows, memory, CLI, and enterprise features but contains no mention of strict typing, schema validation, or build-time error catching for agent outputs or task definitions. Nothing describes Pydantic-based output schemas, type-checked task I/O, or static validation tooling that would let an agent catch its own mistakes before runtime.

   ⚑ flag

   Google ADK—none0/10

   The evidence pack covers ADK's agent orchestration, deployment, and evaluation features, but contains no mention of strict typing, schema validation, or build-time error detection for tool/agent definitions — the evaluation features described (docs-20, docs-21, docs-25) are runtime test-set based, not compile/build-time type checks.

   ⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to CrewAI
   CrewAI~partialcommunity5/10

   CrewAI's task/flow/crew orchestration model supports building agents that iterate over many items programmatically, and a documented community example shows exactly this — an agent bulk-categorizing, drafting, and auto-deleting many emails in an inbox (crewai-comm-1). However, there is no first-party documentation of a dedicated bulk/batch-processing API (e.g., a 'run for each item' or batch endpoint) in the evidence pack, only sequential/hierarchical single-task execution concepts. missing for 10: explicit bulk/batch API or 'for-each' primitive in official docs, independent benchmarks of bulk-scale runs, guardrails/rate-limit guidance for large batch jobs.

   • [community] “Show HN: built AI agents with CrewAI to automate an old Gmail account—categorizing emails, drafting replies, Slack alerts for high-priority …”
   • [claimed-docs] “process = Process.sequential # or Process.hierarchical”
   • [claimed-docs] “Sequential: Tasks are executed in the order they are defined * Hierarchical: Tasks are as”
   • [claimed-docs] “Flows allow you to create structured, event-driven workflows. They provide a seamless way to connect multiple tasks, manage state, and contr…”

   proof ↗⚑ flag

   Google ADK~partialclaimed3/10

   ADK's Workflow Runtime offers fan-out/fan-in and loop constructs that could be used by developers to build bulk-item processing pipelines, but there is no documented built-in 'bulk operations' feature or example for end users acting across many items at once. Missing for 10: explicit bulk-operation tooling/UI, documented examples of processing many items in one call, and evidence of end-user (not just developer-framework) bulk workflows.

   • [github] “Workflow Runtime: A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan…”
   • [claimed-docs] “you can use the ADK development framework to expand them into workflows, which allow you to combine and orchestrate multiple agents and code…”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to Google ADK
   CrewAI~partialcommunity6/10

   CrewAI's Flows provide event-driven workflow orchestration (state changes, task chaining) and webhook streaming/event delivery to external systems, plus decorators like @after_kickoff for triggering follow-up actions, and a community example shows event-triggered automation (Slack alerts, auto-delete) built with CrewAI. However, this is more a code-defined workflow/agent framework than a declarative 'if-event-then-rule' automation engine, and a competitor explicitly notes CrewAI lacks persistent state/deterministic hooks for pausing/controlling agents that some rule-based automation needs. missing for 10: a first-party declarative rule-trigger system (e.g., condition-action rule builder), independent verification of robustness of event triggers beyond one community project, and confirmation of deterministic hook/state support contested by competitor.

   • [claimed-docs] “Flows allow you to create structured, event-driven workflows. They provide a seamless way to connect multiple tasks, manage state, and contr…”
   • [claimed-docs] “Flows allow you to create structured, event-driven workflows. They provide a seamless way to connect multiple tasks, manage state, and contr…”
   • [claimed-docs] “Webhook Streaming Stream real-time events and updates to your systems”
   • [community] “Show HN: built AI agents with CrewAI to automate an old Gmail account—categorizing emails, drafting replies, Slack alerts for high-priority …”
   • [community] “Competitor (Portia) founder's take: 'We think Crew AI is great. It's just that a lot of businesses we spoke to want the autonomy/decision ma…”

   proof ↗⚑ flag

   Google ADK✓fullclaimed7/10

   ADK explicitly supports event-driven automation via Callbacks ("Hook into specific events during an agent's execution lifecycle... without altering core agent logic") and a Workflow Runtime graph engine with routing, retry, fan-out/fan-in and dynamic nodes for triggering actions on execution events, matching the story of defining rules that fire on events. missing for 10: independent/hands-on evidence of callback-triggered rules in production use, and more detail on condition-based rule syntax beyond docs summaries.

   • [claimed-docs] “Callbacks: Hook into specific events during an agent's execution lifecycle to add logging, monitoring, or custom side-effects without alteri…”
   • [github] “Workflow Runtime: A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan…”
   • [github] “A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan-out/fan-in, loops…”
   • [claimed-docs] “In ADK, any agent application that has more than one agent or executable Node is considered a workflow.”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round drawn
   CrewAI—none0/10

   CrewAI's docs describe crews, flows, tasks, memory, CLI, and an enterprise kickoff/resume API, but nothing in the evidence pack mentions cron-style scheduling, recurring triggers, or any built-in job scheduler for automatically re-running crews/flows on a timer.

   ⚑ flag

   Google ADK—none0/10

   The evidence covers agent/workflow orchestration, deployment targets (Cloud Run, GKE, Agent Runtime), and evaluation, but nothing describes scheduling, cron-like triggers, or recurring execution of jobs/workflows. Absence of evidence for this applicable automation-depth capability yields 'none'.

   ⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round to CrewAI
   CrewAI~partialcommunity2/10

   CrewAI's docs note that agent/task configs are defined in YAML which 'allows you to version control your agent settings,' implying git-based versioning of automation definitions, but there is no first-party feature for reviewing changes or rolling back a running crew/flow to a prior state. A competitor explicitly contrasts CrewAI's lack of persistent state/deterministic pause-and-resume hooks, further underscoring the absence of built-in rollback mechanics. missing for 10: dedicated review workflow (diff/approval) for automation changes, built-in rollback/undo of crew or flow state, versioned deployment history beyond generic YAML+git.

   • [claimed-docs] “The YAML configuration allows you to: * Version control your agent settings * Easily switch between different models”
   • [community] “Competitor (Portia) founder's take: 'We think Crew AI is great. It's just that a lot of businesses we spoke to want the autonomy/decision ma…”

   proof ↗⚑ flag

   Google ADK—none0/10

   ADK is a framework for building agents (code, workflows, tools, deployment) but the evidence pack shows no version control, review, or rollback mechanism for automations themselves — no changelog/versioning UI, no approval/review workflow for agent definitions, no rollback feature. Agent code could theoretically be tracked via external git, but ADK itself provides no such capability in the evidence. Missing for 10: any versioning system, review/approval workflow, or rollback capability for automations.

   ⚑ flag

Deployment

1. engineering-leadDeploy an agent to a managed runtime and call it as an API endpoint

   weight 2 · round to Google ADK
   CrewAI~partialclaimed6/10

   CrewAI AMP/Enterprise explicitly offers one-click deployment to managed infrastructure and exposes deployed crews via REST API (kickoff/status/resume endpoints), satisfying the core ask. However, this is a separate paid product (AMP) from the open-source framework, requires additional setup, and evidence lacks independent/hands-on confirmation or details on scaling, auth, or SLAs. Missing for 10: independent/hands-on verification of the managed deployment and API in production, pricing/limits transparency, and confirmation this works outside the enterprise tier.

   • [claimed-docs] “Deploy your crews to a managed infrastructure with a few clicks”
   • [claimed-docs] “Deploy your crews to a managed infrastructure and monitor their execution in real-time.”
   • [claimed-docs] “Deploy your crews to a managed infrastructure with a few clicks... Access your deployed crews via REST API for integration with existing sys…”
   • [claimed-docs] “POST /resume](https://docs.crewai.com/edge/en/api-reference/resume.md): Resume crew execution with human feedback”
   • [claimed-docs] “POST /kickoff](https://docs.crewai.com/edge/en/api-reference/kickoff.md): Start a crew execution - [GET /status/{kickoff_id}](https://docs.c…”
   • [claimed-docs] “CrewAI AMP extends the power of the open-source framework with features designed for production deployments, collaboration, and scalability.…”

   proof ↗⚑ flag

   Google ADK✓fullclaimed8/10

   ADK docs explicitly describe deploying agents to a fully managed, auto-scaling Agent Engine/Agent Runtime on Google Cloud, plus alternative managed options like Cloud Run and GKE, with the stated purpose being to make the agent 'accessed, queried, and used in production' as an API endpoint. Missing for 10: no explicit hands-on/independent confirmation of the API contract (e.g., request/response schema) or third-party verification of endpoint behavior beyond first-party docs.

   • [claimed-docs] “Agent Runtime is a fully managed auto-scaling service on Google Cloud specifically designed for deploying, managing, and scaling AI agents b…”
   • [claimed-docs] “Cloud Run is a managed auto-scaling compute platform on Google Cloud that enables you to run your agent as a container-based application.”
   • [claimed-docs] “GKE is a good option if you need more control over the deployment as well as for running Open Models.”
   • [claimed-docs] “Once you've built and tested your agent using ADK, the next step is to deploy it so it can be accessed, queried, and used in production”
   • [claimed-docs] “You can manually package your Agent into a container image and then run it in any environment that supports container images.”

   proof ↗⚑ flag

2. engineering-leadRun my agents entirely on my own infrastructure with no dependence on the vendor's platform

   weight 2 · round to Google ADK
   CrewAI✓fullcommunity7/10

   CrewAI is an open-source Python framework installable via pip/uv with a local CLI to create, run, and test crews entirely in the developer's own environment, with no forced dependency on CrewAI's hosted platform — the 'AMP'/enterprise offerings (managed infra, Crew Studio, tool repository) are explicitly optional add-ons layered on top of the OSS core. missing for 10: no independent hands-on confirmation of fully air-gapped/self-hosted production runs at scale, and no explicit documentation contrasting self-hosted vs vendor-dependent feature parity.

   • [claimed-docs] “Run the following command to install crewai CLI: uv tool install crewai”
   • [claimed-docs] “Run the following command to install `crewai` CLI: uv tool install crewai ”
   • [claimed-docs] “The CrewAI CLI provides a set of commands to interact with CrewAI, allowing you to create, train, run, and manage crews & flows”
   • [claimed-docs] “The CrewAI CLI provides a set of commands to interact with CrewAI, allowing you to create, train, run, and manage crews & flows.”
   • [claimed-docs] “CrewAI AMP extends the power of the open-source framework with features designed for production deployments, collaboration, and scalability.…”
   • [claimed-docs] “Deploy your crews to a managed infrastructure with a few clicks”
   • [community] “On a CrewAI-based personal email agent: 'Local personal agents are going to be a major app type... They run locally on my computer with my f…”

   proof ↗⚑ flag

   Google ADK✓fullclaimed8/10

   ADK is an open-source framework (google/adk-python) that supports running agents locally via `adk run`/`adk web`, packaging into containers with `adk deploy docker`, deploying to any container-supporting environment, and explicitly documents an offline/disconnected mode with no Google Cloud connection required, alongside optional managed services like Agent Runtime/Cloud Run/GKE. missing for 10: independent/hands-on confirmation of fully vendor-free operation (e.g., third-party report of running ADK completely offline with non-Google models) and clarity on whether any telemetry/model calls still phone home by default.

   • [claimed-docs] “This is a good option if you prefer to run offline or disconnected, or otherwise in a system that has no connection to Google Cloud.”
   • [claimed-docs] “You can manually package your Agent into a container image and then run it in any environment that supports container images.”
   • [github] “adk deploy docker --with_ui <agent-folder>”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”
   • [claimed-docs] “ADK can work with almost any generative AI model. The framework provides easy access to Gemini as well as other leading models, and we provi…”
   • [claimed-docs] “GKE is a good option if you need more control over the deployment as well as for running Open Models.”

   proof ↗⚑ flag

Portability

1. developerSwap the underlying LLM provider or model without rewriting my agent

   weight 3 · round to Google ADK
   CrewAI✓fullclaimed7/10

   CrewAI docs explicitly state it integrates with multiple LLM providers via their native SDKs and that YAML-based agent configuration lets developers 'easily switch between different models' without rewriting agent logic, directly matching the deployment-portability story. missing for 10: independent/hands-on confirmation that switching providers requires zero code changes, and documentation of any provider-specific feature gaps that could break portability.

   • [claimed-docs] “CrewAI integrates with multiple LLM providers through providers native sdks, giving you the flexibility to choose the right model for your s…”
   • [claimed-docs] “The YAML configuration allows you to: * Version control your agent settings * Easily switch between different models”
   • [claimed-docs] “CrewAI integrates with multiple LLM providers through providers native sdks, giving you the flexibility to choose the right model for your s…”

   proof ↗⚑ flag

   Google ADK✓fullclaimed8/10

   Docs explicitly state ADK works with 'almost any generative AI model,' providing adapters for Gemini and many other model providers including locally running models, and the agent definition just takes a `model` string parameter (e.g., 'gemini-flash-latest'), implying swapping providers is a config change rather than a rewrite. Missing for 10: independent/hands-on confirmation that switching providers requires no code changes to agent logic, and no explicit list of supported non-Gemini providers with concrete migration examples.

   • [claimed-docs] “ADK can work with almost any generative AI model. The framework provides easy access to Gemini as well as other leading models, and we provi…”
   • [claimed-docs] “agent = Agent( name="researcher", model="gemini-flash-latest", instruction="You help users research topics thoroughly.", too…”
   • [claimed-docs] “The basic components of an Agent are an artificial intelligence (AI) model, task instructions, and optionally, a set of tools to be used by …”

   proof ↗⚑ flag

Evals

1. engineering-leadScore agent quality with built-in evals and run them as part of CI

   weight 2 · round to Google ADK
   CrewAI~partialclaimed4/10

   CrewAI ships a `crewai test` CLI command that runs the crew for N iterations and reports performance metrics, plus observability/tracing for execution times and token usage, which could be scripted into CI. However, there's no documented eval framework (accuracy/quality scoring, rubrics, pass/fail thresholds) or first-party CI integration guidance. Missing for 10: a dedicated eval scoring API/rubric beyond generic performance metrics, and explicit CI pipeline documentation/examples.

   • [claimed-docs] “We added the CLI command crewai test to make it easy to test your crew. This command will run your crew for a specified number of iterations…”
   • [claimed-docs] “We added the CLI command `crewai test` to make it easy to test your crew.”
   • [claimed-docs] “We added the CLI command `crewai test` to make it easy to test your crew. This command will run your crew for a specified number of iteratio…”
   • [claimed-docs] “Performance Monitoring: Track agent execution times, token usage, and resource consumption”
   • [claimed-docs] “Track agent execution times, token usage, and resource consumption”
   • [github] “Tracing & Observability: Monitor and track your AI agents and workflows in real-time, including metrics, logs, and traces.”

   proof ↗⚑ flag

   Google ADK✓fullclaimed8/10

   ADK ships a first-party evaluation framework with groundtruth and rubric-based metrics, expected tool-use trajectories, evalset.json test files, and a documented CLI command (`adk eval <agent> <evalset>`) that can be scripted/invoked headlessly, which is exactly the shape needed for CI integration. Missing for 10: explicit CI/CD pipeline documentation (e.g., a GitHub Actions example) and independent/third-party corroboration of running adk eval in CI.

   • [claimed-docs] “This approach involves creating individual test files, each representing a single, simple agent-model interaction (a session).”
   • [claimed-docs] “ADK provides both groundtruth based and rubric based tool use evaluation metrics.”
   • [claimed-docs] “This approach involves creating individual test files, each representing a single, simple agent-model interaction (a session). It's most eff…”
   • [claimed-docs] “Expected Intermediate Tool Use Trajectory: The tool calls we expect the agent to make in order to respond correctly to the user query.”
   • [github] “adk eval \ samples_for_testing/hello_world \ samples_for_testing/hello_world/hello_world_eval_set_001.evalset.json”

   proof ↗⚑ flag

Testing

1. developerUnit-test agents with mocked models and tools

   weight 2 · round to Google ADK
   CrewAI—none0/10

   The evidence shows a `crewai test` CLI command that runs a crew for several iterations and reports performance metrics, but this is an integration-style evaluation run, not unit testing with mocked models/tools. There is no documentation of dependency injection, mock LLM/tool interfaces, or pytest-style fixtures for isolating agent logic from real model/tool calls.

   • [claimed-docs] “We added the CLI command crewai test to make it easy to test your crew. This command will run your crew for a specified number of iterations…”
   • [claimed-docs] “We added the CLI command `crewai test` to make it easy to test your crew.”
   • [claimed-docs] “We added the CLI command `crewai test` to make it easy to test your crew. This command will run your crew for a specified number of iteratio…”

   proof ↗⚑ flag

   Google ADK~partialclaimed5/10

   ADK docs describe a test-file based evaluation approach explicitly described as 'a form of unit testing' for single agent-model interactions, with expected tool-use trajectories and groundtruth/rubric metrics plus an `adk eval` CLI — but none of this evidence explicitly describes mocking models or tools (e.g., swapping in fake LLM responses or stub tool implementations) for isolated unit tests. Missing for 10: explicit mocked-model/mocked-tool test fixtures or APIs, independent/hands-on confirmation of mocking support, and unit-test framework integration examples (e.g., pytest with mock objects).

   • [claimed-docs] “This approach involves creating individual test files, each representing a single, simple agent-model interaction (a session).”
   • [claimed-docs] “This approach involves creating individual test files, each representing a single, simple agent-model interaction (a session). It's most eff…”
   • [claimed-docs] “Expected Intermediate Tool Use Trajectory: The tool calls we expect the agent to make in order to respond correctly to the user query.”
   • [claimed-docs] “ADK provides both groundtruth based and rubric based tool use evaluation metrics.”
   • [github] “adk eval \ samples_for_testing/hello_world \ samples_for_testing/hello_world/hello_world_eval_set_001.evalset.json”

   proof ↗⚑ flag

Tracing

1. developerTrace every LLM call and tool invocation of an agent run in an observability UI

   weight 3 · round to CrewAI
   CrewAI~partialclaimed7/10

   CrewAI documents built-in observability with tracing of metrics, logs, and traces for agent/tool calls (crewai-gh-2, crewai-docs-11/29), and CrewAI AMP/enterprise offers a real-time monitoring dashboard (crewai-docs-30/55). However, the evidence is entirely first-party doc claims with no independent/hands-on confirmation of the observability UI's granularity (e.g., per-LLM-call or per-tool-invocation trace views) or screenshots demonstrating it works as described. missing for 10: independent/hands-on validation of the trace UI, detail on per-call/tool-invocation granularity, and evidence of third-party observability integrations (e.g., Langfuse/Arize) being verified in practice.

   • [github] “Tracing & Observability: Monitor and track your AI agents and workflows in real-time, including metrics, logs, and traces.”
   • [claimed-docs] “Performance Monitoring: Track agent execution times, token usage, and resource consumption”
   • [claimed-docs] “Track agent execution times, token usage, and resource consumption”
   • [claimed-docs] “Deploy your crews to a managed infrastructure and monitor their execution in real-time.”
   • [claimed-docs] “CrewAI AMP extends the power of the open-source framework with features designed for production deployments, collaboration, and scalability.…”

   proof ↗⚑ flag

   Google ADK~partialclaimed5/10

   ADK ships a built-in development Web UI explicitly for testing, evaluating, and debugging agents, and provides callbacks to hook into execution lifecycle events for logging/monitoring, which together imply some run-level visibility into tool and model calls. However, the evidence never explicitly describes a trace view showing each LLM call and tool invocation of a run, nor mentions integration with tracing standards (e.g., OpenTelemetry) or a dedicated observability dashboard beyond the dev/eval UI. Missing for 10: explicit documentation of per-call tracing UI, tool-invocation-level trace inspection, and any third-party/hands-on confirmation of this granularity.

   • [github] “A built-in development UI to help you test, evaluate, debug, and showcase your agent(s).”
   • [github] “Web UI (supports multi-agent directories or pointing directly to a single agent folder)”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”
   • [claimed-docs] “Callbacks: Hook into specific events during an agent's execution lifecycle to add logging, monitoring, or custom side-effects without alteri…”

   proof ↗⚑ flag

Guardrails

1. developerAttach input/output guardrails that validate, transform, or block unsafe content

   weight 3 · round to Google ADK
   CrewAI—none0/10

   The evidence pack contains no documentation of a guardrail mechanism for validating, transforming, or blocking agent input/output content — no task-level or agent-level guardrail parameter, content filter, or safety-check API is mentioned anywhere in the docs. In fact, a competitor explicitly claims 'where Crew AI lacks that' regarding dial-up/down guardrails, and a community critique flags CrewAI's lack of safeguards against destructive actions (e.g., no guardrail preventing permanent email deletion on misclassification).

   • [community] “Competitor (Portia) founder's take: 'We think Crew AI is great. It's just that a lot of businesses we spoke to want the autonomy/decision ma…”
   • [community] “Commenter critique of a CrewAI-based email agent: 'Why not archive vs. the destructive delete... Seems like a default aggressive stance,' ra…”

   proof ↗⚑ flag

   Google ADK~partialclaimed5/10

   ADK exposes general extensibility hooks—Callbacks to intercept execution events for custom logic/side-effects, Plugins for pre-packaged behaviors, and a Tool Confirmation (HITL) flow that can guard tool execution—which developers could use to build input/output guardrails, but there is no dedicated 'guardrails' feature, built-in content-safety/validation API, or example showing blocking/transforming unsafe content end-to-end. Missing for 10: explicit guardrail/validation API or moderation integration, documented examples of blocking/transforming unsafe input or output, and any third-party/community confirmation of this pattern in practice.

   • [claimed-docs] “Callbacks: Hook into specific events during an agent's execution lifecycle to add logging, monitoring, or custom side-effects without alteri…”
   • [claimed-docs] “Plugins: Integrate complex, pre-packaged behaviors and third-party services directly into your agent's workflow.”
   • [github] “Tool Confirmation: A tool confirmation flow (HITL) that can guard tool execution with explicit confirmation and custom input.”
   • [github] “A tool confirmation flow (HITL) that can guard tool execution with explicit confirmation and custom input.”

   proof ↗⚑ flag

2. engineering-leadRestrict what an agent may do with fine-grained tool permissions and sandboxed execution

   weight 2 · round to Google ADK
   CrewAI~partialcommunity3/10

   CrewAI lets you assign specific tools to a given Agent/Task (crewai-docs-2, crewai-docs-6), which is a rudimentary form of restricting what an agent can do, but there is no documentation of fine-grained permission scopes, execution sandboxing, or resource isolation. A competitor's founder explicitly states CrewAI 'lacks' dial-up/down guardrails and deterministic pause hooks (crewai-comm-4), and a community example shows a CrewAI agent auto-deleting emails with no safety confirmation, drawing criticism for the lack of guardrails (crewai-comm-2). missing for 10: sandboxed/isolated execution environment, granular permission policies per tool/action, first-party guardrail/approval mechanisms, and evidence rebutting the community claim that such guardrails are absent.

   • [claimed-docs] “a "Task" is a specific assignment completed by an "Agent". Tasks provide all necessary details for execution, such as a description, the age…”
   • [claimed-docs] “CrewAI tools empower agents with capabilities ranging from web searching and data analysis to collaboration and delegating tasks among cowor…”
   • [community] “Commenter critique of a CrewAI-based email agent: 'Why not archive vs. the destructive delete... Seems like a default aggressive stance,' ra…”
   • [community] “Competitor (Portia) founder's take: 'We think Crew AI is great. It's just that a lot of businesses we spoke to want the autonomy/decision ma…”

   proof ↗⚑ flag

   Google ADK~partialclaimed4/10

   ADK provides a Tool Confirmation (HITL) flow that can gate tool execution with explicit confirmation/custom input, plus callbacks/plugins hooks to intercept agent actions, giving some control over agent behavior. However there is no evidence of fine-grained per-tool permission policies or an actual sandboxed execution environment for code/tool runs. Missing for 10: explicit sandboxing of tool/code execution, a permissions/ACL system scoping tool access, and independent verification of these guardrails in practice.

   • [github] “Tool Confirmation: A tool confirmation flow (HITL) that can guard tool execution with explicit confirmation and custom input.”
   • [github] “A tool confirmation flow (HITL) that can guard tool execution with explicit confirmation and custom input.”
   • [claimed-docs] “Callbacks: Hook into specific events during an agent's execution lifecycle to add logging, monitoring, or custom side-effects without alteri…”
   • [claimed-docs] “Plugins: Integrate complex, pre-packaged behaviors and third-party services directly into your agent's workflow.”

   proof ↗⚑ flag

Approval flows

1. developerPause an agent mid-run for human input or approval and resume with the human's decision

   weight 3 · round to Google ADK
   CrewAI~partialcommunity5/10

   CrewAI's Edge API explicitly documents kickoff/status/resume endpoints, including 'Resume crew execution with human feedback' (crewai-docs-56/57), which matches the pause-for-approval-and-resume story. However, this exists mainly as a REST-level control-plane feature (not deeply documented with in-process human_input examples), and a competitor's public claim explicitly disputes CrewAI's pause/resume robustness ('deterministic hooks so you can pause an agent... where Crew AI lacks that' — crewai-comm-4), creating real doubt about how mature this capability is in practice. missing for 10: first-party walkthrough/code sample of pausing mid-task for human approval, independent hands-on confirmation that resume-with-feedback works reliably, and rebuttal to the competitor's lacks-that claim.

   • [claimed-docs] “POST /resume](https://docs.crewai.com/edge/en/api-reference/resume.md): Resume crew execution with human feedback”
   • [claimed-docs] “POST /kickoff](https://docs.crewai.com/edge/en/api-reference/kickoff.md): Start a crew execution - [GET /status/{kickoff_id}](https://docs.c…”
   • [community] “Competitor (Portia) founder's take: 'We think Crew AI is great. It's just that a lot of businesses we spoke to want the autonomy/decision ma…”

   proof ↗⚑ flag

   Google ADK✓fullclaimed8/10

   ADK explicitly documents a Tool Confirmation flow described as HITL that can 'guard tool execution with explicit confirmation and custom input,' plus a Workflow Runtime and Task API both explicitly listing human-in-the-loop support with state management for pausing and resuming execution. This directly matches pausing mid-run for human approval and resuming with the decision, though missing for 10: a concrete end-to-end code example showing pause/resume state persistence and independent third-party corroboration beyond vendor GitHub README claims.

   • [github] “Tool Confirmation: A tool confirmation flow (HITL) that can guard tool execution with explicit confirmation and custom input.”
   • [github] “A tool confirmation flow (HITL) that can guard tool execution with explicit confirmation and custom input.”
   • [github] “Workflow Runtime: A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan…”
   • [github] “A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan-out/fan-in, loops…”
   • [github] “Task API: Structured agent-to-agent delegation with multi-turn task mode, single-turn controlled output, mixed delegation patterns, human-in…”
   • [github] “Structured agent-to-agent delegation with multi-turn task mode, single-turn controlled output, mixed delegation patterns, human-in-the-loop,…”

   proof ↗⚑ flag

2. engineering-leadRequire human approval before specific sensitive tool calls execute

   weight 2 · round to Google ADK
   CrewAI!disputedcontradicted4/10

   CrewAI's Enterprise/Edge API exposes a POST /resume endpoint described as 'Resume crew execution with human feedback,' implying some human-in-the-loop pause/resume mechanism, but this is a crew-level checkpoint, not a documented gate that specifically intercepts sensitive tool calls before execution. A competitor founder (Portia) explicitly states in an independent HN discussion that 'Crew AI lacks' persistent state and deterministic hooks to pause an agent mid-run, directly contradicting the vendor's implied human-in-the-loop guardrail capability. Missing for 10: first-party docs describing per-tool or per-action approval gating, a documented tool-level 'require_confirmation' or callback mechanism, and independent corroboration that the resume API actually functions as a sensitive-action gate rather than a generic crew pause.

   • [claimed-docs] “POST /resume](https://docs.crewai.com/edge/en/api-reference/resume.md): Resume crew execution with human feedback”
   • [community] “Competitor (Portia) founder's take: 'We think Crew AI is great. It's just that a lot of businesses we spoke to want the autonomy/decision ma…”

   proof ↗⚑ flag

   Google ADK✓fullclaimed8/10

   ADK explicitly documents a 'Tool Confirmation' HITL flow that guards tool execution with explicit confirmation and custom input, plus broader human-in-the-loop support in its workflow/task orchestration engines, directly matching the story of requiring approval before sensitive tool calls execute. Missing for 10: no independent/hands-on validation or detailed walkthrough of configuring per-tool approval policies beyond the feature summary.

   • [github] “Tool Confirmation: A tool confirmation flow (HITL) that can guard tool execution with explicit confirmation and custom input.”
   • [github] “A tool confirmation flow (HITL) that can guard tool execution with explicit confirmation and custom input.”
   • [github] “Workflow Runtime: A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan…”
   • [github] “A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan-out/fan-in, loops…”
   • [github] “Task API: Structured agent-to-agent delegation with multi-turn task mode, single-turn controlled output, mixed delegation patterns, human-in…”
   • [github] “Structured agent-to-agent delegation with multi-turn task mode, single-turn controlled output, mixed delegation patterns, human-in-the-loop,…”

   proof ↗⚑ flag

Memory

1. developerTrim, summarize, or filter conversation history to keep an agent inside its context window

   weight 2 · round to Google ADK
   CrewAI—none0/10

   CrewAI's memory docs describe a retrieval system with composite scoring (semantic + recency + importance) for long-term memory recall, but there is no evidence of features to trim, summarize, or filter conversation history specifically to keep an agent within its context window during execution.

   • [claimed-docs] “CrewAI provides a unified memory system -- a single Memory class... supports adaptive-depth recall with composite scoring that blends semant…”
   • [claimed-docs] “Memory uses an LLM to analyze content when saving (inferring scope, categories, and importance) and supports adaptive-depth recall with comp…”
   • [claimed-docs] “CrewAI provides a **unified memory system** -- a single `Memory` class that replaces separate short-term, long-term, entity, and external me…”
   • [claimed-docs] “Retrieve -- results ranked by composite score (semantic + recency + importance)”

   proof ↗⚑ flag

   Google ADK✓fullclaimed7/10

   ADK docs explicitly state it "automatically filters irrelevant events, summarizes older conversational turns, lazy-loads artifacts, and tracks token usage," directly addressing trimming/summarizing/filtering to manage context window, reinforced by mention of designing for AI context window limits. Missing for 10: no code example/API reference showing how a developer configures or customizes this summarization/filtering behavior, and no independent/hands-on corroboration beyond first-party docs.

   • [claimed-docs] “ADK automatically filters irrelevant events, summarizes older conversational turns, lazy-loads artifacts, and tracks token usage.”
   • [claimed-docs] “Use prebuilt or custom Agent Skills to extend agent capabilities in a way that works efficiently inside AI context window limits.”

   proof ↗⚑ flag

2. developerGive agents long-term memory that persists across sessions and threads

   weight 2 · round to CrewAI
   CrewAI✓fullclaimed8/10

   CrewAI's docs describe a unified Memory system that persists agent interactions with semantic, recency, and importance-based recall, explicitly replacing separate short-term/long-term/entity memory types with a single persistent API. This directly supports giving agents durable memory across executions. Missing for 10: explicit documentation confirming memory persists across separate sessions/threads (vs. within a single crew run) and independent/hands-on verification of long-term persistence.

   • [claimed-docs] “CrewAI provides a unified memory system -- a single Memory class... supports adaptive-depth recall with composite scoring that blends semant…”
   • [claimed-docs] “Memory uses an LLM to analyze content when saving (inferring scope, categories, and importance) and supports adaptive-depth recall with comp…”
   • [claimed-docs] “CrewAI provides a **unified memory system** -- a single `Memory` class... Memory uses an LLM to analyze content when saving (inferring scope…”
   • [claimed-docs] “CrewAI provides a **unified memory system** -- a single `Memory` class that replaces separate short-term, long-term, entity, and external me…”
   • [claimed-docs] “Retrieve -- results ranked by composite score (semantic + recency + importance)”

   proof ↗⚑ flag

   Google ADK~partialclaimed3/10

   The docs mention session-based interactions and automatic context management (filtering irrelevant events, summarizing older turns, tracking token usage) but there is no explicit evidence of a dedicated long-term memory service or store that persists agent knowledge across separate sessions/threads. missing for 10: explicit memory/session-store API docs, cross-session persistence guarantees, first-party examples of retrieving memory in a new thread.

   • [claimed-docs] “ADK automatically filters irrelevant events, summarizes older conversational turns, lazy-loads artifacts, and tracks token usage.”
   • [claimed-docs] “This approach involves creating individual test files, each representing a single, simple agent-model interaction (a session).”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Google ADK
   CrewAI~partialprobed4/10

   CrewAI exposes execution-oriented API endpoints (POST /kickoff, /resume, GET /status) and enterprise REST access to deployed crews, so core run/monitor actions are API-reachable, but the no-code Visual Agent Builder / Crew Studio and Tool Repository features are marketed as UI-only conveniences with no documented API equivalent for building/configuring agents that mirrors the visual tools, and a direct probe for a public OpenAPI/swagger spec returned 404s, indicating no confirmed comprehensive API surface matching UI capabilities. missing for 10: documented API parity for visual agent/task building and tool repository management, a public OpenAPI spec confirming full UI-equivalent API coverage, independent confirmation of API completeness.

   • [claimed-docs] “POST /resume](https://docs.crewai.com/edge/en/api-reference/resume.md): Resume crew execution with human feedback”
   • [claimed-docs] “POST /kickoff](https://docs.crewai.com/edge/en/api-reference/kickoff.md): Start a crew execution - [GET /status/{kickoff_id}](https://docs.c…”
   • [claimed-docs] “Deploy your crews to a managed infrastructure with a few clicks... Access your deployed crews via REST API for integration with existing sys…”
   • [claimed-docs] “Crew Studio Create and customize crews using a no-code/low-code interface”
   • [claimed-docs] “Tool Repository Publish and install tools to enhance your crews’ capabilities”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.crewai.com/openapi.json, https://docs.crewai.com/swagger.json, https://docs.crewai.com/…”

   proof ↗⚑ flag

   Google ADK~partialprobed5/10

   ADK is primarily a code-first Python framework where agents are built and orchestrated programmatically (Agent(), Workflow Runtime, Task API), and the CLI (adk run/web/eval/deploy) exposes most dev-loop actions including the same UI functions, suggesting reasonable parity between programmatic/CLI and the built-in dev UI. However, there's no evidence of a documented REST/OpenAPI API for driving the dev UI's specific features programmatically, and probes show no OpenAPI spec or llms.txt discoverability. missing for 10: explicit API/CLI parity documentation for every dev-UI feature (debug, evaluate, showcase), a published OpenAPI/REST spec, and confirmation that UI-only actions (e.g. visual debugging, showcase mode) are fully scriptable.

   • [github] “A built-in development UI to help you test, evaluate, debug, and showcase your agent(s).”
   • [github] “Web UI (supports multi-agent directories or pointing directly to a single agent folder)”
   • [github] “adk run path/to/my_agent”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”
   • [github] “adk eval \ samples_for_testing/hello_world \ samples_for_testing/hello_world/hello_world_eval_set_001.evalset.json”
   • [github] “adk deploy docker --with_ui <agent-folder>”
   • [probe] “PROBE openapi: all candidate paths 404 (https://google.github.io/openapi.json, https://google.github.io/swagger.json, https://google.github.…”
   • [probe] “PROBE llms.txt: HTTP 404 at https://google.github.io/llms.txt”

   proof ↗⚑ flag

2. ai-native userRead the product's source under an open license

   weight 2 · round to Google ADK
   CrewAI~partialclaimed6/10

   The evidence confirms CrewAI's core is described as an 'open-source framework' (crewai-docs-55) and the source is hosted publicly on GitHub (crewai-gh-1, crewai-gh-2), supporting that AI-native users can read the source. However, no explicit license file or license type (e.g., MIT) is cited in the evidence pack. Missing for 10: explicit license identification (e.g., LICENSE file content or SPDX tag), independent confirmation of license terms, and any documentation page explicitly stating open-source licensing terms.

   • [claimed-docs] “CrewAI AMP extends the power of the open-source framework with features designed for production deployments, collaboration, and scalability.…”
   • [github] “It gives developers autonomous agent collaboration through Crews and precise, event-driven control through Flows.”
   • [github] “Tracing & Observability: Monitor and track your AI agents and workflows in real-time, including metrics, logs, and traces.”

   proof ↗⚑ flag

   Google ADK✓fullclaimed7/10

   The evidence repeatedly links to the public GitHub repository https://github.com/google/adk-python, which hosts the full source code and CLI (adk run, adk web, adk eval, adk deploy) that AI-native users can read and inspect directly. Missing for 10: an explicit citation of the license file/type (e.g., Apache-2.0) confirming the open-license terms, and independent third-party confirmation of licensing.

   • [github] “Agent Config: Build agents without code.”
   • [github] “A built-in development UI to help you test, evaluate, debug, and showcase your agent(s).”
   • [github] “adk run path/to/my_agent”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”

   proof ↗⚑ flag

3. ai-native userSelf-host the core product

   weight 3 · round drawn
   CrewAI✓fullclaimed8/10

   CrewAI's core is an open-source Python framework (confirmed by GitHub repo and docs noting 'CrewAI AMP extends the power of the open-source framework'), installable and runnable entirely locally via `uv tool install crewai`/pip, with no dependency on a hosted service for core Crew/Flow/Agent execution. This directly satisfies self-hosting for an AI-native/developer persona. Missing for 10: no dedicated self-hosted production deployment guide (e.g., Docker/Kubernetes) or independent report of large-scale self-hosted operation.

   • [claimed-docs] “CrewAI AMP extends the power of the open-source framework with features designed for production deployments, collaboration, and scalability.…”
   • [claimed-docs] “Run the following command to install crewai CLI: uv tool install crewai”
   • [claimed-docs] “Run the following command to install `crewai` CLI: uv tool install crewai ”
   • [github] “It gives developers autonomous agent collaboration through Crews and precise, event-driven control through Flows.”
   • [claimed-docs] “Build collaborative AI agents, crews, and flows — production ready from day one.”

   proof ↗⚑ flag

   Google ADK✓fullclaimed8/10

   ADK is an open-source framework (github.com/google/adk-python) that can be run entirely locally via `adk run`/`adk web`, packaged into containers, and deployed offline/disconnected from Google Cloud, evidencing full self-hosting capability without requiring the vendor's managed service. Missing for 10: no independent third-party report confirming a full self-hosted production deployment, and no explicit self-hosted infra requirements/scaling guidance beyond container packaging.

   • [claimed-docs] “This is a good option if you prefer to run offline or disconnected, or otherwise in a system that has no connection to Google Cloud.”
   • [claimed-docs] “You can manually package your Agent into a container image and then run it in any environment that supports container images.”
   • [github] “adk run path/to/my_agent”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”
   • [github] “adk deploy docker --with_ui <agent-folder>”
   • [claimed-docs] “GKE is a good option if you need more control over the deployment as well as for running Open Models.”

   proof ↗⚑ flag

Multi agent

1. developerOrchestrate multiple agents — handoffs, subagents, or crews — inside one workflow

   weight 3 · round drawn
   CrewAI✓fullcommunity9/10

   CrewAI's core architecture is explicitly built for multi-agent orchestration: Agents, Tasks, and Crews collaborate with sequential/hierarchical processes and delegation, plus Flows for event-driven coordination across multiple crews/tasks (crewai-docs-3,4,20,21,35,36,46,48). A hands-on community build (Gmail triage agent with multiple collaborating CrewAI agents) corroborates real-world multi-agent use (crewai-comm-1), and a competitor's critique concerns guardrails/state persistence, not whether orchestration works at all (crewai-comm-4). missing for 10: independent large-scale benchmarking of complex multi-crew handoffs beyond docs and one hands-on example.

   • [claimed-docs] “process = Process.sequential # or Process.hierarchical”
   • [claimed-docs] “Flows allow you to create structured, event-driven workflows. They provide a seamless way to connect multiple tasks, manage state, and contr…”
   • [claimed-docs] “A crew in crewAI represents a collaborative group of agents working together to achieve a set of tasks.”
   • [claimed-docs] “Flows allow developers to combine and coordinate coding tasks and Crews efficiently, providing a robust framework for building sophisticated…”
   • [claimed-docs] “A crew in crewAI represents a collaborative group of agents working together to achieve a set of tasks. Each crew defines the strategy for t…”
   • [claimed-docs] “Flows allow you to create structured, event-driven workflows. They provide a seamless way to connect multiple tasks, manage state, and contr…”
   • [claimed-docs] “Perform specific tasks * Make decisions based on its role and goal * Use tools to accomplish objectives * Communicate and collaborate with o…”
   • [claimed-docs] “Sequential: Tasks are executed in the order they are defined * Hierarchical: Tasks are as”
   • [community] “Show HN: built AI agents with CrewAI to automate an old Gmail account—categorizing emails, drafting replies, Slack alerts for high-priority …”
   • [community] “Competitor (Portia) founder's take: 'We think Crew AI is great. It's just that a lot of businesses we spoke to want the autonomy/decision ma…”

   proof ↗⚑ flag

   Google ADK✓fullclaimed9/10

   ADK explicitly supports multi-agent orchestration: workflows are defined as any application with more than one agent/node, with a graph-based Workflow Runtime supporting routing, fan-out/fan-in, loops, nested workflows, and a Task API for structured agent-to-agent delegation including multi-turn task mode and mixed delegation patterns; the CLI/Web UI explicitly supports multi-agent directories. missing for 10: independent third-party hands-on validation of complex multi-agent orchestration at scale.

   • [claimed-docs] “you can use the ADK development framework to expand them into workflows, which allow you to combine and orchestrate multiple agents and code…”
   • [claimed-docs] “In ADK, any agent application that has more than one agent or executable Node is considered a workflow.”
   • [github] “Workflow Runtime: A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan…”
   • [github] “Task API: Structured agent-to-agent delegation with multi-turn task mode, single-turn controlled output, mixed delegation patterns, human-in…”
   • [github] “Structured agent-to-agent delegation with multi-turn task mode, single-turn controlled output, mixed delegation patterns, human-in-the-loop,…”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”

   proof ↗⚑ flag

Workflow control

1. developerCompose agents into an explicit graph or workflow with branching, loops, and parallel steps

   weight 2 · round to Google ADK
   CrewAI~partialclaimed6/10

   CrewAI's Flows feature is explicitly documented as an event-driven workflow system that lets developers 'connect multiple tasks, manage state, and control the flow of execution,' and Crews support sequential/hierarchical process graphs — both point toward explicit workflow composition. However, the evidence never documents specific branching (routers/conditionals), loop, or parallel-step primitives, so the graph capability is implied rather than concretely shown. missing for 10: explicit documentation of branching/router constructs, loop constructs, and parallel-step execution within Flows, plus independent hands-on confirmation of these mechanics.

   • [claimed-docs] “Flows allow you to create structured, event-driven workflows. They provide a seamless way to connect multiple tasks, manage state, and contr…”
   • [claimed-docs] “Flows allow developers to combine and coordinate coding tasks and Crews efficiently, providing a robust framework for building sophisticated…”
   • [claimed-docs] “Flows allow you to create structured, event-driven workflows. They provide a seamless way to connect multiple tasks, manage state, and contr…”
   • [claimed-docs] “Sequential: Tasks are executed in the order they are defined * Hierarchical: Tasks are as”
   • [claimed-docs] “Flows make it super easy to manage and share state between different tasks in your workflow.”
   • [github] “It gives developers autonomous agent collaboration through Crews and precise, event-driven control through Flows.”

   proof ↗⚑ flag

   Google ADK✓fullclaimed9/10

   ADK provides a dedicated graph-based Workflow Runtime with explicit support for routing, fan-out/fan-in (parallel), loops, retry, nested workflows, and dynamic nodes, plus structured Task API for agent delegation and workflow nodes—directly matching branching/loops/parallel composition; docs also describe 'graph-based architectures with explicit execution paths.' Missing for 10: independent/hands-on third-party validation beyond vendor docs and GitHub README.

   • [github] “Workflow Runtime: A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan…”
   • [github] “A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan-out/fan-in, loops…”
   • [github] “Task API: Structured agent-to-agent delegation with multi-turn task mode, single-turn controlled output, mixed delegation patterns, human-in…”
   • [github] “Structured agent-to-agent delegation with multi-turn task mode, single-turn controlled output, mixed delegation patterns, human-in-the-loop,…”
   • [claimed-docs] “Weave deterministic code with adaptive AI reasoning. Orchestrate complex tasks through structured, graph-based architectures, with explicit …”
   • [claimed-docs] “In ADK, any agent application that has more than one agent or executable Node is considered a workflow.”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round drawn
   CrewAI—none0/10

   No evidence in the pack mentions data residency, region selection, or storage location controls for CrewAI (open-source framework or AMP/enterprise offering); deployment docs describe managed infrastructure but never region choice or compliance certifications.

   ⚑ flag

   Google ADK—none0/10

   ADK is a framework that can be deployed via Cloud Run, GKE, or self-hosted/offline (google-adk-docs-7, google-adk-docs-14, google-adk-docs-19), which implies developers control infrastructure location, but there is no explicit documentation about data residency, region selection, or storage location controls for agent data.

   • [claimed-docs] “This is a good option if you prefer to run offline or disconnected, or otherwise in a system that has no connection to Google Cloud.”
   • [claimed-docs] “Cloud Run is a managed auto-scaling compute platform on Google Cloud that enables you to run your agent as a container-based application.”
   • [claimed-docs] “GKE is a good option if you need more control over the deployment as well as for running Open Models.”

   proof ↗⚑ flag

2. ai-native userControl data retention and deletion

   weight 2 · round drawn
   CrewAI—none0/10

   The evidence pack contains no documentation of data retention policies, deletion controls, or privacy/compliance settings for CrewAI's memory or storage systems. Community items describe an agent programmed to delete emails as a task behavior, not CrewAI providing user-facing retention/deletion controls over its own stored data.

   ⚑ flag

   Google ADK—none0/10

   The evidence describes ADK as a self-hosted/deployable agent framework (Cloud Run, GKE, offline/disconnected deployment) but contains no documentation of explicit data retention policies, session/state deletion APIs, or user-facing controls for purging stored data. missing for 10: explicit retention/deletion controls, session data lifecycle docs, any privacy/compliance statements about stored artifacts or memory.

   • [claimed-docs] “This is a good option if you prefer to run offline or disconnected, or otherwise in a system that has no connection to Google Cloud.”
   • [claimed-docs] “ADK automatically filters irrelevant events, summarizes older conversational turns, lazy-loads artifacts, and tracks token usage.”

   proof ↗⚑ flag

3. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   CrewAI—none0/10

   No evidence in the pack mentions telemetry collection or an opt-out/disable-tracking mechanism for CrewAI; observability docs describe monitoring agent performance but say nothing about product usage telemetry or opt-out controls.

   ⚑ flag

   Google ADK—none0/10

   No evidence in the pack addresses telemetry collection or an opt-out mechanism for ADK; the docs cover agent building, deployment, evaluation, and workflows but never mention usage tracking or privacy controls. This is a fair axis for a developer framework/SDK, but absence of evidence means it counts as none. missing for 10: any mention of telemetry collection, an opt-out flag/env var, or a privacy policy describing data tracking.

   ⚑ flag

Durable state

1. developerCheckpoint agent state so a run can resume exactly where it left off after a crash or restart

   weight 3 · round to CrewAI
   CrewAI!disputedcontradicted4/10

   Docs describe Flows managing/sharing state across tasks and a unified Memory system for persistence (crewai-docs-4, crewai-docs-36, crewai-docs-49, crewai-docs-5, crewai-docs-50), plus an enterprise API with kickoff/status/resume endpoints (crewai-docs-56, crewai-docs-57), but the resume endpoint is described for human-feedback loops, not crash recovery. A competitor's founder explicitly claims 'Crew AI lacks' persistent state and deterministic pause/resume hooks that their own product has (crewai-comm-4), directly contradicting the notion of reliable crash-resume checkpointing. Missing for 10: first-party documentation of an explicit crash-recovery/checkpoint-resume mechanism, and independent hands-on confirmation that a run can resume exactly where it left off after a crash.

   • [claimed-docs] “Flows allow you to create structured, event-driven workflows. They provide a seamless way to connect multiple tasks, manage state, and contr…”
   • [claimed-docs] “Flows allow you to create structured, event-driven workflows. They provide a seamless way to connect multiple tasks, manage state, and contr…”
   • [claimed-docs] “Flows make it super easy to manage and share state between different tasks in your workflow.”
   • [claimed-docs] “CrewAI provides a unified memory system -- a single Memory class... supports adaptive-depth recall with composite scoring that blends semant…”
   • [claimed-docs] “CrewAI provides a **unified memory system** -- a single `Memory` class that replaces separate short-term, long-term, entity, and external me…”
   • [claimed-docs] “POST /resume](https://docs.crewai.com/edge/en/api-reference/resume.md): Resume crew execution with human feedback”
   • [claimed-docs] “POST /kickoff](https://docs.crewai.com/edge/en/api-reference/kickoff.md): Start a crew execution - [GET /status/{kickoff_id}](https://docs.c…”
   • [community] “Competitor (Portia) founder's take: 'We think Crew AI is great. It's just that a lot of businesses we spoke to want the autonomy/decision ma…”

   proof ↗⚑ flag

   Google ADK—none0/10

   Evidence only mentions generic 'state management' as one feature in the workflow runtime engine, with no documentation of session/state persistence, checkpointing, or resuming an agent run after a crash or restart. Missing for 10: explicit checkpoint/save-state API, resume-from-crash mechanism, persistence backend documentation, and any hands-on confirmation of durable resumption.

   • [github] “Workflow Runtime: A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan…”
   • [github] “A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan-out/fan-in, loops…”
   • [claimed-docs] “ADK automatically filters irrelevant events, summarizes older conversational turns, lazy-loads artifacts, and tracks token usage.”

   proof ↗⚑ flag

2. engineering-leadRun long-lived agents durably across process restarts and deploys, natively or via durable-execution integrations

   weight 2 · round to Google ADK
   CrewAI!disputedcontradicted3/10

   CrewAI's enterprise/edge API exposes kickoff/status/resume endpoints suggesting some ability to pause and resume crew execution (crewai-docs-56, crewai-docs-57), but there is no documentation of durable state persistence across process restarts/deploys or integration with a durable-execution engine (e.g., Temporal, Restate). A competitor explicitly claims 'Crew AI lacks' persistent state and deterministic pause hooks that their own product offers (crewai-comm-4), directly contradicting any durability claim. missing for 10: native crash/restart recovery mechanism, documented durable-execution integration, independent verification of the resume API's persistence guarantees.

   • [claimed-docs] “POST /resume](https://docs.crewai.com/edge/en/api-reference/resume.md): Resume crew execution with human feedback”
   • [claimed-docs] “POST /kickoff](https://docs.crewai.com/edge/en/api-reference/kickoff.md): Start a crew execution - [GET /status/{kickoff_id}](https://docs.c…”
   • [community] “Competitor (Portia) founder's take: 'We think Crew AI is great. It's just that a lot of businesses we spoke to want the autonomy/decision ma…”

   proof ↗⚑ flag

   Google ADK~partialclaimed3/10

   ADK's Workflow Runtime mentions 'state management' and 'retry' in its graph-based execution engine, and deployment docs describe scalable hosting (Agent Runtime, Cloud Run), but there is no explicit evidence of session/state persistence surviving process restarts or redeploys, nor any named durable-execution integration (e.g., Temporal, Cloud Workflows checkpointing). Missing for 10: documented durable state store or checkpoint/resume mechanism, explicit claim of surviving restarts/redeploys, and any third-party durable-execution integration.

   • [github] “Workflow Runtime: A graph-based execution engine for composing deterministic execution flows for agentic apps, with support for routing, fan…”
   • [claimed-docs] “Agent Runtime is a fully managed auto-scaling service on Google Cloud specifically designed for deploying, managing, and scaling AI agents b…”
   • [claimed-docs] “Cloud Run is a managed auto-scaling compute platform on Google Cloud that enables you to run your agent as a container-based application.”
   • [claimed-docs] “ADK automatically filters irrelevant events, summarizes older conversational turns, lazy-loads artifacts, and tracks token usage.”

   proof ↗⚑ flag

Streaming

1. developerStream tokens and intermediate agent events (tool calls, steps) to my UI in real time

   weight 3 · round drawn
   CrewAI~partialclaimed4/10

   Docs mention 'Webhook Streaming' and real-time execution monitoring, but these are enterprise (CrewAI AMP) features aimed at systems integration, not documented developer-facing token streaming or step/tool-call event hooks for building a live UI. No evidence of an open-source streaming API, callback for individual tokens, or intermediate agent step events exposed to a UI. Missing for 10: token-level streaming API/SDK, documented callback hooks for tool calls/steps, and any independent/hands-on confirmation of real-time UI event feeds.

   • [claimed-docs] “Stream real-time events and updates to your systems”
   • [claimed-docs] “Webhook Streaming Stream real-time events and updates to your systems”
   • [claimed-docs] “Performance Monitoring: Track agent execution times, token usage, and resource consumption”
   • [claimed-docs] “Track agent execution times, token usage, and resource consumption”
   • [claimed-docs] “CrewAI AMP extends the power of the open-source framework with features designed for production deployments, collaboration, and scalability.…”

   proof ↗⚑ flag

   Google ADK~partialclaimed4/10

   The evidence shows a built-in Web/dev UI (`adk web`) for testing/debugging agents and a Callbacks mechanism to hook into execution-lifecycle events (tool calls, steps), which implies some visibility into intermediate agent activity, but nothing explicitly documents token-level streaming to a custom UI (no mention of SSE/websocket/streaming API). missing for 10: explicit documentation of real-time token streaming API/protocol, evidence of streaming tool-call/step events to an arbitrary UI beyond the built-in dev UI, independent confirmation of streaming behavior.

   • [github] “A built-in development UI to help you test, evaluate, debug, and showcase your agent(s).”
   • [github] “Web UI (supports multi-agent directories or pointing directly to a single agent folder)”
   • [github] “adk run path/to/my_agent # Web UI (supports multi-agent directories or pointing directly to a single agent folder) adk web path/to/agents_d…”
   • [claimed-docs] “Callbacks: Hook into specific events during an agent's execution lifecycle to add logging, monitoring, or custom side-effects without alteri…”

   proof ↗⚑ flag

Structured output

1. developerGet schema-validated structured output from an agent, with automatic retries when validation fails

   weight 3 · round drawn
   CrewAI—none0/10

   Evidence pack has no mention of Pydantic/schema output validation or automatic retry-on-validation-failure mechanisms for structured outputs; it covers agents, tasks, memory, tools, CLI, and enterprise features but nothing about structured output validation or retries.

   ⚑ flag

   Google ADK—none0/10

   No evidence in the pack mentions schema-validated structured output (e.g., Pydantic output_schema) or automatic retry-on-validation-failure behavior for ADK agents; the evidence covers agent setup, tools, workflows, deployment, and evaluation but not structured output validation. Missing for 10: any mention of output schema enforcement, structured output configuration, or validation-retry mechanism.

   ⚑ flag