Codegen vs Jules

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to Codegen
   Codegen✓fullprobed9/10

   A probe confirms Codegen's docs site serves a working llms.txt at https://docs.codegen.com/llms.txt (HTTP 200) listing structured doc links, directly enabling an agent to be pointed at agent-oriented docs; this is corroborated by extensive first-party documentation of agent-friendly interfaces (SDK, CLI, API). Missing for 10: independent third-party confirmation of agents actually consuming the llms.txt in practice.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.codegen.com/llms.txt # Codegen ## Docs - [Codegen](https://docs.codegen.com/introduction/overview…”
   • [claimed-docs] “Analyze requirements, implement features, fix bugs, write tests, and improve documentation based on your prompts.”
   • [claimed-docs] “Programmatically interact with agents using the Python SDK for advanced automation.”

   proof ↗⚑ flag

   Jules—none0/10

   Probes show no llms.txt, docs.md, or machine-readable API spec exist at expected paths, and no evidence Jules can be pointed at such agent-oriented documentation formats; Jules does support AGENTS.md for repo context but that's a different mechanism than consuming llms.txt-style docs.

   • [probe] “PROBE llms.txt: HTTP 404 at https://jules.google/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://jules.google/docs.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://jules.google/openapi.json, https://jules.google/swagger.json, https://jules.google/api/opena…”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to Codegen
   Codegen✓fullprobed8/10

   Codegen explicitly supports headless/automated usage via a Python SDK for programmatic agent invocation, a REST API with explicit CI/CD trigger support ('Trigger agents from CI/CD pipelines when builds fail'), and a CLI for scripted interactions and telemetry — all first-party documented. Missing for 10: independent/hands-on confirmation of CI usage in the wild and a working OpenAPI spec (probe found 404s), so it's vendor-doc-only evidence.

   • [claimed-docs] “Programmatically interact with agents using the Python SDK for advanced automation.”
   • [claimed-docs] “agent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")”
   • [claimed-docs] “Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic cont…”
   • [claimed-docs] “programmatically retrieve detailed agent traces for analysis and debugging”
   • [claimed-docs] “Trigger agents from CI/CD pipelines when builds fail”
   • [claimed-docs] “Use it to view agents, pull their work, create new agents, and run Claude Code with full telemetry and monitoring.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.codegen.com/openapi.json, https://docs.codegen.com/swagger.json, https://docs.codegen.c…”

   proof ↗⚑ flag

   Jules✓fullcommunity7/10

   Jules provides a public API with API keys for creating custom workflows, sending tasks programmatically, and an optional automationMode to auto-create PRs, plus a community example of building an MCP server to dispatch tasks from another tool — this directly supports headless/CI-style automation. Missing for 10: no official CI/CD integration examples (e.g. GitHub Actions), no documented webhook/polling pattern for task completion, and no independent verification of reliability at scale in automated pipelines.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”
   • [claimed-docs] “To send a message to the agent:”
   • [community] “Was able to build a personal MCP server that connects to the Jules API, letting me dispatch tasks to Jules, from Copilot Chat in VS Code.”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round to Codegen
   Codegen~partialclaimed6/10

   Codegen's docs explicitly mention connecting to Slack, Linear, Figma, databases and extending capabilities 'with custom MCP tools', and separately reference provisioning MCP servers across an org — direct evidence of MCP client integration. However, the evidence is a single brief doc mention with no detailed setup guide, supported server list, or independent/hands-on confirmation of the feature working. Missing for 10: detailed MCP integration docs/tutorial, list of supported MCP servers, independent verification of the feature in practice.

   • [claimed-docs] “Connect with Slack, Linear, Figma, databases, and extend capabilities with custom MCP tools.”
   • [claimed-docs] “Log local Claude Code instances to the cloud and provision MCP servers across your org.”

   proof ↗⚑ flag

   Jules—none0/10

   No evidence that Jules can connect to or consume external MCP servers as tool providers; the only MCP-related evidence (jules-comm-19) describes someone building an MCP server that calls INTO the Jules API, which is the reverse integration direction, not Jules plugging in MCP servers for its own tool use.

   • [community] “Was able to build a personal MCP server that connects to the Jules API, letting me dispatch tasks to Jules, from Copilot Chat in VS Code.”

   proof ↗⚑ flag

4. ai-native userUse an official CLI

   weight 2 · round to Codegen
   Codegen✓fullprobed8/10

   Codegen documents an official CLI (docs.codegen.com/introduction/cli) with concrete capabilities: viewing/pulling agent work, creating new agents, running Claude Code with telemetry, and managing organizations/repositories — matching an AI-native agentic workflow tool. missing for 10: independent/hands-on corroboration beyond first-party docs, and no details on installation/versioning maturity.

   • [claimed-docs] “Use it to view agents, pull their work, create new agents, and run Claude Code with full telemetry and monitoring.”
   • [claimed-docs] “Pull agent work - Download branches and code changes created by agents directly to your local environment”
   • [claimed-docs] “Manage organizations - Switch between organizations and configure repositories”
   • [probe] “official CLI documented at https://docs.codegen.com/introduction/cli”

   proof ↗⚑ flag

   Jules—none0/10

   Evidence shows Jules offers a web app, GitHub integration, and a REST API for custom workflows, but no official CLI tool is documented or mentioned anywhere in the evidence pack.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”

   proof ↗⚑ flag

5. ai-native userDrive the product through a documented public API

   weight 3 · round drawn
   Codegen✓fullprobed8/10

   Codegen documents a full API reference (agent creation, traces, CI/CD triggers), a Python SDK for programmatic agent control, and a CLI for managing agents/orgs, giving AI-native users multiple documented ways to drive the product programmatically. Missing for 10: a discoverable machine-readable OpenAPI/swagger spec (probe returned 404s) and independent third-party corroboration of the API's reliability.

   • [claimed-docs] “Programmatically interact with agents using the Python SDK for advanced automation.”
   • [claimed-docs] “Use it to view agents, pull their work, create new agents, and run Claude Code with full telemetry and monitoring.”
   • [claimed-docs] “agent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")”
   • [claimed-docs] “Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic cont…”
   • [claimed-docs] “programmatically retrieve detailed agent traces for analysis and debugging”
   • [claimed-docs] “Trigger agents from CI/CD pipelines when builds fail”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.codegen.com/openapi.json, https://docs.codegen.com/swagger.json, https://docs.codegen.c…”
   • [probe] “official CLI documented at https://docs.codegen.com/introduction/cli”

   proof ↗⚑ flag

   Jules✓fullprobed8/10

   Jules ships a documented public API (developers.google.com/jules/api) with API key management, custom workflows, sending messages to the agent, and automationMode config; a community user independently built an MCP server on top of the API confirming real-world programmatic access. Missing for 10: no discoverable OpenAPI/swagger spec or llms.txt (probes returned 404s), reducing machine-readability/self-service tooling confidence.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”
   • [claimed-docs] “To send a message to the agent:”
   • [community] “Was able to build a personal MCP server that connects to the Jules API, letting me dispatch tasks to Jules, from Copilot Chat in VS Code.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://jules.google/openapi.json, https://jules.google/swagger.json, https://jules.google/api/opena…”

   proof ↗⚑ flag

6. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round drawn
   Codegen—none0/10

   Codegen's SDK/API uses a simple org_id + token authentication (codegen-docs-15) with no documentation of scoped, role-based, or least-privilege credential issuance, token permissions, or granular access controls. Missing for 10: any mention of scoped API keys, permission levels, or restricted-access tokens for agents.

   • [claimed-docs] “agent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")”
   • [claimed-docs] “Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic cont…”

   proof ↗⚑ flag

   Jules—none0/10

   Jules documents a basic API key creation flow (max 3 keys) but provides no evidence of scoped or least-privilege permission controls — no mention of scopes, roles, or restricted-access tokens for the agent.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”

   proof ↗⚑ flag

7. ai-native userBuild against official SDKs

   weight 2 · round to Codegen
   Codegen✓fullprobed8/10

   Codegen documents an official Python SDK for programmatic agent control (agent.run, org_id/token auth), a full API reference with traces and CI/CD triggers, and a companion CLI — all first-party docs. missing for 10: independent/hands-on corroboration beyond docs, and no public OpenAPI spec was found (probe returned 404s).

   • [claimed-docs] “Programmatically interact with agents using the Python SDK for advanced automation.”
   • [claimed-docs] “agent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")”
   • [claimed-docs] “Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic cont…”
   • [claimed-docs] “programmatically retrieve detailed agent traces for analysis and debugging”
   • [claimed-docs] “Trigger agents from CI/CD pipelines when builds fail”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.codegen.com/openapi.json, https://docs.codegen.com/swagger.json, https://docs.codegen.c…”
   • [probe] “official CLI documented at https://docs.codegen.com/introduction/cli”

   proof ↗⚑ flag

   Jules✓fullprobed7/10

   Jules ships an official public API with documented endpoints, API key management, and use cases for building custom workflows/integrations, and a community member confirms building a personal MCP server against the Jules API. missing for 10: no official OpenAPI/SDK spec discoverable (openapi probes 404), no first-party language SDKs mentioned, and no independent SDK-quality corroboration beyond one community integration example.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”
   • [claimed-docs] “To send a message to the agent:”
   • [community] “Was able to build a personal MCP server that connects to the Jules API, letting me dispatch tasks to Jules, from Copilot Chat in VS Code.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://jules.google/openapi.json, https://jules.google/swagger.json, https://jules.google/api/opena…”

   proof ↗⚑ flag

8. ai-native userSubscribe to events via webhooks

   weight 2 · round drawn
   Codegen—none0/10

   No evidence in the pack mentions webhooks or event-subscription mechanisms; only SDK/API polling, CI/CD triggers, and integrations (Slack, Linear, Jira) are documented, none of which describe webhook subscriptions.

   ⚑ flag

   Jules—none0/10

   Evidence shows Jules has an API for creating tasks/sending messages, notifications for task completion, and GitHub integration, but there is no mention of webhooks or event subscription mechanisms anywhere in the docs or community evidence. Missing for 10: any documentation of a webhook endpoint, event subscription API, or push-based notification mechanism to external systems.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [claimed-docs] “To send a message to the agent:”
   • [claimed-docs] “You’ll be notified when the task completes or needs your input.”

   proof ↗⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round to Jules
   Codegen~partialclaimed5/10

   Codegen generates AI-driven insights specifically around code artifacts—summarizing PR diffs, running security/secrets scans, checking test coverage, and suggesting changes/comments on PRs and issues—which counts as data-driven insight generation within its coding-agent context. However, these insights are narrowly scoped to code/repo/PR data rather than broader 'your data' (e.g., databases, business metrics) despite a mention of database connectivity via MCP with no insight-generation detail there. Missing for 10: evidence of insights/suggestions generated from non-code data sources (databases, analytics), and no independent/hands-on corroboration of insight quality.

   • [claimed-docs] “Reads the PR diff Summarizes changes across files Runs security and secrets scan Checks test coverage Posts a review comment on failure”
   • [claimed-docs] “Review PRs, suggest changes, comment on issues, create branches, commit code, and manage repositories.”
   • [claimed-docs] “Connect with Slack, Linear, Figma, databases, and extend capabilities with custom MCP tools.”

   proof ↗⚑ flag

   Jules~partialcommunity6/10

   Jules generates AI-derived plans, diffs, and PR suggestions based on analysis of the user's repository data, which is the coding-agent analog of 'AI-generated insights/suggestions from your data.' Community evidence is mixed on the quality of these suggestions, with some praising usable PRs and others criticizing low-quality output on complex codebases. Missing for 10: no evidence of broader analytics-style insights beyond code-change suggestions, and no independent benchmarking confirming insight quality across use cases.

   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [community] “its way better than the github thing in my experience it produces usable PRs”
   • [community] “I've tried using Jules for a side project, and the code quality it emits is much worse than GH Copilot, Gemini CLI, and Claude Code. It also…”

   proof ↗⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Codegen
   Codegen✓fullclaimed7/10

   Codegen documents agents that trigger autonomously from CI/CD failures, Slack/Linear/Jira events, and PR activity (auto-review, auto-comment), plus SDK/CLI for programmatic background automation — this matches the story of autonomous background automations. Missing for 10: independent/hands-on verification of reliability, and no explicit scheduled/cron-trigger mechanism documented.

   • [claimed-docs] “Trigger agents from CI/CD pipelines when builds fail”
   • [claimed-docs] “Reads the PR diff Summarizes changes across files Runs security and secrets scan Checks test coverage Posts a review comment on failure”
   • [claimed-docs] “Send notifications, ask for clarification, report progress, and interact directly with your team in Slack channels.”
   • [claimed-docs] “Update statuses, add comments, link PRs to issues, and create new tasks. Support for Jira, Linear, Clickup and Monday.com.”
   • [claimed-docs] “Programmatically interact with agents using the Python SDK for advanced automation.”
   • [claimed-docs] “agent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")”
   • [claimed-docs] “Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic cont…”

   proof ↗⚑ flag

   Jules~partialcommunity6/10

   Jules supports background/autonomous execution via GitHub label-triggered tasks and a public API for building custom automations (e.g., automated bug-fixing, code review) with an optional automationMode for auto-PR creation, and it runs tasks in a cloud VM without needing the user present. However, docs also show a human-approval gate (plan approval, PR review) rather than fully hands-off automation, and community reports note tasks getting stuck, hitting limits, or needing babysitting, undercutting reliability of unattended runs. Missing for 10: evidence of true scheduled/cron-style recurring automations, and independent confirmation that automations run to completion without manual intervention.

   • [claimed-docs] “Use the "jules" label in an issue to assign a task directly in GitHub.”
   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [community] “I've moved infrastructure to Railway and taught the customer to use Jules. They make their own PRs now, Railway spins up an environment with…”
   • [community] “Jules was unable to complete the task in time. Please review the work done so far and provide feedback for Jules to continue.”
   • [community] “Jules quickly after a few messages/prompts just gets stuck in an endless loop like Gemini-cli does. The worst part is that there is no 'STOP…”

   proof ↗⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round drawn
   Codegen✓fullclaimed8/10

   Codegen's documentation clearly describes delegating coding tasks to a built-in AI agent that analyzes requirements, implements features, fixes bugs, writes tests, and reports progress via prompts, accessible through chat, Slack, CLI, and SDK. Missing for 10: independent/hands-on verification of the assistant's actual task-completion quality beyond vendor docs.

   • [claimed-docs] “Analyze requirements, implement features, fix bugs, write tests, and improve documentation based on your prompts.”
   • [claimed-docs] “Send notifications, ask for clarification, report progress, and interact directly with your team in Slack channels.”
   • [claimed-docs] “Review PRs, suggest changes, comment on issues, create branches, commit code, and manage repositories.”
   • [claimed-docs] “agent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")”
   • [claimed-docs] “Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic cont…”

   proof ↗⚑ flag

   Jules✓fullcommunity8/10

   Jules is exactly this kind of built-in agentic assistant: users delegate coding tasks via GitHub issue labels, the web app, or API, and Jules autonomously plans, clones repos, runs in a VM, and returns diffs/PRs for approval (jules-docs-2,3,4,7,8,9). Community evidence corroborates real-world task delegation and usable output (jules-comm-2, jules-comm-11, jules-comm-15, jules-comm-16), though mixed reports of loops, babysitting needs, and reliability issues (jules-comm-14, jules-comm-18) temper quality. Missing for 10: independent benchmarking of task success rates and evidence of consistent reliability at scale.

   • [claimed-docs] “Use the "jules" label in an issue to assign a task directly in GitHub.”
   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [community] “I used Jules three times today, very impressive! It also handles coding-adjacent work. Good github integrations.”
   • [community] “I've used this tool for a few months now and have been pretty impressed by it. It handles large quantities of tasks very well and is good at…”
   • [community] “I gave Jules a try on a new, very unorganized Python project. It failed the first time, I gave it the error message and it fixed it. I was s…”
   • [community] “I've moved infrastructure to Railway and taught the customer to use Jules. They make their own PRs now, Railway spins up an environment with…”
   • [community] “Jules quickly after a few messages/prompts just gets stuck in an endless loop like Gemini-cli does. The worst part is that there is no 'STOP…”
   • [community] “Do people really find Jules useful? I find it needs babysitting much more than Cursor.”

   proof ↗⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round drawn
   Codegen✓fullclaimed8/10

   Codegen is fundamentally natural-language driven: docs show prompt-based task execution ('Leave a review on PR #123'), Slack/Jira/Linear interactions, and SDK agent.run(prompt=...) all operate via NL commands across code, PRs, and issue trackers. This spans chat, CLI, and SDK surfaces, consistently prompt/NL-driven. Missing for 10: independent/hands-on corroboration of NL command interpretation quality and edge cases beyond first-party docs.

   • [claimed-docs] “Analyze requirements, implement features, fix bugs, write tests, and improve documentation based on your prompts.”
   • [claimed-docs] “Send notifications, ask for clarification, report progress, and interact directly with your team in Slack channels.”
   • [claimed-docs] “Update statuses, add comments, link PRs to issues, and create new tasks. Support for Jira, Linear, Clickup and Monday.com.”
   • [claimed-docs] “Review PRs, suggest changes, comment on issues, create branches, commit code, and manage repositories.”
   • [claimed-docs] “agent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")”
   • [claimed-docs] “Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic cont…”

   proof ↗⚑ flag

   Jules✓fullcommunity8/10

   Jules is designed around natural-language task submission and messaging: users submit a task description, Jules generates a plan, and you can send follow-up messages to the agent via API or web app (jules-docs-4, jules-docs-12). Community reports confirm real-world use of NL prompts to drive coding tasks, including iterative feedback and error-message follow-ups (jules-comm-2, jules-comm-15, jules-comm-16). Missing for 10: no detailed documentation of the full range/complexity of natural-language commands supported (e.g., multi-step conversational control, command reference) and no independent benchmark of NL command robustness beyond anecdotal reports.

   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “To send a message to the agent:”
   • [community] “I used Jules three times today, very impressive! It also handles coding-adjacent work. Good github integrations.”
   • [community] “I gave Jules a try on a new, very unorganized Python project. It failed the first time, I gave it the error message and it fixed it. I was s…”
   • [community] “I've moved infrastructure to Railway and taught the customer to use Jules. They make their own PRs now, Railway spins up an environment with…”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   Codegen—none0/10

   Codegen has an API reference page (overview, SDK docs) but no evidence of an interactive reference with runnable/executable examples; the OpenAPI/spec probe returned 404 for all candidate paths, suggesting no interactive API explorer exists.

   • [claimed-docs] “Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic cont…”
   • [claimed-docs] “programmatically retrieve detailed agent traces for analysis and debugging”
   • [claimed-docs] “Trigger agents from CI/CD pipelines when builds fail”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.codegen.com/openapi.json, https://docs.codegen.com/swagger.json, https://docs.codegen.c…”

   proof ↗⚑ flag

   Jules—none0/10

   Jules documents an API (create tasks, send messages, API keys) but there is no evidence of an interactive API reference or runnable code examples; probes for openapi/swagger specs all returned 404s, suggesting no such interactive reference exists.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [claimed-docs] “To send a message to the agent:”
   • [probe] “PROBE openapi: all candidate paths 404 (https://jules.google/openapi.json, https://jules.google/swagger.json, https://jules.google/api/opena…”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   Codegen—none0/10

   Codegen has an API reference (docs-16/17/18) but explicit probes for machine-readable spec files (openapi.json, swagger.json, etc.) all returned 404, indicating no downloadable OpenAPI/machine-readable spec is available.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.codegen.com/openapi.json, https://docs.codegen.com/swagger.json, https://docs.codegen.c…”
   • [claimed-docs] “Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic cont…”

   proof ↗⚑ flag

   Jules—none0/10

   Jules ships a documented REST API (jules-docs-9..12) but there is no evidence of a downloadable OpenAPI/Swagger spec; explicit probes for llms.txt, docs.md, and standard OpenAPI paths all returned 404 (jules-probe-1, jules-probe-2, jules-probe-3), confirming no machine-readable spec is published.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [probe] “PROBE llms.txt: HTTP 404 at https://jules.google/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://jules.google/docs.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://jules.google/openapi.json, https://jules.google/swagger.json, https://jules.google/api/opena…”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round drawn
   Codegen~partialclaimed5/10

   Codegen docs state that code changes run in 'robust isolated environments' to safely execute code and test changes, which implies sandboxed testing separate from production. However, there's no explicit description of how production data is excluded or protected, no detail on the sandbox's scope/persistence, and no independent verification of this isolation claim. missing for 10: explicit production-data isolation guarantees, technical detail on sandbox architecture, independent/hands-on confirmation.

   • [claimed-docs] “Safely run code, install dependencies, and test changes in robust isolated environments”

   proof ↗⚑ flag

   Jules~partialclaimed5/10

   Jules runs tasks in an isolated cloud VM that clones the repo, meaning code changes happen in a sandboxed environment rather than directly on production, and PRs must be reviewed/approved before merging to the real branch. However, this is a code-sandbox for making changes, not a dedicated 'test against sandbox data/environment' feature, and there's no evidence of test-data isolation, staging environment provisioning, or explicit protection of production data/services beyond the VM/PR review flow. missing for 10: explicit sandbox test-data isolation, staging/production separation guarantees, independent verification that production systems are never touched.

   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Codegen—none0/10

   There's an API reference and SDK docs, but no evidence of API versioning scheme or a documented deprecation policy; the openapi probe even returned 404s, suggesting no formal spec surfacing version info.

   • [claimed-docs] “Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic cont…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.codegen.com/openapi.json, https://docs.codegen.com/swagger.json, https://docs.codegen.c…”

   proof ↗⚑ flag

   Jules—none0/10

   Jules has a public API (jules-docs-9, jules-docs-10) but there is no evidence of API versioning scheme or a documented deprecation policy; probes for OpenAPI spec/docs.md/llms.txt all returned 404 (jules-probe-1, jules-probe-2, jules-probe-3), suggesting no formal machine-readable API contract or lifecycle documentation is available.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [probe] “PROBE llms.txt: HTTP 404 at https://jules.google/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://jules.google/docs.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://jules.google/openapi.json, https://jules.google/swagger.json, https://jules.google/api/opena…”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round drawn
   Codegen~partialclaimed5/10

   The Python SDK and API allow programmatic creation/monitoring of agents and triggering from CI/CD, which could be scripted in a loop to perform bulk operations, but there is no documented batch/bulk endpoint or explicit guidance for operating across many items at once. missing for 10: dedicated bulk/batch API or CLI command, documented examples of multi-item automation, and evidence of scale/performance handling for bulk workloads.

   • [claimed-docs] “Programmatically interact with agents using the Python SDK for advanced automation.”
   • [claimed-docs] “agent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")”
   • [claimed-docs] “Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic cont…”
   • [claimed-docs] “Trigger agents from CI/CD pipelines when builds fail”

   proof ↗⚑ flag

   Jules~partialcommunity5/10

   Jules exposes an API that lets users script custom workflows and dispatch tasks programmatically (jules-docs-9, jules-docs-12), and usage-limit tiers explicitly target 'power users & agent-heavy workflows' with up to 300 tasks (jules-docs-13), while community reports confirm it 'handles large quantities of tasks very well' and users built custom dispatch tooling via the API/MCP bridge (jules-comm-11, jules-comm-19). However, there's no documented native UI or endpoint for a single bulk action across many items (e.g., batch-applying one task to many repos/issues) — each task still appears to be created and reviewed individually via GitHub labels or API calls. Missing for 10: a documented batch/bulk endpoint or UI feature, first-party bulk-operation examples, and independent verification of true parallel bulk execution rather than just high per-day task volume.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “To send a message to the agent:”
   • [claimed-docs] “Power users & agent-heavy workflows ... 300 ... 60”
   • [community] “I've used this tool for a few months now and have been pretty impressed by it. It handles large quantities of tasks very well and is good at…”
   • [community] “Was able to build a personal MCP server that connects to the Jules API, letting me dispatch tasks to Jules, from Copilot Chat in VS Code.”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round drawn
   Codegen~partialclaimed5/10

   Codegen supports event-driven automation via CI/CD triggers (docs-18) and PR review triggers on failure (docs-20), plus SDK/API for programmatic agent invocation, but there's no evidence of a general user-facing rules engine for defining arbitrary trigger-condition-action automations across events. missing for 10: a documented rules/automation builder UI, broader event types beyond CI/CD failures and PR diffs, and independent confirmation of custom trigger logic.

   • [claimed-docs] “Trigger agents from CI/CD pipelines when builds fail”
   • [claimed-docs] “Reads the PR diff Summarizes changes across files Runs security and secrets scan Checks test coverage Posts a review comment on failure”
   • [claimed-docs] “Programmatically interact with agents using the Python SDK for advanced automation.”
   • [claimed-docs] “Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic cont…”

   proof ↗⚑ flag

   Jules~partialclaimed5/10

   Jules supports some event-driven automation — e.g., assigning a 'jules' label to a GitHub issue automatically triggers a task, and the API lets developers build custom workflows/automations around Jules (jules-docs-2, jules-docs-9). However, there's no evidence of a general rules/conditions engine where users define arbitrary triggers and actions beyond this specific label mechanism and API calls. Missing for 10: a configurable rules interface (conditions + multiple trigger types), documentation of scheduled/webhook-based triggers, and independent confirmation that automated rule-based triggering works reliably in practice.

   • [claimed-docs] “Use the "jules" label in an issue to assign a task directly in GitHub.”
   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round drawn
   Codegen—none0/10

   No evidence of any scheduling, cron-like, or recurring job/workflow capability; the docs cover SDK/API triggers, CI/CD triggers, and manual agent runs, but nothing about time-based or recurring automation.

   ⚑ flag

   Jules—none0/10

   Evidence covers task assignment via GitHub labels, API-driven custom workflows, and one-off task automation, but nothing describes recurring/scheduled jobs (e.g., cron-like triggers) — the API docs only mention creating tasks and sending messages, not recurrence.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”
   • [claimed-docs] “To send a message to the agent:”

   proof ↗⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round drawn
   Codegen—none0/10

   Evidence covers PR/code review, commits, branches, and traces/telemetry, but there is no mention of versioning automations themselves, reviewing automation history, or rolling back agent configurations/workflows. missing for 10: automation versioning system, rollback mechanism for agent workflows, audit/history UI for automations.

   ⚑ flag

   Jules—none0/10

   Jules produces PRs and diffs via GitHub but there is no evidence of a mechanism to version, review history of, or roll back automations/tasks themselves (as opposed to code changes tracked by git/GitHub). missing for 10: automation versioning/history feature, rollback/undo of Jules tasks, changelog or audit trail for automations.

   ⚑ flag

End to end feature delivery

1. ai-native userHave an agent automatically generate and run tests to validate its own code changes before proposing them

   weight 2 · round to Codegen
   Codegen~partialclaimed6/10

   Docs state the agent writes tests as part of implementing features and can run code/install dependencies/test changes in isolated environments, and a PR-review flow checks test coverage before flagging issues — together implying self-validation via generated tests. However, there is no explicit end-to-end walkthrough showing the agent generating tests, running them, and iterating/fixing before proposing a PR, nor independent/hands-on confirmation. missing for 10: explicit documented workflow tying test generation+execution to pre-PR validation, independent corroboration of this loop working in practice.

   • [claimed-docs] “Analyze requirements, implement features, fix bugs, write tests, and improve documentation based on your prompts.”
   • [claimed-docs] “Safely run code, install dependencies, and test changes in robust isolated environments”
   • [claimed-docs] “Reads the PR diff Summarizes changes across files Runs security and secrets scan Checks test coverage Posts a review comment on failure”

   proof ↗⚑ flag

   Jules~partialcommunity4/10

   Jules runs code changes in an isolated VM and generates diffs/PRs for review, and one community report notes it is 'good at making tests for very specific/isolated functions,' implying some test generation capability, but there is no first-party documentation describing an explicit test-generate-and-run validation loop before proposing changes. missing for 10: official docs on automated test generation/execution as a validation step, evidence of test results being surfaced to users before PR creation, and independent confirmation across varied codebases.

   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [community] “I've used this tool for a few months now and have been pretty impressed by it. It handles large quantities of tasks very well and is good at…”

   proof ↗⚑ flag

2. developerHave an agent autonomously diagnose and fix a reported bug

   weight 3 · round to Jules
   Codegen~partialclaimed6/10

   Docs explicitly claim the agent can 'analyze requirements, implement features, fix bugs, write tests' and can be triggered from issue trackers/CI to autonomously act, plus run in isolated environments to test changes — directly supporting autonomous bug diagnosis and fix. However, all evidence is first-party marketing/docs with no independent or hands-on validation of end-to-end autonomous diagnosis-to-fix workflows, and no detailed walkthrough of the diagnosis step itself (e.g., root-cause analysis) beyond generic 'fix bugs' language. Missing for 10: independent/hands-on verification of successful autonomous bug fixes, detailed diagnostic reasoning process, and success-rate or case-study evidence.

   • [claimed-docs] “Analyze requirements, implement features, fix bugs, write tests, and improve documentation based on your prompts.”
   • [claimed-docs] “Update statuses, add comments, link PRs to issues, and create new tasks. Support for Jira, Linear, Clickup and Monday.com.”
   • [claimed-docs] “Safely run code, install dependencies, and test changes in robust isolated environments”
   • [claimed-docs] “Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic cont…”
   • [claimed-docs] “Trigger agents from CI/CD pipelines when builds fail”

   proof ↗⚑ flag

   Jules✓fullcommunity8/10

   Jules is designed to autonomously diagnose and fix issues: it clones the repo into a VM, generates a plan, makes code changes, and opens a PR for review, and can be triggered directly from a GitHub issue via the 'jules' label (a common bug-report workflow). Community evidence corroborates real-world bug-fixing use, including a case where it failed then successfully fixed the issue after being given the error message. Missing for 10: independent benchmarking on bug-fix success rate and more consistent evidence across complex codebases (some reports of failures/loops on harder tasks).

   • [claimed-docs] “Use the "jules" label in an issue to assign a task directly in GitHub.”
   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [community] “I gave Jules a try on a new, very unorganized Python project. It failed the first time, I gave it the error message and it fixed it. I was s…”
   • [community] “I've moved infrastructure to Railway and taught the customer to use Jules. They make their own PRs now, Railway spins up an environment with…”

   proof ↗⚑ flag

3. product-managerGo from a mockup or design to a working implementation without an engineering handoff

   weight 2 · round to Codegen
   Codegen~partialclaimed4/10

   Codegen's docs show relevant building blocks — a Figma connector, Slack/Linear interfaces that let non-engineers prompt the agent, and full PR/branch creation — implying a PM could describe a design and get code without direct engineering handoff. However, there is no concrete workflow, case study, or example showing a mockup/design being turned into a working implementation end-to-end. Missing for 10: an explicit design-to-code workflow example, evidence of Figma-to-PR pipeline, and PM-specific case studies or testimonials showing the handoff-free process actually works.

   • [claimed-docs] “Connect with Slack, Linear, Figma, databases, and extend capabilities with custom MCP tools.”
   • [claimed-docs] “Send notifications, ask for clarification, report progress, and interact directly with your team in Slack channels.”
   • [claimed-docs] “Review PRs, suggest changes, comment on issues, create branches, commit code, and manage repositories.”
   • [claimed-docs] “Add the Codegen Slack App to communicate with the agent directly in your workspace.”
   • [claimed-docs] “Connect your Linear workspace to enable agent interactions with your issues.”

   proof ↗⚑ flag

   Jules—none0/10

   Evidence shows Jules operates on code repositories via GitHub issues/tasks, plans, diffs, and PRs, but there is no mention of accepting mockups/designs as input or any PM-oriented no-code workflow — usage still requires repo access, issue creation, and reviewing technical diffs/plans. Nothing in the pack demonstrates a design-to-implementation path bypassing engineering.

   • [claimed-docs] “Jules needs access to your repositories in order to work.”
   • [claimed-docs] “Use the "jules" label in an issue to assign a task directly in GitHub.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”

   proof ↗⚑ flag

4. developerHave an agent implement a requested feature end-to-end, including writing tests

   weight 3 · round to Codegen
   Codegen✓fullclaimed8/10

   Docs explicitly describe agents analyzing requirements, implementing features, writing tests, and running/testing changes in isolated environments, plus PR/branch management for end-to-end delivery. Missing for 10: independent/hands-on validation of end-to-end feature delivery and no concrete example transcript showing a full feature+test cycle.

   • [claimed-docs] “Analyze requirements, implement features, fix bugs, write tests, and improve documentation based on your prompts.”
   • [claimed-docs] “Safely run code, install dependencies, and test changes in robust isolated environments”
   • [claimed-docs] “Review PRs, suggest changes, comment on issues, create branches, commit code, and manage repositories.”
   • [claimed-docs] “Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic cont…”

   proof ↗⚑ flag

   Jules~partialcommunity6/10

   Jules is explicitly built for autonomous end-to-end coding: it clones the repo, generates a plan, modifies files, produces a diff, and opens a PR (jules-docs-3/4/7/8), and one user specifically notes it's 'good at making tests for very specific/isolated functions' (jules-comm-11), with others confirming usable PRs and time savings (jules-comm-6, jules-comm-16). However, community evidence shows significant caveats: it struggles on complex/monorepo codebases, gets stuck in loops without a stop button, needs heavy babysitting, and often fails to complete tasks unassisted (jules-comm-3, jules-comm-9, jules-comm-10, jules-comm-13, jules-comm-14, jules-comm-18). Missing for 10: consistent reliability across complex real-world features, broader evidence of comprehensive test coverage (not just isolated functions), and independent benchmarks confirming end-to-end success rate.

   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [community] “I've used this tool for a few months now and have been pretty impressed by it. It handles large quantities of tasks very well and is good at…”
   • [community] “its way better than the github thing in my experience it produces usable PRs”
   • [community] “I've moved infrastructure to Railway and taught the customer to use Jules. They make their own PRs now, Railway spins up an environment with…”
   • [community] “I've been playing with it, and I've been generally not impressed. There are obvious annoying UI bugs and the output isn't very good for anyt…”
   • [community] “I've tried using Jules for a side project, and the code quality it emits is much worse than GH Copilot, Gemini CLI, and Claude Code. It also…”
   • [community] “Jules quickly after a few messages/prompts just gets stuck in an endless loop like Gemini-cli does. The worst part is that there is no 'STOP…”
   • [community] “Do people really find Jules useful? I find it needs babysitting much more than Cursor.”

   proof ↗⚑ flag

Environment setup

1. developerHave an agent automatically clone the repo, install dependencies, and configure its own working environment

   weight 2 · round drawn
   Codegen✓fullclaimed8/10

   Docs describe agents that access repos via GitHub App integration, run in isolated environments, and 'safely run code, install dependencies, and test changes' automatically without complex setup, matching the story of autonomous environment setup. Missing for 10: independent/hands-on verification of environment provisioning behavior and more detail on how dependency installation is auto-configured beyond the brief doc mention.

   • [claimed-docs] “Safely run code, install dependencies, and test changes in robust isolated environments”
   • [claimed-docs] “Install the GitHub App to grant the agent access to your repositories. No complex setup required.”
   • [claimed-docs] “Review PRs, suggest changes, comment on issues, create branches, commit code, and manage repositories.”

   proof ↗⚑ flag

   Jules✓fullcommunity8/10

   Jules' core workflow is documented: it runs in a cloud VM, clones the repo, installs dependencies, and configures its environment automatically before making changes, corroborated by community usage reports of automated PR generation. missing for 10: independent technical verification of dependency-install robustness across complex/monorepo setups, and some community reports note environment/config confusion in bespoke or monorepo projects.

   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [claimed-docs] “Jules needs access to your repositories in order to work.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [community] “I've moved infrastructure to Railway and taught the customer to use Jules. They make their own PRs now, Railway spins up an environment with…”
   • [community] “Jules runs in cloud-based VMs instead of on my local machine, making it much less useful than Claude Code. My projects have bespoke build sc…”
   • [community] “I've tried using Jules for a side project, and the code quality it emits is much worse than GH Copilot, Gemini CLI, and Claude Code. It also…”

   proof ↗⚑ flag

Interactive takeover

1. developerTake over an in-progress agent task in my editor, terminal, or browser to finish or redirect the work

   weight 2 · round drawn
   Codegen~partialclaimed5/10

   Codegen supports interacting with agents across multiple surfaces (Slack notifications/clarification requests, CLI to pull agent branches locally, Python SDK to run/monitor agents) and lets you download in-progress work to your local environment, which implies some ability to pick up and continue work. However there is no explicit documentation of pausing, redirecting, or steering a task mid-execution from editor/terminal/browser as a unified handoff flow. Missing for 10: explicit 'takeover' or redirect mid-task feature, editor/IDE integration for live task control, and evidence of seamless cross-surface handoff (Slack->CLI->browser) of the same in-progress task.

   • [claimed-docs] “Send notifications, ask for clarification, report progress, and interact directly with your team in Slack channels.”
   • [claimed-docs] “Use it to view agents, pull their work, create new agents, and run Claude Code with full telemetry and monitoring.”
   • [claimed-docs] “Pull agent work - Download branches and code changes created by agents directly to your local environment”
   • [claimed-docs] “agent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")”
   • [claimed-docs] “Add the Codegen Slack App to communicate with the agent directly in your workspace.”

   proof ↗⚑ flag

   Jules~partialcommunity5/10

   Jules supports reviewing plans, diffs, and sending follow-up messages to redirect a task (jules-docs-4, jules-docs-6, jules-docs-7, jules-docs-12), and third parties have wired the API into VS Code/Copilot Chat to dispatch tasks (jules-comm-19), but this is dispatch, not mid-task takeover. A hands-on report explicitly notes there's no 'STOP' button to interrupt a looping task (jules-comm-14), undercutting redirect control, and all takeover is browser/API-based with no native terminal or editor 'take over' UI documented. Missing for 10: documented in-editor/terminal takeover UI, ability to pause/interrupt an in-progress run, and evidence the API-based messaging genuinely redirects rather than just appends instructions.

   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “You’ll be notified when the task completes or needs your input.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “To send a message to the agent:”
   • [community] “Jules quickly after a few messages/prompts just gets stuck in an endless loop like Gemini-cli does. The worst part is that there is no 'STOP…”
   • [community] “Was able to build a personal MCP server that connects to the Jules API, letting me dispatch tasks to Jules, from Copilot Chat in VS Code.”

   proof ↗⚑ flag

2. developerSend follow-up instructions to an active agent session to steer its work without restarting

   weight 2 · round to Jules
   Codegen—none0/10

   No evidence describes sending follow-up instructions to an already-running agent session to redirect its work; docs only describe initiating agents via prompts, Slack notifications, or SDK task creation, not mid-session steering. Missing for 10: any documentation of a 'continue/steer/update' API, chat-like follow-up in an active run, or session-resume mechanism.

   ⚑ flag

   Jules✓fullcommunity7/10

   Jules's API docs explicitly describe sending a message to an active agent session (jules-docs-12), and community reports confirm the workflow of reviewing partial work and providing feedback for Jules to continue rather than restarting (jules-comm-3). This shows the steering capability works in practice, though documentation on how follow-ups affect an in-progress plan/execution is thin. Missing for 10: detailed first-party documentation of mid-task message handling/UI chat thread, and independent hands-on confirmation of steering effectiveness beyond one anecdote.

   • [claimed-docs] “To send a message to the agent:”
   • [claimed-docs] “You’ll be notified when the task completes or needs your input.”
   • [community] “Jules was unable to complete the task in time. Please review the work done so far and provide feedback for Jules to continue.”

   proof ↗⚑ flag

Sandbox execution

1. developerHave an agent safely execute code and install dependencies inside an isolated sandbox

   weight 3 · round to Jules
   Codegen✓fullclaimed7/10

   First-party docs explicitly state the agent can 'Safely run code, install dependencies, and test changes in robust isolated environments,' directly matching the story. missing for 10: independent/hands-on corroboration of sandbox isolation, technical details on sandbox implementation (e.g., container tech, resource limits) beyond a single doc line.

   • [claimed-docs] “Safely run code, install dependencies, and test changes in robust isolated environments”

   proof ↗⚑ flag

   Jules✓fullcommunity8/10

   Jules explicitly runs in a cloud VM that clones code and installs dependencies, isolated from the user's local machine, with a plan-review step before code changes are made — directly matching the sandboxed execution story. Community reports corroborate it actually running tasks end-to-end (producing PRs) in this VM environment, though some criticize reliability/loops on complex codebases. Missing for 10: detailed docs on sandbox security boundaries (network isolation, resource limits) and independent security audit of the VM isolation.

   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [community] “Jules runs in cloud-based VMs instead of on my local machine, making it much less useful than Claude Code. My projects have bespoke build sc…”
   • [community] “I've moved infrastructure to Railway and taught the customer to use Jules. They make their own PRs now, Railway spins up an environment with…”

   proof ↗⚑ flag

Approval controls

1. developerConfigure an agent to auto-approve all its actions instead of confirming each one

   weight 2 · round to Jules
   Codegen—none0/10

   The evidence describes Codegen's autonomous capabilities (running code, committing, creating PRs, Slack/Jira integration) but never mentions any per-action confirmation mode or an auto-approve toggle/setting for oversight control. Missing for 10: any documentation of an approval/confirmation workflow, a setting to enable/disable auto-approval, or human-in-the-loop gating that this story presupposes.

   ⚑ flag

   Jules~partialclaimed4/10

   Jules's default workflow requires manual approval at multiple stages (plan approval, diff review, PR approval), but the API docs mention an optional `automationMode` field that changes default behavior around automatic PR creation, suggesting some configurable auto-approve path exists via API rather than the standard UI flow. Missing for 10: explicit documentation of a setting that suppresses plan-approval and diff-review confirmations entirely, and any hands-on/community confirmation that this automation mode actually skips human checkpoints.

   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”

   proof ↗⚑ flag

2. product-managerApprove key agent decisions from my phone while agents continue working

   weight 1 · round to Jules
   Codegen~partialclaimed3/10

   Codegen supports Slack-based notifications and agent interaction (docs-2, docs-9), which could technically be used from a phone via the Slack app, but there is no explicit mobile app, push-approval workflow, or documented 'approve on the go' feature for key agent decisions. missing for 10: dedicated mobile/approval UI, evidence of approving specific agent decisions (e.g., merge/deploy gates) from a phone, and any hands-on confirmation of this workflow.

   • [claimed-docs] “Send notifications, ask for clarification, report progress, and interact directly with your team in Slack channels.”
   • [claimed-docs] “Add the Codegen Slack App to communicate with the agent directly in your workspace.”

   proof ↗⚑ flag

   Jules~partialcommunity6/10

   Jules is a cloud-based agent with a web app and notification system, plan approval, diff review, and PR approval steps, and community evidence confirms users review/approve PRs from a browser including 'coding on my phone' via mobile web. However, there is no dedicated native mobile app or explicit mobile-optimized approval UI, and no evidence of push-notification-driven approval flows tailored to on-the-go PM decision-making. missing for 10: dedicated mobile app or mobile-specific UI, evidence of push notifications enabling quick phone-based approvals, PM-specific (non-developer) approval workflow.

   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “You’ll be notified when the task completes or needs your input.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [community] “I've been actually kind-of enjoying using Jules as a way of 'coding' my side project using my phone. By the time I get home in the evening I…”

   proof ↗⚑ flag

3. engineering-leadSet tiered autonomy levels controlling what an agent can do without manual confirmation

   weight 3 · round to Jules
   Codegen—none0/10

   No evidence describes configurable autonomy tiers, approval gates, or permission levels for agent actions; the docs describe what agents can do (code, PRs, Slack, Jira/Linear) but nothing about setting graduated confirmation thresholds for an engineering lead to control autonomy.

   ⚑ flag

   Jules~partialclaimed4/10

   Jules has a binary plan-approval workflow (review/approve plan, review diffs, approve PRs) and an API `automationMode` field that toggles whether PRs are auto-created, but there's no evidence of configurable tiered autonomy levels (e.g., low/medium/high trust settings, granular permission scopes, or per-action confirmation thresholds) that an engineering-lead could set. Missing for 10: explicit multi-tier autonomy/permission settings, admin-configurable trust levels, and any org-wide policy controls beyond the single automationMode toggle and default plan-approval gate.

   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”

   proof ↗⚑ flag

Model control

1. ai-native userHave each task prompt automatically routed to the most suitable underlying model

   weight 2 · round drawn
   Codegen—none0/10

   No evidence anywhere in the pack of automatic model routing per task/prompt; Codegen's docs describe agent capabilities, integrations, CLI/SDK, but nothing about selecting or routing between underlying LLMs based on task type.

   ⚑ flag

   Jules—none0/10

   No evidence Jules performs automatic model routing per task; docs mention only a single flagship model (Gemini 3 Pro) as 'priority access,' with no mention of routing prompts to different models based on suitability. Missing for 10: any documentation of multi-model routing logic, model-selection criteria, or automatic switching between models.

   • [claimed-docs] “Priority access to the latest models, starting with Gemini 3 Pro”

   proof ↗⚑ flag

2. engineering-leadSwitch away from automatic model selection to a specific model of my choice

   weight 1 · round drawn
   Codegen—none0/10

   No evidence in the pack mentions model selection, choice of underlying LLM, or any setting to override automatic model routing; the docs cover integrations, CLI, SDK, and agent workflows but never reference model choice.

   ⚑ flag

   Jules—none0/10

   No evidence pack item mentions model selection settings or the ability to choose a specific underlying model; the only related mention is 'Priority access to the latest models, starting with Gemini 3 Pro' which describes access tiers, not user-controlled model switching.

   ⚑ flag

Visibility monitoring

1. developerWatch what a running agent is doing in real time, including its current status

   weight 3 · round to Codegen
   Codegen~partialclaimed6/10

   Codegen's CLI lets developers view agents and run them with 'full telemetry and monitoring,' and agents can 'report progress' via Slack, giving some real-time visibility into agent activity and status. However, there's no explicit documentation of a live dashboard/status stream showing step-by-step actions as they happen, and the closest 'trace' feature is framed as post-hoc analysis/debugging rather than live watching. Missing for 10: a documented real-time UI/dashboard showing live agent status, explicit streaming of in-progress actions, and independent/hands-on confirmation of live monitoring accuracy.

   • [claimed-docs] “Use it to view agents, pull their work, create new agents, and run Claude Code with full telemetry and monitoring.”
   • [claimed-docs] “Send notifications, ask for clarification, report progress, and interact directly with your team in Slack channels.”
   • [claimed-docs] “programmatically retrieve detailed agent traces for analysis and debugging”
   • [claimed-docs] “Pull agent work - Download branches and code changes created by agents directly to your local environment”

   proof ↗⚑ flag

   Jules~partialcommunity5/10

   Jules provides notifications on task completion/input-needed and a plan-approval step, implying some status visibility, plus a diff/PR review flow, but there is no evidence of a live real-time activity feed or streaming log of the agent's current actions while running; community feedback even notes the absence of a stop/interrupt control mid-run, suggesting limited real-time observability. Missing for 10: evidence of a real-time execution log/console view, granular step-by-step status updates, and independent confirmation of live monitoring UX.

   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “You’ll be notified when the task completes or needs your input.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [community] “Jules quickly after a few messages/prompts just gets stuck in an endless loop like Gemini-cli does. The worst part is that there is no 'STOP…”

   proof ↗⚑ flag

2. developerGet notified when an agent completes a task or needs my input

   weight 2 · round drawn
   Codegen✓fullclaimed8/10

   Codegen explicitly supports sending notifications, asking for clarification, and reporting progress directly in Slack channels, plus updating statuses/comments in Jira, Linear, Clickup, and Monday.com, covering both completion and input-needed scenarios. missing for 10: independent/hands-on confirmation of real-time notification delivery, and no explicit mention of email/other channel notification options.

   • [claimed-docs] “Send notifications, ask for clarification, report progress, and interact directly with your team in Slack channels.”
   • [claimed-docs] “Update statuses, add comments, link PRs to issues, and create new tasks. Support for Jira, Linear, Clickup and Monday.com.”
   • [claimed-docs] “Add the Codegen Slack App to communicate with the agent directly in your workspace.”
   • [claimed-docs] “Connect your Linear workspace to enable agent interactions with your issues.”

   proof ↗⚑ flag

   Jules✓fullcommunity8/10

   First-party docs explicitly state users are notified when a task completes or needs input, and the workflow (plan approval, diff review, PR creation) reinforces oversight checkpoints where notifications matter. Community evidence corroborates the async, check-back-later usage pattern (e.g. reviewing PRs after time away), consistent with notification-driven workflows. Missing for 10: independent verification of notification channels (email/push/Slack), and no detail on notification reliability or configurability.

   • [claimed-docs] “You’ll be notified when the task completes or needs your input.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [community] “I've been actually kind-of enjoying using Jules as a way of 'coding' my side project using my phone. By the time I get home in the evening I…”
   • [community] “Jules was unable to complete the task in time. Please review the work done so far and provide feedback for Jules to continue.”

   proof ↗⚑ flag

Natural language task intake

1. developerDescribe a feature or bug in plain language and have it automatically turned into a scoped implementation task

   weight 3 · round to Codegen
   Codegen✓fullclaimed7/10

   Codegen's docs describe taking natural-language prompts and turning them into scoped agent tasks: analyzing requirements, implementing features, fixing bugs, writing tests, and running agents from prompts via SDK/CLI/Slack/issue trackers. This directly matches the intent-to-spec story of describing a feature/bug and having it become an actionable implementation task. Missing for 10: no independent/hands-on evidence showing the quality or accuracy of the generated scoped task (e.g. how well it decomposes ambiguous requests), and no detail on how it handles clarification/scoping edge cases beyond a general 'ask for clarification' mention.

   • [claimed-docs] “Analyze requirements, implement features, fix bugs, write tests, and improve documentation based on your prompts.”
   • [claimed-docs] “agent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")”
   • [claimed-docs] “Update statuses, add comments, link PRs to issues, and create new tasks. Support for Jira, Linear, Clickup and Monday.com.”
   • [claimed-docs] “Send notifications, ask for clarification, report progress, and interact directly with your team in Slack channels.”

   proof ↗⚑ flag

   Jules~partialcommunity6/10

   Jules docs clearly show a plain-language task submission flow that generates a reviewable/approvable plan before code changes, then produces a diff and PR (jules-docs-4, jules-docs-7, jules-docs-8), directly matching the intent-to-spec story. However community reports show inconsistent scoping quality — confusion in monorepos, endless loops, and needing heavy babysitting — indicating the automatic scoping doesn't always hold up in practice. Missing for 10: independent verification of plan/spec quality on complex codebases, and consistent evidence the generated plan reliably matches developer intent without back-and-forth correction.

   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [community] “I've tried using Jules for a side project, and the code quality it emits is much worse than GH Copilot, Gemini CLI, and Claude Code. It also…”
   • [community] “Jules quickly after a few messages/prompts just gets stuck in an endless loop like Gemini-cli does. The worst part is that there is no 'STOP…”
   • [community] “Do people really find Jules useful? I find it needs babysitting much more than Cursor.”

   proof ↗⚑ flag

2. product-managerConvert user feedback submissions into structured tasks with proposed scope

   weight 2 · round to Codegen
   Codegen~partialclaimed3/10

   Codegen can analyze requirements and create/link tasks in Jira, Linear, Clickup, Monday.com, which could theoretically be used to turn feedback into tasks, but there is no evidence of a feedback-intake feature or automatic scope proposal tailored for PM workflows. Missing for 10: dedicated feedback ingestion mechanism, structured scope/estimate output, PM-specific workflow examples or case studies.

   • [claimed-docs] “Analyze requirements, implement features, fix bugs, write tests, and improve documentation based on your prompts.”
   • [claimed-docs] “Update statuses, add comments, link PRs to issues, and create new tasks. Support for Jira, Linear, Clickup and Monday.com.”

   proof ↗⚑ flag

   Jules—none0/10

   Jules's documented workflow is a coding-agent pipeline (clone repo, generate a plan, produce diffs/PRs) driven by developer-specified tasks or GitHub issues, not a mechanism for ingesting unstructured user-feedback and converting it into a structured task with proposed scope for a PM. No evidence shows feedback intake, requirement structuring, or scope proposal features aimed at product managers.

   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Use the "jules" label in an issue to assign a task directly in GitHub.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”

   proof ↗⚑ flag

3. developerAttach a marked-up screenshot or mockup to a task so the agent implements the correct visual change

   weight 2 · round drawn
   Codegen—none0/10

   No evidence in the pack mentions attaching screenshots, images, or marked-up mockups to a task/prompt; Codegen's docs cover text prompts, Slack/Jira/Linear integrations, CLI/SDK, and PR review but nothing about visual/image input for implementing UI changes. Figma is mentioned only as a connectable integration, not as an image-annotation workflow.

   • [claimed-docs] “Connect with Slack, Linear, Figma, databases, and extend capabilities with custom MCP tools.”

   proof ↗⚑ flag

   Jules—none0/10

   No evidence Jules supports attaching images, screenshots, or mockups to a task — task submission is described only via text prompts, GitHub issue labels, or API messages. Missing for 10: any mention of image/screenshot attachment, multimodal input handling, or visual mockup interpretation.

   ⚑ flag

Plan approval

1. developerReview and approve an agent's implementation plan before any code changes are made

   weight 3 · round to Jules
   Codegen—none0/10

   No evidence in the pack of a plan-preview or approval step prior to code changes; agents appear to run prompts directly to implementation (create branches, commit code, open PRs) without a documented plan-review gate. missing for 10: any mention of a plan/preview mode, approval gate, or dry-run step before code execution.

   • [claimed-docs] “Analyze requirements, implement features, fix bugs, write tests, and improve documentation based on your prompts.”
   • [claimed-docs] “Review PRs, suggest changes, comment on issues, create branches, commit code, and manage repositories.”
   • [claimed-docs] “agent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")”

   proof ↗⚑ flag

   Jules✓fullclaimed8/10

   Jules docs explicitly state the agent generates a plan upon task submission that users can review and approve before any code changes are made, directly matching the story. No independent/hands-on evidence specifically confirms or contradicts this plan-approval step, though other workflow claims (diffs, PRs) are corroborated by community mentions. Missing for 10: independent/hands-on confirmation of the plan-review step specifically, and more detail on what the plan interface looks like.

   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”

   proof ↗⚑ flag

2. engineering-leadApprove a task's scope and contract before an agent is allowed to modify the repository

   weight 2 · round to Jules
   Codegen—none0/10

   Evidence shows Codegen agents can be given prompts and run autonomously with PR review/creation, but there is no mention of any scope/contract approval gate or engineering-lead sign-off step required before an agent is allowed to modify a repository.

   ⚑ flag

   Jules~partialclaimed6/10

   Jules generates a plan that the user can review and approve before any code changes are made, giving an engineering-lead a gate before modification, and it also produces diffs/PRs requiring approval before merge. However, this is a general 'plan approval' UX rather than a formal scope/contract sign-off workflow tailored for engineering-lead governance (e.g., no evidence of role-based approval gates, policy enforcement, or blocking unauthorized starts). missing for 10: explicit lead/role-based approval gating before task execution starts, evidence of enforceable contract/scope definitions beyond a plan preview, independent confirmation that the plan-approval step reliably blocks modification.

   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”

   proof ↗⚑ flag

Ticket driven tasking

1. developerAssign a coding task to an agent directly from an existing issue or ticket

   weight 3 · round to Jules
   Codegen✓fullclaimed7/10

   Docs show native integrations with Jira, Linear, ClickUp and Monday.com allowing the agent to update issue statuses, add comments, link PRs and create tasks, and specifically state connecting a Linear workspace 'enables agent interactions with your issues,' which directly supports assigning work from a ticket. Missing for 10: a concrete end-to-end example of assigning a task from an issue/ticket, and independent/hands-on corroboration beyond vendor docs.

   • [claimed-docs] “Update statuses, add comments, link PRs to issues, and create new tasks. Support for Jira, Linear, Clickup and Monday.com.”
   • [claimed-docs] “Connect your Linear workspace to enable agent interactions with your issues.”
   • [claimed-docs] “Install the GitHub App to grant the agent access to your repositories. No complex setup required.”

   proof ↗⚑ flag

   Jules✓fullcommunity8/10

   Jules explicitly supports assigning tasks directly from GitHub issues via a 'jules' label, and API/community evidence corroborates task dispatch and PR creation workflows. Missing for 10: evidence of ticket-tracker integrations beyond GitHub (e.g., Jira/Linear) and independent hands-on confirmation of the label-to-task flow itself.

   • [claimed-docs] “Use the "jules" label in an issue to assign a task directly in GitHub.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [community] “its way better than the github thing in my experience it produces usable PRs”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round drawn
   Codegen~partialprobed5/10

   Codegen offers a Python SDK, CLI, and API to create/run agents, pull work, and monitor executions, showing solid programmatic control (codegen-docs-11,15,16,17,18). However, there's no evidence of a complete OpenAPI/API surface matching every UI capability (e.g., Slack/Jira/Linear integration management, PR review workflows) — the OpenAPI probe returned 404s, suggesting no public full API spec. missing for 10: OpenAPI reference confirming full UI parity, evidence that integration configs and review workflows are API-accessible, independent confirmation of API completeness.

   • [claimed-docs] “Programmatically interact with agents using the Python SDK for advanced automation.”
   • [claimed-docs] “agent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")”
   • [claimed-docs] “Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic cont…”
   • [claimed-docs] “programmatically retrieve detailed agent traces for analysis and debugging”
   • [claimed-docs] “Trigger agents from CI/CD pipelines when builds fail”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.codegen.com/openapi.json, https://docs.codegen.com/swagger.json, https://docs.codegen.c…”

   proof ↗⚑ flag

   Jules~partialprobed5/10

   Jules exposes a public API for creating tasks, sending messages, and enabling automation (PR creation) — core building blocks of the UI workflow — and a community user confirms dispatching tasks via the API from an external MCP client, showing real usage beyond the UI. However, there's no documented API coverage for plan approval, diff browsing/approval, or settings/API-key management, and no OpenAPI spec is discoverable (probes 404), so full UI/API parity is unconfirmed. Missing for 10: API endpoints for plan review/approval, diff/PR review parity, settings management, and a public API spec confirming full feature coverage.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”
   • [claimed-docs] “To send a message to the agent:”
   • [community] “Was able to build a personal MCP server that connects to the Jules API, letting me dispatch tasks to Jules, from Copilot Chat in VS Code.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://jules.google/openapi.json, https://jules.google/swagger.json, https://jules.google/api/opena…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round drawn
   Codegen—none0/10

   No evidence of a bulk data-export feature, open-format export of agent histories/traces, or account-portability tooling; only partial building blocks exist (pulling agent branches via CLI, retrieving traces via API) but nothing documenting a full data export/exit path.

   • [claimed-docs] “Pull agent work - Download branches and code changes created by agents directly to your local environment”
   • [claimed-docs] “programmatically retrieve detailed agent traces for analysis and debugging”

   proof ↗⚑ flag

   Jules—none0/10

   No evidence of any data export feature, open-format export, or account/data portability mechanism for Jules; the product works via GitHub repos/PRs but nothing indicates users can export Jules-specific data (task history, configs, etc.) and leave. missing for 10: any documentation of data export, open format support, or account portability/deletion workflow.

   ⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round drawn
   Codegen—none0/10

   No evidence of an open-license source repository or public codebase for Codegen; all evidence relates to product features, docs, CLI, and SDK usage, not source availability or licensing.

   ⚑ flag

   Jules—none0/10

   Jules is a closed, proprietary Google product (cloud VM, API keys, usage limits); no evidence anywhere of source code being published or licensed openly, and probes for docs/openapi artifacts return 404s, further suggesting no open publishing.

   • [claimed-docs] “Jules needs access to your repositories in order to work.”
   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [probe] “PROBE llms.txt: HTTP 404 at https://jules.google/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://jules.google/docs.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://jules.google/openapi.json, https://jules.google/swagger.json, https://jules.google/api/opena…”

   proof ↗⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round drawn
   Codegen—none0/10

   Codegen is presented as a cloud-hosted SaaS agent platform (GitHub App, Slack App, cloud CLI/SDK, hosted API); no evidence of any self-hosted deployment option, on-prem package, or open-source core repository. Absence of evidence for this applicable capability yields 'none'.

   ⚑ flag

   Jules—none0/10

   Jules is explicitly a cloud-based VM service (Google-hosted) with no evidence of any self-hosting option; community even notes cloud-only design as a drawback compared to local tools like Claude Code.

   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [community] “Jules runs in cloud-based VMs instead of on my local machine, making it much less useful than Claude Code. My projects have bespoke build sc…”
   • [community] “It's a shame Google picked the wrong system design for Jules. Claude Code's system design is clearly superior at this point.”

   proof ↗⚑ flag

Enterprise licensing

1. engineering-leadLicense an enterprise deployment with SSO and commercial support for organization-wide rollout

   weight 2 · round drawn
   Codegen—none0/10

   No evidence in the pack mentions SSO, enterprise licensing, commercial support tiers, or org-wide rollout plans; only feature-level docs and API/CLI references are provided. Missing for 10: SSO/SAML support, enterprise plan/pricing tier documentation, commercial support SLA, org-wide admin/rollout tooling.

   ⚑ flag

   Jules—none0/10

   No evidence of enterprise licensing, SSO, or commercial support offerings; docs mention only individual API keys and usage-tier limits, and community discussion references a free beta plan, not enterprise deployment. missing for 10: SSO integration, enterprise licensing/contract terms, commercial support SLAs, org-wide admin/rollout tooling.

   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [claimed-docs] “Power users & agent-heavy workflows ... 300 ... 60”
   • [community] “Is Jules free of charge? Yes, for now, Jules is free of charge. Jules is in beta and available without payment while we learn from usage.”

   proof ↗⚑ flag

Model flexibility

1. engineering-leadBring my own LLM or API key so agents run on the model of my choice

   weight 2 · round drawn
   Codegen—none0/10

   No evidence pack item mentions BYO-LLM, custom API key configuration, or model selection; Codegen's docs focus on its own agents, integrations, and SDK/CLI without any option to swap the underlying model provider.

   ⚑ flag

   Jules—none0/10

   No evidence that Jules supports bringing a custom LLM or third-party API key; it is tied to Google's own models (Gemini), with documentation only mentioning Jules's own API key for accessing Jules itself, not for configuring underlying model providers. missing for 10: any mention of BYO-LLM support, model selection options, or third-party API key configuration.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [claimed-docs] “Priority access to the latest models, starting with Gemini 3 Pro”

   proof ↗⚑ flag

Usage quotas

1. engineering-leadSee and manage plan-based daily task and concurrency limits for agent workflows

   weight 2 · round to Jules
   Codegen—none0/10

   No evidence pack item mentions plan-based daily task limits, concurrency limits, or any admin dashboard/controls for engineering leads to monitor or manage usage quotas; documentation covers agent capabilities, integrations, CLI, and SDK but nothing on pricing tiers or limit management.

   ⚑ flag

   Jules~partialcommunity4/10

   Jules docs reference a usage-limits page with plan-tiered daily task numbers (e.g., 300/60) and community confirms daily task limits exist and change between releases (60→15 on free plan), showing plan-based limits are real and documented. However there's no evidence of an engineering-lead-facing dashboard or admin console to monitor team usage, set/adjust concurrency, or manage limits across a team — only a static limits reference page and anecdotal user experience of hitting caps. Missing for 10: team/org usage dashboard, per-user concurrency visibility, ability to configure or request limit changes, and any admin/lead-specific management UI.

   • [claimed-docs] “Power users & agent-heavy workflows ... 300 ... 60”
   • [community] “The daily task limit went down from 60 to 15 (on the free plan) with this release. Personally I wasn't close to exhausting the limit because…”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round drawn
   Codegen—none0/10

   No evidence in the pack mentions data residency, region selection, or storage location options for Codegen; the docs cover integrations, CLI, SDK, and agent workflows but nothing about data storage location controls.

   ⚑ flag

   Jules—none0/10

   No evidence in the pack mentions data residency, regional storage options, or any control over where Jules stores/processes data; docs only describe VM execution and GitHub integration without residency settings.

   ⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   Codegen—none0/10

   No evidence in the pack addresses data-training opt-out, privacy controls, or policies about AI model training usage; all citations concern feature functionality (agents, CLI, SDK, integrations) rather than privacy posture.

   ⚑ flag

   Jules—none0/10

   No evidence in the pack addresses data usage/training opt-out policies, privacy controls, or any statement about whether user code/data is used to train AI models; the docs focus on functionality (VM execution, PR creation, API) rather than data governance.

   ⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round drawn
   Codegen—none0/10

   No evidence pack items discuss data retention policies, deletion controls, or privacy/compliance settings; the docs cover functionality (agents, integrations, CLI, SDK) but nothing about controlling or deleting stored data.

   ⚑ flag

   Jules—none0/10

   No evidence pack item mentions data retention policies, data deletion controls, or privacy settings for repositories/code processed by Jules; docs cover workflow, VM execution, and API usage but not retention/deletion controls.

   ⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Codegen—none0/10

   No evidence pack item mentions a telemetry opt-out setting or privacy controls; in fact docs-12 explicitly notes the CLI runs Claude Code 'with full telemetry and monitoring,' with no mention of a way to disable it.

   • [claimed-docs] “Use it to view agents, pull their work, create new agents, and run Claude Code with full telemetry and monitoring.”

   proof ↗⚑ flag

   Jules—none0/10

   No evidence in the pack mentions telemetry, usage tracking, or an opt-out mechanism; the docs focus on repo access, VM execution, PR workflow, and API usage without addressing privacy/telemetry controls.

   ⚑ flag

Chat integration

1. developerTag an agent in a chat thread to discuss and delegate a bug or task

   weight 2 · round to Codegen
   Codegen~partialclaimed6/10

   Codegen supports Slack-based chat interactions where users can message/tag the agent to report progress, ask for clarification, and delegate tasks (codegen-docs-2, codegen-docs-9), and Linear/Jira integration allows commenting and task creation to delegate bugs (codegen-docs-3, codegen-docs-10). However, evidence doesn't explicitly confirm an '@mention' tagging mechanism within an existing chat thread to delegate a specific bug — it's inferred from general Slack/Linear interaction docs rather than a documented tagging workflow. Missing for 10: explicit documentation of @-mention/tagging syntax in threads, confirmation of thread-level context retention when delegating, and independent/hands-on verification of this specific workflow.

   • [claimed-docs] “Send notifications, ask for clarification, report progress, and interact directly with your team in Slack channels.”
   • [claimed-docs] “Add the Codegen Slack App to communicate with the agent directly in your workspace.”
   • [claimed-docs] “Update statuses, add comments, link PRs to issues, and create new tasks. Support for Jira, Linear, Clickup and Monday.com.”
   • [claimed-docs] “Connect your Linear workspace to enable agent interactions with your issues.”

   proof ↗⚑ flag

   Jules~partialcommunity4/10

   Jules supports delegating tasks via a GitHub issue label ('jules') and via API messages to the agent, which is a form of task delegation but not the classic 'tag an agent in a chat thread to discuss' interaction pattern; one community user built a custom MCP server to dispatch Jules tasks from VS Code Copilot Chat, showing it's possible but not a native feature. missing for 10: native chat-thread tagging/mention UI, evidence of back-and-forth discussion in a thread before delegation, first-party support for Slack/Teams-style @mentions.

   • [claimed-docs] “Use the "jules" label in an issue to assign a task directly in GitHub.”
   • [claimed-docs] “To send a message to the agent:”
   • [community] “Was able to build a personal MCP server that connects to the Jules API, letting me dispatch tasks to Jules, from Copilot Chat in VS Code.”

   proof ↗⚑ flag

Knowledge context

1. developerAdd a context file describing my codebase conventions so agents generate more relevant plans and code

   weight 3 · round to Jules
   Codegen—none0/10

   The evidence pack covers Codegen's integrations, CLI, SDK, and PR/issue automation, but nowhere mentions a repo-level context or rules file (e.g., conventions file) that developers can add to steer agent plans/code generation. missing for 10: any mention of a context/config file for codebase conventions, documentation on custom instructions, or repo-level rules ingestion.

   ⚑ flag

   Jules✓fullclaimed7/10

   Jules automatically reads an AGENTS.md file in the repo root to guide its plans and code generation, directly matching the story of adding a context file describing codebase conventions. Missing for 10: no independent/hands-on confirmation of how well conventions in AGENTS.md actually improve plan/code relevance, and no detail on supported format/scope beyond the single doc mention.

   • [claimed-docs] “Jules now automatically looks for a file named AGENTS.md in the root of your repository.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”

   proof ↗⚑ flag

2. developerQuery generated documentation for any public or private repository

   weight 1 · round drawn
   Codegen—none0/10

   Evidence shows Codegen can 'improve documentation' as part of coding tasks, but there is no evidence of a feature to generate and then query documentation for a repository (public or private) in a Q&A style. No docs, API, or CLI reference describe a documentation-query capability.

   ⚑ flag

   Jules—none0/10

   Evidence describes Jules as a task-based coding agent that clones repos, edits code, and creates PRs, but there is no mention of generating or letting users query documentation for a repository (public or private). missing for 10: any docs-generation feature, a documentation query/search interface, evidence of indexing repo docs for Q&A.

   • [claimed-docs] “Jules needs access to your repositories in order to work.”
   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”

   proof ↗⚑ flag

Project management integration

1. product-managerConnect issue trackers like Jira, Linear, ClickUp, or Monday.com so agents can manage tickets directly

   weight 3 · round to Codegen
   Codegen~partialclaimed6/10

   Docs explicitly claim support for Jira, Linear, ClickUp, and Monday.com (update statuses, add comments, link PRs, create tasks), but detailed setup instructions are only shown for Linear (codegen-docs-10); Jira, ClickUp, and Monday.com lack dedicated connection/integration docs equivalent to Linear or Slack. Missing for 10: explicit setup guides or independent verification for Jira/ClickUp/Monday.com, and hands-on confirmation beyond first-party docs.

   • [claimed-docs] “Update statuses, add comments, link PRs to issues, and create new tasks. Support for Jira, Linear, Clickup and Monday.com.”
   • [claimed-docs] “Connect your Linear workspace to enable agent interactions with your issues.”
   • [claimed-docs] “Connect with Slack, Linear, Figma, databases, and extend capabilities with custom MCP tools.”

   proof ↗⚑ flag

   Jules—none0/10

   Evidence only shows Jules integrating with GitHub (repos, issues, PRs) and provides an API for custom workflows, but there is no mention of Jira, Linear, ClickUp, Monday.com, or any issue-tracker integration beyond GitHub.

   ⚑ flag

Version control integration

1. developerConnect a GitHub repository so an agent can access the code and open pull requests against it

   weight 3 · round to Jules
   Codegen✓fullclaimed8/10

   Codegen provides a GitHub App install for repo access with no complex setup, and docs describe the agent creating branches, committing code, reviewing/opening PRs, and managing repositories. missing for 10: independent/hands-on confirmation beyond vendor docs, and explicit detail on multi-repo/org-level permission scoping.

   • [claimed-docs] “Install the GitHub App to grant the agent access to your repositories. No complex setup required.”
   • [claimed-docs] “Review PRs, suggest changes, comment on issues, create branches, commit code, and manage repositories.”
   • [claimed-docs] “Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic cont…”
   • [claimed-docs] “Trigger agents from CI/CD pipelines when builds fail”

   proof ↗⚑ flag

   Jules✓fullcommunity9/10

   Jules is built around GitHub repo access: docs describe cloning repos, GitHub issue label assignment, generating diffs/plans, and creating PRs that can be merged on GitHub, corroborated by community reports of receiving usable PRs from Jules-driven tasks. Missing for 10: independent verification of the full connect-repo setup flow and edge-case reliability across repo types.

   • [claimed-docs] “Jules needs access to your repositories in order to work.”
   • [claimed-docs] “Use the "jules" label in an issue to assign a task directly in GitHub.”
   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [community] “its way better than the github thing in my experience it produces usable PRs”
   • [community] “I've moved infrastructure to Railway and taught the customer to use Jules. They make their own PRs now, Railway spins up an environment with…”

   proof ↗⚑ flag

2. developerGrant an agent access to my repositories with a one-click install, without complex setup

   weight 2 · round to Codegen
   Codegen✓fullclaimed8/10

   Docs explicitly state installing the GitHub App grants repo access with 'no complex setup required', directly matching the story, and this is corroborated by broader repo-integration capabilities (PR review, branch/commit management). Missing for 10: independent/hands-on confirmation of the one-click flow beyond first-party docs.

   • [claimed-docs] “Install the GitHub App to grant the agent access to your repositories. No complex setup required.”
   • [claimed-docs] “Review PRs, suggest changes, comment on issues, create branches, commit code, and manage repositories.”

   proof ↗⚑ flag

   Jules~partialcommunity6/10

   Jules integrates with GitHub (label-based task assignment, PR creation) and needs repo access to work, implying a connect/authorize flow, and community evidence confirms GitHub integration works well for many users. However, there's no explicit documentation of a literal 'one-click install' onboarding flow or app installation process described in the evidence pack. missing for 10: explicit one-click GitHub App install/authorization flow documentation, independent confirmation of setup simplicity.

   • [claimed-docs] “Jules needs access to your repositories in order to work.”
   • [claimed-docs] “Use the "jules" label in an issue to assign a task directly in GitHub.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [community] “I used Jules three times today, very impressive! It also handles coding-adjacent work. Good github integrations.”
   • [community] “its way better than the github thing in my experience it produces usable PRs”

   proof ↗⚑ flag

Ci remediation

1. engineering-leadHave failed CI workflows automatically diagnosed and fixed with a proposed pull request

   weight 3 · round to Codegen
   Codegen~partialclaimed6/10

   Docs explicitly mention triggering agents from CI/CD pipelines when builds fail and agents that fix bugs/write code and open PRs, directly supporting automated diagnosis-and-fix-PR workflows for failed CI. However, there is no dedicated end-to-end CI-failure-diagnosis feature description, no example of the proposed PR format for a failing build, and no independent/hands-on confirmation of this specific workflow. missing for 10: dedicated CI-failure-diagnosis workflow documentation, concrete example of a CI-triggered fix PR, independent verification of this specific capability.

   • [claimed-docs] “Trigger agents from CI/CD pipelines when builds fail”
   • [claimed-docs] “Analyze requirements, implement features, fix bugs, write tests, and improve documentation based on your prompts.”
   • [claimed-docs] “Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic cont…”

   proof ↗⚑ flag

   Jules~partialclaimed3/10

   Jules offers building blocks—an API for custom automation that can 'automate tasks like bug fixing' and auto-create PRs via automationMode—but there is no direct evidence of a native integration that detects a failed CI workflow and automatically diagnoses/fixes it with a PR; this would require custom API wiring, not an out-of-box feature. Missing for 10: documented CI-failure trigger/integration, evidence of automatic diagnosis of CI logs, and case studies of this specific end-to-end flow.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [claimed-docs] “Use the "jules" label in an issue to assign a task directly in GitHub.”

   proof ↗⚑ flag

2. developerTrigger an agent from CI/CD pipelines to fix a broken build or failing test

   weight 2 · round to Codegen
   Codegen✓fullclaimed8/10

   Docs explicitly state agents can be 'Triggered from CI/CD pipelines when builds fail' and support fixing bugs/tests via API/SDK with programmatic control, which directly matches the story. Missing for 10: independent/hands-on confirmation of a real CI/CD integration example (e.g., GitHub Actions snippet) and details on how failing test context is passed to the agent.

   • [claimed-docs] “Trigger agents from CI/CD pipelines when builds fail”
   • [claimed-docs] “Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic cont…”
   • [claimed-docs] “Programmatically interact with agents using the Python SDK for advanced automation.”
   • [claimed-docs] “agent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")”

   proof ↗⚑ flag

   Jules~partialcommunity5/10

   Jules exposes a documented API for creating custom workflows and automating tasks like bug fixing (jules-docs-9, jules-docs-10, jules-docs-12), which could be wired into a CI/CD pipeline to trigger a fix, and community evidence shows users building custom integrations to dispatch Jules tasks programmatically (jules-comm-19). However, there is no first-party CI/CD-specific integration (e.g. GitHub Actions step, build-failure webhook) or documented example of triggering Jules to fix a broken build/failing test directly from a pipeline. missing for 10: explicit CI/CD pipeline integration/example, evidence of triggering on build/test failure events, hands-on confirmation of this specific workflow succeeding.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”
   • [claimed-docs] “To send a message to the agent:”
   • [community] “Was able to build a personal MCP server that connects to the Jules API, letting me dispatch tasks to Jules, from Copilot Chat in VS Code.”

   proof ↗⚑ flag

Diff review

1. developerConfigure an agent to automatically open a pull request when its task completes

   weight 2 · round to Jules
   Codegen✓fullclaimed7/10

   Codegen's agent creates branches, commits code, and manages repositories via GitHub App integration, and can be run programmatically via SDK/CLI/API to complete tasks; PR creation is a core documented capability of the agent workflow. Missing for 10: explicit first-party documentation/example showing a config flag or setting to 'auto-open PR on task completion' and independent/hands-on confirmation of this exact behavior.

   • [claimed-docs] “Review PRs, suggest changes, comment on issues, create branches, commit code, and manage repositories.”
   • [claimed-docs] “Install the GitHub App to grant the agent access to your repositories. No complex setup required.”
   • [claimed-docs] “Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic cont…”
   • [claimed-docs] “Trigger agents from CI/CD pipelines when builds fail”

   proof ↗⚑ flag

   Jules✓fullcommunity8/10

   Jules natively creates a PR when a task completes (jules-docs-8), and the API exposes an `automationMode` field to configure automatic PR creation, defaulting to off unless explicitly configured (jules-docs-11), directly matching the story of configuring automatic PR-on-completion. Community reports corroborate that Jules produces usable PRs and users regularly review PRs it opens (jules-comm-6, jules-comm-12, jules-comm-16). missing for 10: independent hands-on verification of the automationMode toggle specifically, and more detail on configuration options/edge cases (e.g., partial completions, failed tasks) affecting whether a PR is always opened.

   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”
   • [community] “its way better than the github thing in my experience it produces usable PRs”
   • [community] “I've been actually kind-of enjoying using Jules as a way of 'coding' my side project using my phone. By the time I get home in the evening I…”
   • [community] “I've moved infrastructure to Railway and taught the customer to use Jules. They make their own PRs now, Railway spins up an environment with…”

   proof ↗⚑ flag

2. developerReview a diff of an agent's changes and approve it before it becomes a pull request

   weight 3 · round to Jules
   Codegen~partialclaimed5/10

   Codegen agents create branches, commit code, and open PRs, and the agent itself can review PRs and post comments, but the evidence pack does not show a dedicated human-in-the-loop diff review/approval gate before a PR is created — the CLI lets you 'pull agent work' locally which could support manual review, but no explicit approval workflow is documented. missing for 10: explicit pre-PR diff approval UI/workflow, evidence of a human approve/reject gate, independent confirmation that PR creation is blocked pending review.

   • [claimed-docs] “Review PRs, suggest changes, comment on issues, create branches, commit code, and manage repositories.”
   • [claimed-docs] “Use it to view agents, pull their work, create new agents, and run Claude Code with full telemetry and monitoring.”
   • [claimed-docs] “Pull agent work - Download branches and code changes created by agents directly to your local environment”
   • [claimed-docs] “Reads the PR diff Summarizes changes across files Runs security and secrets scan Checks test coverage Posts a review comment on failure”

   proof ↗⚑ flag

   Jules✓fullcommunity9/10

   Jules explicitly provides a diff of changes for review and approval before it creates/publishes a PR, with plan approval and diff approval steps documented; community mentions confirm PRs are generated for review. Missing for 10: independent hands-on confirmation of the diff-approval UI flow itself (comments focus on PR quality/output rather than the diff-review step specifically).

   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [community] “I've been actually kind-of enjoying using Jules as a way of 'coding' my side project using my phone. By the time I get home in the evening I…”

   proof ↗⚑ flag

Pr review automation

1. ai-native userHave incoming issues automatically triaged with severity suggested and routed to the right owner

   weight 2 · round drawn
   Codegen—none0/10

   Evidence shows Codegen integrates with issue trackers (Jira, Linear, etc.) to update statuses, add comments, and create tasks, but there is no mention of automatic severity assessment or routing incoming issues to the correct owner. Missing for 10: automatic severity classification, owner/routing logic, any triage workflow evidence.

   • [claimed-docs] “Update statuses, add comments, link PRs to issues, and create new tasks. Support for Jira, Linear, Clickup and Monday.com.”
   • [claimed-docs] “Connect your Linear workspace to enable agent interactions with your issues.”

   proof ↗⚑ flag

   Jules—none0/10

   Jules is a coding agent that executes tasks assigned via label or API, but there is no evidence of automatic issue triage, severity classification, or routing to owners — the closest feature is manual label-based task assignment, not automated triage. Missing for 10: any mention of severity scoring, triage logic, or owner-routing automation.

   • [claimed-docs] “Use the "jules" label in an issue to assign a task directly in GitHub.”
   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”

   proof ↗⚑ flag

2. engineering-leadHave every pull request automatically reviewed with AI-generated inline comments

   weight 3 · round to Codegen
   Codegen✓fullclaimed7/10

   Docs explicitly describe automated PR review functionality: reading the diff, summarizing changes, running security/secrets scans, checking test coverage, and posting a review comment, plus general 'Review PRs, suggest changes, comment on issues' capability tied to GitHub App integration for repo access. This matches the engineering-lead story of automatic AI-reviewed PRs with comments. Missing for 10: explicit confirmation that comments are inline (per-line) rather than a single summary comment, and independent/hands-on evidence corroborating this in production use.

   • [claimed-docs] “Review PRs, suggest changes, comment on issues, create branches, commit code, and manage repositories.”
   • [claimed-docs] “Reads the PR diff Summarizes changes across files Runs security and secrets scan Checks test coverage Posts a review comment on failure”
   • [claimed-docs] “Install the GitHub App to grant the agent access to your repositories. No complex setup required.”
   • [claimed-docs] “Trigger agents from CI/CD pipelines when builds fail”

   proof ↗⚑ flag

   Jules—none0/10

   Jules is documented as a task-execution agent that clones repos, makes code changes, and creates its own PRs for approval — not as a bot that automatically reviews incoming pull requests with inline comments. The only tangential mention is jules-docs-9's passing reference to 'automate tasks like ... code reviews' via API, but there is no documentation of an automatic inline-comment review gate applied to every PR.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”

   proof ↗⚑ flag

Readiness checks

1. engineering-leadAutomatically fix failing agent-readiness criteria in my repository

   weight 1 · round to Codegen
   Codegen~partialclaimed5/10

   Codegen agents can fix bugs, write tests, and run CI/CD-triggered fixes (codegen-docs-1, codegen-docs-18, codegen-docs-20), which could be applied to remediate failing agent-readiness checks, but there is no explicit documentation of an 'agent-readiness criteria' concept, scoring rubric, or automated fix workflow tied specifically to such criteria. missing for 10: explicit agent-readiness criteria/checklist, automated detection-to-fix pipeline for those criteria, evidence of hands-on success fixing readiness gates.

   • [claimed-docs] “Analyze requirements, implement features, fix bugs, write tests, and improve documentation based on your prompts.”
   • [claimed-docs] “Trigger agents from CI/CD pipelines when builds fail”
   • [claimed-docs] “Reads the PR diff Summarizes changes across files Runs security and secrets scan Checks test coverage Posts a review comment on failure”

   proof ↗⚑ flag

   Jules—none0/10

   Jules can be assigned generic bug-fix/coding tasks and reads an AGENTS.md file if present, but there is no evidence of it detecting or automatically remediating 'agent-readiness' quality-gate criteria (e.g., missing/invalid AGENTS.md, agent-compatibility checks) as a review gate — it only consumes such files, it doesn't audit or fix them as a compliance gate.

   • [claimed-docs] “Jules now automatically looks for a file named AGENTS.md in the root of your repository.”
   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [community] “I gave Jules a try on a new, very unorganized Python project. It failed the first time, I gave it the error message and it fixed it. I was s…”

   proof ↗⚑ flag

2. engineering-leadRun a readiness report that evaluates how ready my repository is for autonomous agents

   weight 2 · round drawn
   Codegen—none0/10

   The evidence pack shows PR review, security scanning, and CI/CD triggers, but there is no mention of a dedicated 'readiness report' feature that evaluates overall repository readiness for autonomous agents. Missing for 10: any documentation of a repo-wide readiness/scorecard report, criteria for agent-readiness, or a command/API that produces such an assessment.

   • [claimed-docs] “Reads the PR diff Summarizes changes across files Runs security and secrets scan Checks test coverage Posts a review comment on failure”
   • [claimed-docs] “Review PRs, suggest changes, comment on issues, create branches, commit code, and manage repositories.”
   • [claimed-docs] “Safely run code, install dependencies, and test changes in robust isolated environments”

   proof ↗⚑ flag

   Jules—none0/10

   No evidence of a readiness report, scorecard, or repository-assessment feature for autonomous agent suitability; evidence only covers task execution, PR creation, and API usage. Missing for 10: any readiness-scoring feature, repo audit/checklist, or report output evaluating agent-readiness.

   ⚑ flag

Security remediation

1. engineering-leadHave security alerts automatically validated and remediated with an opened pull request

   weight 2 · round to Codegen
   Codegen~partialclaimed5/10

   Codegen documents a security/secrets scan that posts a review comment on failure (codegen-docs-20) and general bug-fixing/PR-creation capabilities plus CI/CD-triggered agents (codegen-docs-1, codegen-docs-4, codegen-docs-18), suggesting it could act on a flagged issue and open a fix PR. However, there is no explicit evidence of ingesting external security alerts (e.g., Dependabot/CodeQL/Snyk), validating them as true positives, or automatically opening a remediation PR end-to-end — the documented scan flow stops at commenting, not remediating. missing for 10: explicit security-alert ingestion/validation workflow, evidence of an opened remediation PR tied to a specific alert, independent confirmation of this flow working in practice.

   • [claimed-docs] “Reads the PR diff Summarizes changes across files Runs security and secrets scan Checks test coverage Posts a review comment on failure”
   • [claimed-docs] “Trigger agents from CI/CD pipelines when builds fail”
   • [claimed-docs] “Review PRs, suggest changes, comment on issues, create branches, commit code, and manage repositories.”
   • [claimed-docs] “Analyze requirements, implement features, fix bugs, write tests, and improve documentation based on your prompts.”

   proof ↗⚑ flag

   Jules—none0/10

   Jules is a general coding agent that can be assigned tasks (via GitHub label or API) and opens PRs with diffs for review, but there is no evidence of any security-alert scanning, vulnerability detection/validation, or a workflow that automatically triggers remediation from a security alert (e.g., Dependabot/CVE integration). The evidence pack shows generic task-to-PR flow, not a security-alert-specific pipeline.

   • [claimed-docs] “Jules needs access to your repositories in order to work.”
   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”

   proof ↗⚑ flag

Concurrent execution

1. engineering-leadRun many agent tasks concurrently to scale delivery throughput

   weight 3 · round drawn
   Codegen~partialclaimed6/10

   Codegen's SDK/API let you programmatically create and run multiple agents (agent.run, API 'full programmatic control over execution and monitoring') and CLI can view multiple running agents, implying support for parallel task execution, and CI/CD triggers suggest scaling automation into pipelines. However, there is no explicit documentation of concurrency limits, throughput guarantees, or dashboards for managing many simultaneous tasks at scale. Missing for 10: explicit concurrency/parallelism claims, scaling limits or throughput benchmarks, and independent evidence of running many agents simultaneously in practice.

   • [claimed-docs] “agent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")”
   • [claimed-docs] “Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic cont…”
   • [claimed-docs] “Use it to view agents, pull their work, create new agents, and run Claude Code with full telemetry and monitoring.”
   • [claimed-docs] “Trigger agents from CI/CD pipelines when builds fail”

   proof ↗⚑ flag

   Jules~partialcommunity6/10

   Jules architecture (cloud VMs per task, API for creating tasks, GitHub issue labeling) supports dispatching many tasks in parallel, and usage-limit docs explicitly reference 'agent-heavy workflows' with numeric caps (e.g. 300/60), and community reports (jules-comm-11, jules-comm-16) describe handling large task volumes successfully. However, other evidence shows real caveats: daily task limits were cut from 60 to 15 on the free plan (jules-comm-8), tasks can get stuck in loops with no stop button (jules-comm-14), and users report needing heavy babysitting (jules-comm-18), undercutting smooth high-throughput scaling. Missing for 10: dedicated documentation of a concurrency/queue dashboard, enterprise-tier concurrency guarantees, and independent benchmarks confirming reliable parallel execution at scale.

   • [claimed-docs] “Power users & agent-heavy workflows ... 300 ... 60”
   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [community] “I've used this tool for a few months now and have been pretty impressed by it. It handles large quantities of tasks very well and is good at…”
   • [community] “I've moved infrastructure to Railway and taught the customer to use Jules. They make their own PRs now, Railway spins up an environment with…”
   • [community] “The daily task limit went down from 60 to 15 (on the free plan) with this release. Personally I wasn't close to exhausting the limit because…”
   • [community] “Jules quickly after a few messages/prompts just gets stuck in an endless loop like Gemini-cli does. The worst part is that there is no 'STOP…”
   • [community] “Do people really find Jules useful? I find it needs babysitting much more than Cursor.”

   proof ↗⚑ flag

2. engineering-leadCreate agent sessions on behalf of other users in my organization

   weight 2 · round to Codegen
   Codegen~partialclaimed4/10

   Codegen's org-level SDK/CLI/API (org_id+token, 'manage organizations', programmatic agent creation) implies an engineering lead with org credentials could spin up agents at scale, but no docs explicitly describe creating or assigning a session to another named user in the org. Missing for 10: explicit user-impersonation or 'create on behalf of' API/UI feature, role-based permissions for delegating agent sessions, and any admin console for assigning sessions to teammates.

   • [claimed-docs] “Manage organizations - Switch between organizations and configure repositories”
   • [claimed-docs] “agent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")”
   • [claimed-docs] “Use it to view agents, pull their work, create new agents, and run Claude Code with full telemetry and monitoring.”
   • [claimed-docs] “Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic cont…”

   proof ↗⚑ flag

   Jules—none0/10

   No evidence of any organization, team, or admin/delegation features allowing an engineering-lead to create sessions on behalf of other users; Jules docs only describe individual API keys, per-user GitHub connections, and personal task creation. Missing for 10: org/team management, delegated session creation, role-based admin controls, multi-user account provisioning.

   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “Jules needs access to your repositories in order to work.”

   proof ↗⚑ flag

Deployment flexibility

1. developerUse a managed cloud offering to run agents without operating my own backend infrastructure

   weight 2 · round to Jules
   Codegen✓fullclaimed7/10

   Codegen offers cloud-run agents with isolated execution environments, API/SDK/CLI access, and no need to self-host infrastructure (codegen-docs-5, codegen-docs-11, codegen-docs-15, codegen-docs-16). This directly matches running agents without operating own backend. Missing for 10: independent/hands-on verification of the managed infrastructure's reliability at scale, and explicit pricing/SLA details for the managed cloud offering.

   • [claimed-docs] “Safely run code, install dependencies, and test changes in robust isolated environments”
   • [claimed-docs] “Programmatically interact with agents using the Python SDK for advanced automation.”
   • [claimed-docs] “agent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")”
   • [claimed-docs] “Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic cont…”
   • [claimed-docs] “programmatically retrieve detailed agent traces for analysis and debugging”
   • [claimed-docs] “Trigger agents from CI/CD pipelines when builds fail”

   proof ↗⚑ flag

   Jules✓fullcommunity8/10

   Jules is explicitly a cloud-hosted agent service: tasks run in Google-managed VMs, integrated with GitHub, with an API and usage limits/plans, requiring no self-hosted backend. Community evidence corroborates real-world usage at scale (75% success on customer PRs, teams dispatching many tasks) though some users report reliability/UI issues. Missing for 10: no independent infra/SLA details, no discoverable OpenAPI/docs.md confirming API completeness, and mixed community reliability reports.

   • [claimed-docs] “Jules needs access to your repositories in order to work.”
   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “Power users & agent-heavy workflows ... 300 ... 60”
   • [community] “I've moved infrastructure to Railway and taught the customer to use Jules. They make their own PRs now, Railway spins up an environment with…”
   • [community] “Was able to build a personal MCP server that connects to the Jules API, letting me dispatch tasks to Jules, from Copilot Chat in VS Code.”

   proof ↗⚑ flag

2. engineering-leadSelf-host agent infrastructure locally, in containers, or on my own VMs

   weight 2 · round drawn
   Codegen—none0/10

   Codegen is presented entirely as a cloud/SaaS agent service (GitHub App, Slack App, API, SDK, isolated cloud environments) with no evidence of a self-hostable runtime, container image, or on-prem/VM deployment option; the CLI logs local Claude Code sessions to Codegen's own cloud rather than hosting infrastructure locally.

   • [claimed-docs] “Safely run code, install dependencies, and test changes in robust isolated environments”
   • [claimed-docs] “Log local Claude Code instances to the cloud and provision MCP servers across your org.”
   • [claimed-docs] “Install the GitHub App to grant the agent access to your repositories. No complex setup required.”
   • [claimed-docs] “Use it to view agents, pull their work, create new agents, and run Claude Code with full telemetry and monitoring.”
   • [claimed-docs] “Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic cont…”

   proof ↗⚑ flag

   Jules—none0/10

   Jules is explicitly cloud-hosted: it runs tasks in Google's own VMs, and community feedback confirms it is not local (jules-comm-1). There is no evidence of any self-hosting, on-prem, container, or private-VM deployment option.

   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [community] “Jules runs in cloud-based VMs instead of on my local machine, making it much less useful than Claude Code. My projects have bespoke build sc…”

   proof ↗⚑ flag

Headless automation

1. developerRun an agent headlessly inside CI/CD pipelines and shell scripts

   weight 2 · round to Codegen
   Codegen~partialprobed6/10

   Codegen explicitly documents triggering agents from CI/CD pipelines (e.g., on build failures) and offers a Python SDK plus CLI for programmatic/scripted invocation, which supports headless usage in pipelines and scripts. However, there's no dedicated CI/CD integration guide, no example pipeline configs (e.g., GitHub Actions YAML), and no independent/hands-on evidence confirming real-world headless CI usage. missing for 10: concrete CI/CD pipeline examples or config snippets, exit-code/status handling for scripts, independent verification of headless operation in automated pipelines.

   • [claimed-docs] “Trigger agents from CI/CD pipelines when builds fail”
   • [claimed-docs] “Programmatically interact with agents using the Python SDK for advanced automation.”
   • [claimed-docs] “agent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")”
   • [claimed-docs] “Use it to view agents, pull their work, create new agents, and run Claude Code with full telemetry and monitoring.”
   • [probe] “official CLI documented at https://docs.codegen.com/introduction/cli”

   proof ↗⚑ flag

   Jules~partialcommunity5/10

   Jules exposes a public API (create tasks, send messages, optional automationMode) explicitly pitched for 'custom workflows' and embedding into other tools, which can be scripted/curled headlessly, and a community member built an MCP server hitting the API to dispatch tasks programmatically. However there's no documented CLI, no explicit CI/CD pipeline examples (e.g., GitHub Actions), and Jules always runs its own cloud VM rather than a lightweight headless process invokable inline in a shell script/pipeline step. missing for 10: official CLI or CI/CD pipeline integration docs (e.g. GitHub Actions step), examples of shell-script invocation, confirmation that API calls run synchronously enough for pipeline gating.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”
   • [claimed-docs] “To send a message to the agent:”
   • [community] “Was able to build a personal MCP server that connects to the Jules API, letting me dispatch tasks to Jules, from Copilot Chat in VS Code.”
   • [community] “I've moved infrastructure to Railway and taught the customer to use Jules. They make their own PRs now, Railway spins up an environment with…”

   proof ↗⚑ flag