Devin vs Jules

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to Devin
   Devin✓fullprobed7/10

   Devin explicitly supports AGENTS.md, an agent-oriented documentation file that Devin reads before starting a coding session to gain project context — directly matching the story's intent of pointing an agent at agent-oriented docs. Devin's own docs site also serves an llms.txt endpoint (verified via probe), showing familiarity with the convention, though this is Devin's own site rather than evidence Devin ingests arbitrary llms.txt files from user projects. Missing for 10: explicit documentation that Devin consumes/parses llms.txt files themselves (as opposed to AGENTS.md) and independent hands-on confirmation of this workflow.

   • [claimed-docs] “Just put an AGENTS.md file in your project root (or anywhere else). Devin will look for the file before it starts coding.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.devin.ai/llms.txt # Devin Docs - [Desktop (100 pages)](https://docs.devin.ai/_llms/en/desktop.md):…”

   proof ↗⚑ flag

   Jules—none0/10

   Probes show no llms.txt, docs.md, or machine-readable API spec exist at expected paths, and no evidence Jules can be pointed at such agent-oriented documentation formats; Jules does support AGENTS.md for repo context but that's a different mechanism than consuming llms.txt-style docs.

   • [probe] “PROBE llms.txt: HTTP 404 at https://jules.google/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://jules.google/docs.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://jules.google/openapi.json, https://jules.google/swagger.json, https://jules.google/api/opena…”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to Devin
   Devin✓fullclaimed8/10

   Devin exposes a full REST API for session creation and automation (devin-docs-8, devin-docs-9), supports scheduling recurring/one-time headless runs via MCP (devin-docs-17), auto-triggers on CI failures and code review comments via Devin Review/Auto-Fix (devin-docs-20), and supports parallel/headless session orchestration (devin-docs-18). This covers CI/automation use cases well beyond interactive use. Missing for 10: no independent/hands-on CI-integration case study, and no dedicated 'CI runner' or GitHub Actions-style first-class integration doc beyond the review/auto-fix hook.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “Supports cron expressions for recurring schedules, one-time scheduling, notification preferences, and agent selection”
   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures”
   • [claimed-docs] “Run multiple Devins in parallel: Carve out independent tasks and run them simultaneously.”
   • [claimed-docs] “Extend Devin's reach with MCP integrations: Connect Devin to Datadog, Sentry, databases, Figma, Notion, Stripe, and hundreds of other tool”

   proof ↗⚑ flag

   Jules✓fullcommunity7/10

   Jules provides a public API with API keys for creating custom workflows, sending tasks programmatically, and an optional automationMode to auto-create PRs, plus a community example of building an MCP server to dispatch tasks from another tool — this directly supports headless/CI-style automation. Missing for 10: no official CI/CD integration examples (e.g. GitHub Actions), no documented webhook/polling pattern for task completion, and no independent verification of reliability at scale in automated pipelines.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”
   • [claimed-docs] “To send a message to the agent:”
   • [community] “Was able to build a personal MCP server that connects to the Jules API, letting me dispatch tasks to Jules, from Copilot Chat in VS Code.”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round to Devin
   Devin✓fullprobed9/10

   Devin CLI explicitly supports connecting external MCP servers, exposing their tools as slash commands with permission controls, and docs list example integrations (Datadog, Sentry, databases, Figma, Notion, Stripe, etc.). Missing for 10: independent/hands-on community verification of MCP tool usage in practice.

   • [claimed-docs] “MCP (Model Context Protocol) lets you connect external tool servers to Devin CLI, giving the agent access to APIs, databases, issue trackers…”
   • [claimed-docs] “MCP tools are subject to the same permission system as built-in tools. You can control access at multiple levels”
   • [claimed-docs] “Devin CLI exposes each one as a slash command”
   • [claimed-docs] “Extend Devin's reach with MCP integrations: Connect Devin to Datadog, Sentry, databases, Figma, Notion, Stripe, and hundreds of other tool”
   • [probe] “official MCP server documented at https://docs.devin.ai/cli/extensibility/mcp/overview”

   proof ↗⚑ flag

   Jules—none0/10

   No evidence that Jules can connect to or consume external MCP servers as tool providers; the only MCP-related evidence (jules-comm-19) describes someone building an MCP server that calls INTO the Jules API, which is the reverse integration direction, not Jules plugging in MCP servers for its own tool use.

   • [community] “Was able to build a personal MCP server that connects to the Jules API, letting me dispatch tasks to Jules, from Copilot Chat in VS Code.”

   proof ↗⚑ flag

4. ai-native userUse an official CLI

   weight 2 · round to Devin
   Devin✓fullprobed8/10

   Devin ships an official CLI documented extensively (adaptive model routing, slash commands, MCP extensibility) confirmed by both docs and a probe hit, plus a separate Terminal/local CLI mode for quick fixes. Missing for 10: independent hands-on review of the CLI itself and more detail on installation/versioning.

   • [claimed-docs] “Running Devin for Terminal in your local environment for quick fixes, code exploration, and interactive coding right from the command line”
   • [claimed-docs] “Adaptive analyzes your prompt and routes it to the model that will deliver the best result.”
   • [claimed-docs] “You can switch away from Adaptive to a specific model at any time with /model.”
   • [claimed-docs] “MCP (Model Context Protocol) lets you connect external tool servers to Devin CLI, giving the agent access to APIs, databases, issue trackers…”
   • [claimed-docs] “Devin CLI exposes each one as a slash command”
   • [probe] “official CLI documented at https://docs.devin.ai/cli/adaptive”

   proof ↗⚑ flag

   Jules—none0/10

   Evidence shows Jules offers a web app, GitHub integration, and a REST API for custom workflows, but no official CLI tool is documented or mentioned anywhere in the evidence pack.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”

   proof ↗⚑ flag

5. ai-native userDrive the product through a documented public API

   weight 3 · round drawn
   Devin✓fullprobed8/10

   Devin publishes a documented public API for integrating into applications, automating workflows, and creating sessions programmatically (including on behalf of other users), plus an MCP server exposing session/playbook/knowledge/scheduling management to external agents. Missing for 10: a discoverable machine-readable OpenAPI/swagger spec (probe found 404s) and independent third-party corroboration of API usage beyond vendor docs.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “create sessions, manage playbooks and knowledge, set up schedules, and more”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.devin.ai/llms.txt # Devin Docs - [Desktop (100 pages)](https://docs.devin.ai/_llms/en/desktop.md):…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

   Jules✓fullprobed8/10

   Jules ships a documented public API (developers.google.com/jules/api) with API key management, custom workflows, sending messages to the agent, and automationMode config; a community user independently built an MCP server on top of the API confirming real-world programmatic access. Missing for 10: no discoverable OpenAPI/swagger spec or llms.txt (probes returned 404s), reducing machine-readability/self-service tooling confidence.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”
   • [claimed-docs] “To send a message to the agent:”
   • [community] “Was able to build a personal MCP server that connects to the Jules API, letting me dispatch tasks to Jules, from Copilot Chat in VS Code.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://jules.google/openapi.json, https://jules.google/swagger.json, https://jules.google/api/opena…”

   proof ↗⚑ flag

6. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round to Devin
   Devin~partialclaimed3/10

   Devin documents a permission system that restricts MCP tool access at multiple levels and an API parameter (create_as_user_id) for acting on behalf of specific users, but there is no direct documentation of issuing scoped/least-privilege API keys or credentials for the agent itself. Missing for 10: explicit API key/token scoping mechanism, documented permission tiers or roles for API credentials, and any least-privilege credential issuance workflow.

   • [claimed-docs] “MCP tools are subject to the same permission system as built-in tools. You can control access at multiple levels”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”

   proof ↗⚑ flag

   Jules—none0/10

   Jules documents a basic API key creation flow (max 3 keys) but provides no evidence of scoped or least-privilege permission controls — no mention of scopes, roles, or restricted-access tokens for the agent.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”

   proof ↗⚑ flag

7. ai-native userBuild against official SDKs

   weight 2 · round to Devin
   Devin✓fullprobed8/10

   Devin publishes a documented API reference for building integrations (create sessions, manage on behalf of users, automate workflows), plus MCP support for extensibility — clear official SDK-like surface for AI-native developers. Missing for 10: no official language-specific SDK packages (e.g. Python/JS client libraries) are cited, only REST API docs and no discoverable OpenAPI spec (probe found 404s), and no independent/hands-on corroboration of API usage.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “MCP (Model Context Protocol) lets you connect external tool servers to Devin CLI, giving the agent access to APIs, databases, issue trackers…”
   • [claimed-docs] “create sessions, manage playbooks and knowledge, set up schedules, and more”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

   Jules✓fullprobed7/10

   Jules ships an official public API with documented endpoints, API key management, and use cases for building custom workflows/integrations, and a community member confirms building a personal MCP server against the Jules API. missing for 10: no official OpenAPI/SDK spec discoverable (openapi probes 404), no first-party language SDKs mentioned, and no independent SDK-quality corroboration beyond one community integration example.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”
   • [claimed-docs] “To send a message to the agent:”
   • [community] “Was able to build a personal MCP server that connects to the Jules API, letting me dispatch tasks to Jules, from Copilot Chat in VS Code.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://jules.google/openapi.json, https://jules.google/swagger.json, https://jules.google/api/opena…”

   proof ↗⚑ flag

8. ai-native userSubscribe to events via webhooks

   weight 2 · round drawn
   Devin—none0/10

   No evidence in the pack mentions webhooks or event subscription mechanisms for Devin; only API session creation, MCP integrations, and CLI features are documented. As an API-driven agentic product, webhook support would be a plausible and expected capability, so its absence counts as 'none' rather than 'na'.

   ⚑ flag

   Jules—none0/10

   Evidence shows Jules has an API for creating tasks/sending messages, notifications for task completion, and GitHub integration, but there is no mention of webhooks or event subscription mechanisms anywhere in the docs or community evidence. Missing for 10: any documentation of a webhook endpoint, event subscription API, or push-based notification mechanism to external systems.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [claimed-docs] “To send a message to the agent:”
   • [claimed-docs] “You’ll be notified when the task completes or needs your input.”

   proof ↗⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round drawn
   Devin~partialclaimed6/10

   Devin's repo indexing enables 'Ask Devin' and DeepWiki features that generate AI insights and explanations from your codebase data, and Devin can proactively suggest fixes/improvements via code review auto-fix and bug triage. However, evidence is scoped only to codebase/ticket data, not broader business or analytics data, and there's no dedicated insights dashboard or proactive summarization feature documented. Missing for 10: evidence of insights beyond code (e.g., analytics/dashboards), independent corroboration of Ask Devin/DeepWiki quality, and any UI showcasing generated insights.

   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki.”
   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures”
   • [claimed-docs] “Language migrations (e.g. JavaScript to TypeScript) Framework upgrades (e.g. Angular 16 -> 18)”

   proof ↗⚑ flag

   Jules~partialcommunity6/10

   Jules generates AI-derived plans, diffs, and PR suggestions based on analysis of the user's repository data, which is the coding-agent analog of 'AI-generated insights/suggestions from your data.' Community evidence is mixed on the quality of these suggestions, with some praising usable PRs and others criticizing low-quality output on complex codebases. Missing for 10: no evidence of broader analytics-style insights beyond code-change suggestions, and no independent benchmarking confirming insight quality across use cases.

   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [community] “its way better than the github thing in my experience it produces usable PRs”
   • [community] “I've tried using Jules for a side project, and the code quality it emits is much worse than GH Copilot, Gemini CLI, and Claude Code. It also…”

   proof ↗⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Devin
   Devin✓fullclaimed8/10

   Devin supports scheduling via cron expressions (recurring/one-time) and agent selection through its MCP tooling, plus API session creation and Slack/Teams triggering, enabling autonomous background automations beyond interactive chat sessions. missing for 10: independent/hands-on verification of scheduled runs actually executing unattended, and more detail on failure/retry handling for scheduled tasks.

   • [claimed-docs] “Supports cron expressions for recurring schedules, one-time scheduling, notification preferences, and agent selection”
   • [claimed-docs] “create sessions, manage playbooks and knowledge, set up schedules, and more”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “Tag Devin on Slack or Teams: Start sessions directly from conversations about bugs, feature requests, or questions.”
   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures”

   proof ↗⚑ flag

   Jules~partialcommunity6/10

   Jules supports background/autonomous execution via GitHub label-triggered tasks and a public API for building custom automations (e.g., automated bug-fixing, code review) with an optional automationMode for auto-PR creation, and it runs tasks in a cloud VM without needing the user present. However, docs also show a human-approval gate (plan approval, PR review) rather than fully hands-off automation, and community reports note tasks getting stuck, hitting limits, or needing babysitting, undercutting reliability of unattended runs. Missing for 10: evidence of true scheduled/cron-style recurring automations, and independent confirmation that automations run to completion without manual intervention.

   • [claimed-docs] “Use the "jules" label in an issue to assign a task directly in GitHub.”
   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [community] “I've moved infrastructure to Railway and taught the customer to use Jules. They make their own PRs now, Railway spins up an environment with…”
   • [community] “Jules was unable to complete the task in time. Please review the work done so far and provide feedback for Jules to continue.”
   • [community] “Jules quickly after a few messages/prompts just gets stuck in an endless loop like Gemini-cli does. The worst part is that there is no 'STOP…”

   proof ↗⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round drawn
   Devin✓fullcommunity8/10

   Devin itself functions as the built-in AI assistant to which tasks are delegated across web app, terminal, Slack/Teams, and API, with rich first-party documentation of the delegation workflow (docs-1,3,4,5,8,19). Community feedback raises reliability/skepticism concerns but does not concretely contradict that delegation itself works, so it's not disputed. Missing for 10: independent hands-on validation of delegation reliability/success rate beyond old demo critiques.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “using /handoff to send longer tasks to cloud Devin”
   • [claimed-docs] “Tagging Devin on a Slack or Teams thread about a bug you’re discussing with coworkers”
   • [claimed-docs] “Delegating a more complex task via the web application and taking over in Devin’s IDE once it gives you a good first draft.”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “Tag Devin on Slack or Teams: Start sessions directly from conversations about bugs, feature requests, or questions.”
   • [community] “From the graph at the end: 13.8% of issues resolved. Devin may need some additional help for awhile.”

   proof ↗⚑ flag

   Jules✓fullcommunity8/10

   Jules is exactly this kind of built-in agentic assistant: users delegate coding tasks via GitHub issue labels, the web app, or API, and Jules autonomously plans, clones repos, runs in a VM, and returns diffs/PRs for approval (jules-docs-2,3,4,7,8,9). Community evidence corroborates real-world task delegation and usable output (jules-comm-2, jules-comm-11, jules-comm-15, jules-comm-16), though mixed reports of loops, babysitting needs, and reliability issues (jules-comm-14, jules-comm-18) temper quality. Missing for 10: independent benchmarking of task success rates and evidence of consistent reliability at scale.

   • [claimed-docs] “Use the "jules" label in an issue to assign a task directly in GitHub.”
   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [community] “I used Jules three times today, very impressive! It also handles coding-adjacent work. Good github integrations.”
   • [community] “I've used this tool for a few months now and have been pretty impressed by it. It handles large quantities of tasks very well and is good at…”
   • [community] “I gave Jules a try on a new, very unorganized Python project. It failed the first time, I gave it the error message and it fixed it. I was s…”
   • [community] “I've moved infrastructure to Railway and taught the customer to use Jules. They make their own PRs now, Railway spins up an environment with…”
   • [community] “Jules quickly after a few messages/prompts just gets stuck in an endless loop like Gemini-cli does. The worst part is that there is no 'STOP…”
   • [community] “Do people really find Jules useful? I find it needs babysitting much more than Cursor.”

   proof ↗⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round drawn
   Devin✓fullcommunity8/10

   Devin is fundamentally driven by natural-language task requests across web app, Slack/Teams tagging, CLI, and API (create_as_user_id sessions), letting users describe tasks like bug fixes, features, and migrations in plain language rather than issuing structured commands. Community evidence corroborates that Devin operates via conversational/task-based prompts even amid skepticism about output quality. missing for 10: independent hands-on confirmation of NL command fidelity across all surfaces (CLI/Slack/API), and no evidence addressing ambiguous or multi-step NL instruction handling limits.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Tagging Devin on a Slack or Teams thread about a bug you’re discussing with coworkers”
   • [claimed-docs] “Delegating a more complex task via the web application and taking over in Devin’s IDE once it gives you a good first draft.”
   • [claimed-docs] “Tag Devin on Slack or Teams: Start sessions directly from conversations about bugs, feature requests, or questions.”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [community] “Clearly an extremely impressive demo... I notice in the first demo Devin hits a KeyError and decides to resolve it by wrapping the code in a…”

   proof ↗⚑ flag

   Jules✓fullcommunity8/10

   Jules is designed around natural-language task submission and messaging: users submit a task description, Jules generates a plan, and you can send follow-up messages to the agent via API or web app (jules-docs-4, jules-docs-12). Community reports confirm real-world use of NL prompts to drive coding tasks, including iterative feedback and error-message follow-ups (jules-comm-2, jules-comm-15, jules-comm-16). Missing for 10: no detailed documentation of the full range/complexity of natural-language commands supported (e.g., multi-step conversational control, command reference) and no independent benchmark of NL command robustness beyond anecdotal reports.

   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “To send a message to the agent:”
   • [community] “I used Jules three times today, very impressive! It also handles coding-adjacent work. Good github integrations.”
   • [community] “I gave Jules a try on a new, very unorganized Python project. It failed the first time, I gave it the error message and it fixed it. I was s…”
   • [community] “I've moved infrastructure to Railway and taught the customer to use Jules. They make their own PRs now, Railway spins up an environment with…”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   Devin—none0/10

   Devin documents an API reference overview and parameters (e.g. create_as_user_id) but nothing describes an interactive, in-browser 'try it' experience with runnable code examples; a probe for standard OpenAPI/Swagger interactive-doc endpoints returned 404s, suggesting no such interactive explorer exists.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

   Jules—none0/10

   Jules documents an API (create tasks, send messages, API keys) but there is no evidence of an interactive API reference or runnable code examples; probes for openapi/swagger specs all returned 404s, suggesting no such interactive reference exists.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [claimed-docs] “To send a message to the agent:”
   • [probe] “PROBE openapi: all candidate paths 404 (https://jules.google/openapi.json, https://jules.google/swagger.json, https://jules.google/api/opena…”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   Devin—none0/10

   Devin has a documented API (devin-docs-8, devin-docs-9) but a direct probe for OpenAPI/swagger spec files at common paths returned 404s (devin-probe-2), and no docs page offers a downloadable machine-readable spec.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

   Jules—none0/10

   Jules ships a documented REST API (jules-docs-9..12) but there is no evidence of a downloadable OpenAPI/Swagger spec; explicit probes for llms.txt, docs.md, and standard OpenAPI paths all returned 404 (jules-probe-1, jules-probe-2, jules-probe-3), confirming no machine-readable spec is published.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [probe] “PROBE llms.txt: HTTP 404 at https://jules.google/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://jules.google/docs.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://jules.google/openapi.json, https://jules.google/swagger.json, https://jules.google/api/opena…”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round to Jules
   Devin~partialclaimed3/10

   Devin's docs describe each session running in its own configurable workspace/environment (devin-docs-25) and connecting via VPN to reach internal services rather than being embedded in production (devin-docs-23), which implies some environment isolation, but there is no explicit claim of a dedicated 'sandbox vs production' testing mode or safeguards against touching production data. missing for 10: explicit documentation of a production-safe/sandboxed test environment, evidence of data isolation guarantees, and independent confirmation that production systems are never touched.

   • [claimed-docs] “Configure it once, and every session boots into that known-good state.”
   • [claimed-docs] “Devin can connect to a VPN from inside its workspace, so sessions can reach internal services such as package registries, databases, and int…”

   proof ↗⚑ flag

   Jules~partialclaimed5/10

   Jules runs tasks in an isolated cloud VM that clones the repo, meaning code changes happen in a sandboxed environment rather than directly on production, and PRs must be reviewed/approved before merging to the real branch. However, this is a code-sandbox for making changes, not a dedicated 'test against sandbox data/environment' feature, and there's no evidence of test-data isolation, staging environment provisioning, or explicit protection of production data/services beyond the VM/PR review flow. missing for 10: explicit sandbox test-data isolation, staging/production separation guarantees, independent verification that production systems are never touched.

   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Devin—none0/10

   There is an API reference (devin-docs-8) but no evidence of API versioning scheme or a documented deprecation policy; the openapi.json probe returned 404s, suggesting no formal machine-readable spec either. Missing for 10: versioning scheme documentation, deprecation policy/notice, changelog for breaking changes.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

   Jules—none0/10

   Jules has a public API (jules-docs-9, jules-docs-10) but there is no evidence of API versioning scheme or a documented deprecation policy; probes for OpenAPI spec/docs.md/llms.txt all returned 404 (jules-probe-1, jules-probe-2, jules-probe-3), suggesting no formal machine-readable API contract or lifecycle documentation is available.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [probe] “PROBE llms.txt: HTTP 404 at https://jules.google/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://jules.google/docs.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://jules.google/openapi.json, https://jules.google/swagger.json, https://jules.google/api/opena…”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Devin
   Devin~partialclaimed6/10

   Devin's docs describe running multiple Devins in parallel for independent tasks, an API for creating sessions programmatically (including on behalf of other users), and scheduling capabilities—together supporting bulk/automated task dispatch across many items. However, there's no concrete example or independent evidence of actually orchestrating large-scale bulk operations (e.g., processing hundreds of tickets at once), and community evidence focuses on single-task demos rather than bulk workflows. Missing for 10: hands-on/independent evidence of large-scale bulk runs, explicit parallelism limits, and a documented case study of bulk item processing.

   • [claimed-docs] “Run multiple Devins in parallel: Carve out independent tasks and run them simultaneously.”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “Supports cron expressions for recurring schedules, one-time scheduling, notification preferences, and agent selection”

   proof ↗⚑ flag

   Jules~partialcommunity5/10

   Jules exposes an API that lets users script custom workflows and dispatch tasks programmatically (jules-docs-9, jules-docs-12), and usage-limit tiers explicitly target 'power users & agent-heavy workflows' with up to 300 tasks (jules-docs-13), while community reports confirm it 'handles large quantities of tasks very well' and users built custom dispatch tooling via the API/MCP bridge (jules-comm-11, jules-comm-19). However, there's no documented native UI or endpoint for a single bulk action across many items (e.g., batch-applying one task to many repos/issues) — each task still appears to be created and reviewed individually via GitHub labels or API calls. Missing for 10: a documented batch/bulk endpoint or UI feature, first-party bulk-operation examples, and independent verification of true parallel bulk execution rather than just high per-day task volume.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “To send a message to the agent:”
   • [claimed-docs] “Power users & agent-heavy workflows ... 300 ... 60”
   • [community] “I've used this tool for a few months now and have been pretty impressed by it. It handles large quantities of tasks very well and is good at…”
   • [community] “Was able to build a personal MCP server that connects to the Jules API, letting me dispatch tasks to Jules, from Copilot Chat in VS Code.”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to Devin
   Devin~partialclaimed7/10

   Devin supports event-driven automation via scheduled/cron triggers (create schedules with cron expressions, notification preferences) and via Devin Review's Auto-Fix which automatically responds to code review comments and CI failures, plus API-based workflow automation. However, these are fixed built-in trigger types (schedule, CI/review events) rather than a general user-defined rules engine for arbitrary events. Missing for 10: evidence of a flexible custom rule/trigger builder covering arbitrary event sources (e.g., webhooks, ticket status changes) beyond scheduling and code-review auto-fix, and independent confirmation these triggers work reliably in practice.

   • [claimed-docs] “Supports cron expressions for recurring schedules, one-time scheduling, notification preferences, and agent selection”
   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “create sessions, manage playbooks and knowledge, set up schedules, and more”

   proof ↗⚑ flag

   Jules~partialclaimed5/10

   Jules supports some event-driven automation — e.g., assigning a 'jules' label to a GitHub issue automatically triggers a task, and the API lets developers build custom workflows/automations around Jules (jules-docs-2, jules-docs-9). However, there's no evidence of a general rules/conditions engine where users define arbitrary triggers and actions beyond this specific label mechanism and API calls. Missing for 10: a configurable rules interface (conditions + multiple trigger types), documentation of scheduled/webhook-based triggers, and independent confirmation that automated rule-based triggering works reliably in practice.

   • [claimed-docs] “Use the "jules" label in an issue to assign a task directly in GitHub.”
   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round to Devin
   Devin✓fullclaimed8/10

   Devin's MCP documentation explicitly states scheduling support with 'cron expressions for recurring schedules, one-time scheduling, notification preferences, and agent selection,' and this scheduling capability is exposed via MCP for session/playbook management. Missing for 10: independent/hands-on corroboration of recurring job execution in practice and more detail on scheduling UI/reliability beyond the docs mention.

   • [claimed-docs] “create sessions, manage playbooks and knowledge, set up schedules, and more”
   • [claimed-docs] “Supports cron expressions for recurring schedules, one-time scheduling, notification preferences, and agent selection”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling.”

   proof ↗⚑ flag

   Jules—none0/10

   Evidence covers task assignment via GitHub labels, API-driven custom workflows, and one-off task automation, but nothing describes recurring/scheduled jobs (e.g., cron-like triggers) — the API docs only mention creating tasks and sending messages, not recurrence.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”
   • [claimed-docs] “To send a message to the agent:”

   proof ↗⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round drawn
   Devin—none0/10

   No evidence of version history, review/diff UI, or rollback of Devin's automations (playbooks, schedules, sessions); docs mention creating/scheduling sessions and playbooks but never versioning, auditing changes, or reverting them. missing for 10: version history for playbooks/sessions, review/approval workflow for automation changes, rollback/undo mechanism.

   • [claimed-docs] “create sessions, manage playbooks and knowledge, set up schedules, and more”
   • [claimed-docs] “Supports cron expressions for recurring schedules, one-time scheduling, notification preferences, and agent selection”
   • [claimed-docs] “Knowledge is the best way to share codebase-level (vs. task-level) context that can help Devin when working in your codebase.”

   proof ↗⚑ flag

   Jules—none0/10

   Jules produces PRs and diffs via GitHub but there is no evidence of a mechanism to version, review history of, or roll back automations/tasks themselves (as opposed to code changes tracked by git/GitHub). missing for 10: automation versioning/history feature, rollback/undo of Jules tasks, changelog or audit trail for automations.

   ⚑ flag

End to end feature delivery

1. ai-native userHave an agent automatically generate and run tests to validate its own code changes before proposing them

   weight 2 · round to Devin
   Devin~partialclaimed6/10

   Devin's docs describe autonomous CI-failure iteration and auto-fix behavior (devin-docs-20) and a terminal for executing/observing command output (devin-docs-6), implying it runs and reacts to test/build results before finalizing changes, but there is no explicit documentation of Devin proactively writing new tests and running them pre-emptively to self-validate before proposing a PR. missing for 10: explicit test-generation workflow documentation, evidence of Devin authoring test cases for new code, independent hands-on confirmation of this specific validation loop.

   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures”
   • [claimed-docs] “Devin’s terminal, where you can watch commands being executed and view output logs. You can also copy the shell output for debugging purpose…”
   • [claimed-docs] “Language migrations (e.g. JavaScript to TypeScript) Framework upgrades (e.g. Angular 16 -> 18)”

   proof ↗⚑ flag

   Jules~partialcommunity4/10

   Jules runs code changes in an isolated VM and generates diffs/PRs for review, and one community report notes it is 'good at making tests for very specific/isolated functions,' implying some test generation capability, but there is no first-party documentation describing an explicit test-generate-and-run validation loop before proposing changes. missing for 10: official docs on automated test generation/execution as a validation step, evidence of test results being surfaced to users before PR creation, and independent confirmation across varied codebases.

   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [community] “I've used this tool for a few months now and have been pretty impressed by it. It handles large quantities of tasks very well and is good at…”

   proof ↗⚑ flag

2. developerHave an agent autonomously diagnose and fix a reported bug

   weight 3 · round to Jules
   Devin!disputedcontradicted5/10

   Devin's docs explicitly describe autonomous bug repro/fix workflows, Slack-triggered bug sessions, and auto-fix on CI/review failures (devin-docs-1, devin-docs-4, devin-docs-19, devin-docs-20), supporting the core story. However, independent commentary cites a low real-world resolution rate (13.8% of issues resolved) and hands-on accounts describing shallow fixes (e.g., wrapping errors in try/catch rather than真正 diagnosing root cause), directly contradicting the strength of the autonomous-fix claim (devin-comm-2, devin-comm-6, devin-comm-4). Missing for 10: independent benchmark showing high success rate, more recent hands-on verification of bug-fix quality, and resolution of the discrepancy between vendor claims and community-reported performance.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Tagging Devin on a Slack or Teams thread about a bug you’re discussing with coworkers”
   • [claimed-docs] “Tag Devin on Slack or Teams: Start sessions directly from conversations about bugs, feature requests, or questions.”
   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures”
   • [community] “From the graph at the end: 13.8% of issues resolved. Devin may need some additional help for awhile.”
   • [community] “Clearly an extremely impressive demo... I notice in the first demo Devin hits a KeyError and decides to resolve it by wrapping the code in a…”
   • [community] “Although the demos are impressive, they seem short and limited in scope... How often will the solution only be 90% correct, which is 100% no…”

   proof ↗⚑ flag

   Jules✓fullcommunity8/10

   Jules is designed to autonomously diagnose and fix issues: it clones the repo into a VM, generates a plan, makes code changes, and opens a PR for review, and can be triggered directly from a GitHub issue via the 'jules' label (a common bug-report workflow). Community evidence corroborates real-world bug-fixing use, including a case where it failed then successfully fixed the issue after being given the error message. Missing for 10: independent benchmarking on bug-fix success rate and more consistent evidence across complex codebases (some reports of failures/loops on harder tasks).

   • [claimed-docs] “Use the "jules" label in an issue to assign a task directly in GitHub.”
   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [community] “I gave Jules a try on a new, very unorganized Python project. It failed the first time, I gave it the error message and it fixed it. I was s…”
   • [community] “I've moved infrastructure to Railway and taught the customer to use Jules. They make their own PRs now, Railway spins up an environment with…”

   proof ↗⚑ flag

3. product-managerGo from a mockup or design to a working implementation without an engineering handoff

   weight 2 · round to Devin
   Devin~partialclaimed5/10

   Devin's docs show it can autonomously implement features from tickets (Jira/Linear) and be tagged by non-engineers on Slack/Teams, supporting the general 'PM delegates, Devin implements' workflow, and it can connect to Figma via MCP integrations. However, there is no documented mockup-to-code pipeline, no PM-specific onboarding flow, and no evidence of turning a design file into a working UI end-to-end. missing for 10: direct design/mockup ingestion workflow, PM-focused case study or walkthrough, independent evidence of successful design-to-implementation outcomes.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Tag Devin on Slack or Teams: Start sessions directly from conversations about bugs, feature requests, or questions.”
   • [claimed-docs] “Extend Devin's reach with MCP integrations: Connect Devin to Datadog, Sentry, databases, Figma, Notion, Stripe, and hundreds of other tool”
   • [claimed-docs] “Delegating a more complex task via the web application and taking over in Devin’s IDE once it gives you a good first draft.”

   proof ↗⚑ flag

   Jules—none0/10

   Evidence shows Jules operates on code repositories via GitHub issues/tasks, plans, diffs, and PRs, but there is no mention of accepting mockups/designs as input or any PM-oriented no-code workflow — usage still requires repo access, issue creation, and reviewing technical diffs/plans. Nothing in the pack demonstrates a design-to-implementation path bypassing engineering.

   • [claimed-docs] “Jules needs access to your repositories in order to work.”
   • [claimed-docs] “Use the "jules" label in an issue to assign a task directly in GitHub.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”

   proof ↗⚑ flag

4. developerHave an agent implement a requested feature end-to-end, including writing tests

   weight 3 · round to Jules
   Devin!disputedcontradicted5/10

   Devin's docs explicitly claim end-to-end feature implementation (docs-1, docs-31) and even test-writing is implied as part of full feature delivery, but independent hands-on reporting from the original demo showed only 13.8% of issues resolved and reviewers noted the work was often only ~90% correct or used hacky fixes (e.g. wrapping in try-catch) rather than proper fixes (devin-comm-2, devin-comm-4, devin-comm-6), directly contradicting the 'implements features end-to-end reliably' claim. missing for 10: verified success-rate data on test-writing specifically, more recent independent benchmarks showing improved reliability, and evidence tests are written/passed as part of the workflow rather than just code changes.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Language migrations (e.g. JavaScript to TypeScript) Framework upgrades (e.g. Angular 16 -> 18)”
   • [community] “From the graph at the end: 13.8% of issues resolved. Devin may need some additional help for awhile.”
   • [community] “Although the demos are impressive, they seem short and limited in scope... How often will the solution only be 90% correct, which is 100% no…”
   • [community] “Clearly an extremely impressive demo... I notice in the first demo Devin hits a KeyError and decides to resolve it by wrapping the code in a…”

   proof ↗⚑ flag

   Jules~partialcommunity6/10

   Jules is explicitly built for autonomous end-to-end coding: it clones the repo, generates a plan, modifies files, produces a diff, and opens a PR (jules-docs-3/4/7/8), and one user specifically notes it's 'good at making tests for very specific/isolated functions' (jules-comm-11), with others confirming usable PRs and time savings (jules-comm-6, jules-comm-16). However, community evidence shows significant caveats: it struggles on complex/monorepo codebases, gets stuck in loops without a stop button, needs heavy babysitting, and often fails to complete tasks unassisted (jules-comm-3, jules-comm-9, jules-comm-10, jules-comm-13, jules-comm-14, jules-comm-18). Missing for 10: consistent reliability across complex real-world features, broader evidence of comprehensive test coverage (not just isolated functions), and independent benchmarks confirming end-to-end success rate.

   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [community] “I've used this tool for a few months now and have been pretty impressed by it. It handles large quantities of tasks very well and is good at…”
   • [community] “its way better than the github thing in my experience it produces usable PRs”
   • [community] “I've moved infrastructure to Railway and taught the customer to use Jules. They make their own PRs now, Railway spins up an environment with…”
   • [community] “I've been playing with it, and I've been generally not impressed. There are obvious annoying UI bugs and the output isn't very good for anyt…”
   • [community] “I've tried using Jules for a side project, and the code quality it emits is much worse than GH Copilot, Gemini CLI, and Claude Code. It also…”
   • [community] “Jules quickly after a few messages/prompts just gets stuck in an endless loop like Gemini-cli does. The worst part is that there is no 'STOP…”
   • [community] “Do people really find Jules useful? I find it needs babysitting much more than Cursor.”

   proof ↗⚑ flag

Environment setup

1. developerHave an agent automatically clone the repo, install dependencies, and configure its own working environment

   weight 2 · round drawn
   Devin✓fullclaimed8/10

   Devin's docs describe indexing repos, configuring a persistent environment/workspace that boots into a known-good state every session, VPN access to internal registries/git hosts, and AGENTS.md-based setup instructions it follows before coding — together these show it autonomously clones, configures, and prepares its own working environment. missing for 10: independent/hands-on verification of the clone+dependency-install flow end-to-end, and explicit documentation of automatic dependency installation step-by-step.

   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki.”
   • [claimed-docs] “Devin can connect to a VPN from inside its workspace, so sessions can reach internal services such as package registries, databases, and int…”
   • [claimed-docs] “Just put an AGENTS.md file in your project root (or anywhere else). Devin will look for the file before it starts coding.”
   • [claimed-docs] “Configure it once, and every session boots into that known-good state.”

   proof ↗⚑ flag

   Jules✓fullcommunity8/10

   Jules' core workflow is documented: it runs in a cloud VM, clones the repo, installs dependencies, and configures its environment automatically before making changes, corroborated by community usage reports of automated PR generation. missing for 10: independent technical verification of dependency-install robustness across complex/monorepo setups, and some community reports note environment/config confusion in bespoke or monorepo projects.

   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [claimed-docs] “Jules needs access to your repositories in order to work.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [community] “I've moved infrastructure to Railway and taught the customer to use Jules. They make their own PRs now, Railway spins up an environment with…”
   • [community] “Jules runs in cloud-based VMs instead of on my local machine, making it much less useful than Claude Code. My projects have bespoke build sc…”
   • [community] “I've tried using Jules for a side project, and the code quality it emits is much worse than GH Copilot, Gemini CLI, and Claude Code. It also…”

   proof ↗⚑ flag

Interactive takeover

1. developerTake over an in-progress agent task in my editor, terminal, or browser to finish or redirect the work

   weight 2 · round to Devin
   Devin✓fullclaimed8/10

   Docs explicitly describe multi-surface handoff: delegate via web app then take over in Devin's IDE, use /handoff to send terminal tasks to cloud Devin, jump into the Interactive Browser to help Devin navigate, and tag Devin in Slack/Teams threads to redirect work — covering editor, terminal, and browser takeover. Missing for 10: independent/hands-on confirmation of a smooth mid-task handoff experience (community evidence is generic skepticism, not a concrete handoff failure).

   • [claimed-docs] “using /handoff to send longer tasks to cloud Devin”
   • [claimed-docs] “Delegating a more complex task via the web application and taking over in Devin’s IDE once it gives you a good first draft.”
   • [claimed-docs] “Devin’s terminal, where you can watch commands being executed and view output logs. You can also copy the shell output for debugging purpose…”
   • [claimed-docs] “You can jump in to help Devin navigate through browsing tasks via the Interactive Browser.”
   • [claimed-docs] “Tag Devin on Slack or Teams: Start sessions directly from conversations about bugs, feature requests, or questions.”
   • [claimed-docs] “Running Devin for Terminal in your local environment for quick fixes, code exploration, and interactive coding right from the command line”

   proof ↗⚑ flag

   Jules~partialcommunity5/10

   Jules supports reviewing plans, diffs, and sending follow-up messages to redirect a task (jules-docs-4, jules-docs-6, jules-docs-7, jules-docs-12), and third parties have wired the API into VS Code/Copilot Chat to dispatch tasks (jules-comm-19), but this is dispatch, not mid-task takeover. A hands-on report explicitly notes there's no 'STOP' button to interrupt a looping task (jules-comm-14), undercutting redirect control, and all takeover is browser/API-based with no native terminal or editor 'take over' UI documented. Missing for 10: documented in-editor/terminal takeover UI, ability to pause/interrupt an in-progress run, and evidence the API-based messaging genuinely redirects rather than just appends instructions.

   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “You’ll be notified when the task completes or needs your input.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “To send a message to the agent:”
   • [community] “Jules quickly after a few messages/prompts just gets stuck in an endless loop like Gemini-cli does. The worst part is that there is no 'STOP…”
   • [community] “Was able to build a personal MCP server that connects to the Jules API, letting me dispatch tasks to Jules, from Copilot Chat in VS Code.”

   proof ↗⚑ flag

2. developerSend follow-up instructions to an active agent session to steer its work without restarting

   weight 2 · round to Jules
   Devin~partialclaimed5/10

   Devin's docs describe delegating tasks and then 'taking over in Devin's IDE' or 'jumping in' to help with browsing tasks, implying some interactivity with an active session, but there is no explicit documentation of sending follow-up chat instructions to steer an in-progress session without restarting it. missing for 10: explicit docs on mid-session chat/steering, confirmation that follow-up messages are incorporated live, independent hands-on evidence of this workflow.

   • [claimed-docs] “Delegating a more complex task via the web application and taking over in Devin’s IDE once it gives you a good first draft.”
   • [claimed-docs] “You can jump in to help Devin navigate through browsing tasks via the Interactive Browser.”
   • [claimed-docs] “Tagging Devin on a Slack or Teams thread about a bug you’re discussing with coworkers”
   • [claimed-docs] “Tag Devin on Slack or Teams: Start sessions directly from conversations about bugs, feature requests, or questions.”

   proof ↗⚑ flag

   Jules✓fullcommunity7/10

   Jules's API docs explicitly describe sending a message to an active agent session (jules-docs-12), and community reports confirm the workflow of reviewing partial work and providing feedback for Jules to continue rather than restarting (jules-comm-3). This shows the steering capability works in practice, though documentation on how follow-ups affect an in-progress plan/execution is thin. Missing for 10: detailed first-party documentation of mid-task message handling/UI chat thread, and independent hands-on confirmation of steering effectiveness beyond one anecdote.

   • [claimed-docs] “To send a message to the agent:”
   • [claimed-docs] “You’ll be notified when the task completes or needs your input.”
   • [community] “Jules was unable to complete the task in time. Please review the work done so far and provide feedback for Jules to continue.”

   proof ↗⚑ flag

Sandbox execution

1. developerHave an agent safely execute code and install dependencies inside an isolated sandbox

   weight 3 · round to Jules
   Devin~partialclaimed6/10

   Docs describe Devin operating in its own workspace/VM with terminal execution, environment configuration ('known-good state'), VPN access to internal services, and multi-platform (Android/macOS/Windows) build environments, implying isolated sandboxed execution, but there is no explicit documentation of sandbox security guarantees, dependency isolation, or safety controls around code execution, and no independent/hands-on verification of sandbox isolation. missing for 10: explicit sandbox/isolation security documentation, independent verification of safe execution, details on dependency install safety and containment.

   • [claimed-docs] “Devin’s terminal, where you can watch commands being executed and view output logs. You can also copy the shell output for debugging purpose…”
   • [claimed-docs] “Devin can connect to a VPN from inside its workspace, so sessions can reach internal services such as package registries, databases, and int…”
   • [claimed-docs] “Configure it once, and every session boots into that known-good state.”
   • [claimed-docs] “Devin can build and run Android applications directly on its own machine — giving it the Android equivalent of Computer Use and browser inte…”
   • [claimed-docs] “Devin now has access to macOS virtual machines. This means Devin can now build and test iOS and macOS applications.”
   • [claimed-docs] “Devin supports Windows as a build and session platform.”

   proof ↗⚑ flag

   Jules✓fullcommunity8/10

   Jules explicitly runs in a cloud VM that clones code and installs dependencies, isolated from the user's local machine, with a plan-review step before code changes are made — directly matching the sandboxed execution story. Community reports corroborate it actually running tasks end-to-end (producing PRs) in this VM environment, though some criticize reliability/loops on complex codebases. Missing for 10: detailed docs on sandbox security boundaries (network isolation, resource limits) and independent security audit of the VM isolation.

   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [community] “Jules runs in cloud-based VMs instead of on my local machine, making it much less useful than Claude Code. My projects have bespoke build sc…”
   • [community] “I've moved infrastructure to Railway and taught the customer to use Jules. They make their own PRs now, Railway spins up an environment with…”

   proof ↗⚑ flag

Approval controls

1. developerConfigure an agent to auto-approve all its actions instead of confirming each one

   weight 2 · round to Jules
   Devin—none0/10

   The evidence pack describes permission systems for MCP tools (devin-docs-13) and autonomous features like Auto-Fix (devin-docs-20), but there is no documentation of a setting that lets a developer toggle between 'confirm each action' and 'auto-approve all actions' for Devin's own task execution. missing for 10: explicit auto-approve/confirmation toggle documentation, evidence of a per-action approval workflow that can be disabled, any community confirmation of this specific control.

   • [claimed-docs] “MCP tools are subject to the same permission system as built-in tools. You can control access at multiple levels”
   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures”

   proof ↗⚑ flag

   Jules~partialclaimed4/10

   Jules's default workflow requires manual approval at multiple stages (plan approval, diff review, PR approval), but the API docs mention an optional `automationMode` field that changes default behavior around automatic PR creation, suggesting some configurable auto-approve path exists via API rather than the standard UI flow. Missing for 10: explicit documentation of a setting that suppresses plan-approval and diff-review confirmations entirely, and any hands-on/community confirmation that this automation mode actually skips human checkpoints.

   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”

   proof ↗⚑ flag

2. product-managerApprove key agent decisions from my phone while agents continue working

   weight 1 · round to Jules
   Devin—none0/10

   No evidence describes a mobile app or phone-based approval interface for agent decisions; docs mention Slack/Teams tagging, web app, IDE, and scheduling notifications but nothing about approving decisions from a phone while sessions continue. The axis is plausible for an agentic dev tool with human-in-the-loop needs, but no supporting evidence exists.

   ⚑ flag

   Jules~partialcommunity6/10

   Jules is a cloud-based agent with a web app and notification system, plan approval, diff review, and PR approval steps, and community evidence confirms users review/approve PRs from a browser including 'coding on my phone' via mobile web. However, there is no dedicated native mobile app or explicit mobile-optimized approval UI, and no evidence of push-notification-driven approval flows tailored to on-the-go PM decision-making. missing for 10: dedicated mobile app or mobile-specific UI, evidence of push notifications enabling quick phone-based approvals, PM-specific (non-developer) approval workflow.

   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “You’ll be notified when the task completes or needs your input.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [community] “I've been actually kind-of enjoying using Jules as a way of 'coding' my side project using my phone. By the time I get home in the evening I…”

   proof ↗⚑ flag

3. engineering-leadSet tiered autonomy levels controlling what an agent can do without manual confirmation

   weight 3 · round drawn
   Devin~partialclaimed4/10

   Devin documents a permission system for MCP/tool access that can be controlled 'at multiple levels' and features like Auto-Fix, human takeover in the IDE, and interactive browser intervention imply some control over autonomous action, but there is no documented feature specifically framed as configurable 'tiered autonomy levels' for an engineering lead to set organization-wide policies on unattended agent actions. missing for 10: explicit autonomy-tier configuration UI/API, organization-wide policy controls, and confirmation-threshold settings distinct from ad-hoc MCP tool permissions.

   • [claimed-docs] “MCP tools are subject to the same permission system as built-in tools. You can control access at multiple levels”
   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures”
   • [claimed-docs] “You can jump in to help Devin navigate through browsing tasks via the Interactive Browser.”
   • [claimed-docs] “Delegating a more complex task via the web application and taking over in Devin’s IDE once it gives you a good first draft.”

   proof ↗⚑ flag

   Jules~partialclaimed4/10

   Jules has a binary plan-approval workflow (review/approve plan, review diffs, approve PRs) and an API `automationMode` field that toggles whether PRs are auto-created, but there's no evidence of configurable tiered autonomy levels (e.g., low/medium/high trust settings, granular permission scopes, or per-action confirmation thresholds) that an engineering-lead could set. Missing for 10: explicit multi-tier autonomy/permission settings, admin-configurable trust levels, and any org-wide policy controls beyond the single automationMode toggle and default plan-approval gate.

   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”

   proof ↗⚑ flag

Model control

1. ai-native userHave each task prompt automatically routed to the most suitable underlying model

   weight 2 · round to Devin
   Devin✓fullprobed8/10

   Devin's CLI has an explicit "Adaptive" model router that analyzes each prompt and automatically routes it to the best-performing model, with an override via /model if manual control is desired. This directly matches the story of automatic per-task model routing, though it's documented only for the CLI product surface and lacks independent/hands-on validation of routing quality. Missing for 10: independent corroboration of routing accuracy, and confirmation this applies across all Devin surfaces (web/IDE) not just CLI.

   • [claimed-docs] “Adaptive analyzes your prompt and routes it to the model that will deliver the best result.”
   • [claimed-docs] “You can switch away from Adaptive to a specific model at any time with /model.”
   • [claimed-docs] “Adaptive is an intelligent model router that automatically selects the best AI model for each task.”
   • [probe] “official CLI documented at https://docs.devin.ai/cli/adaptive”

   proof ↗⚑ flag

   Jules—none0/10

   No evidence Jules performs automatic model routing per task; docs mention only a single flagship model (Gemini 3 Pro) as 'priority access,' with no mention of routing prompts to different models based on suitability. Missing for 10: any documentation of multi-model routing logic, model-selection criteria, or automatic switching between models.

   • [claimed-docs] “Priority access to the latest models, starting with Gemini 3 Pro”

   proof ↗⚑ flag

2. engineering-leadSwitch away from automatic model selection to a specific model of my choice

   weight 1 · round to Devin
   Devin✓fullprobed8/10

   Devin's docs explicitly state that Adaptive automatically routes prompts to the best model, but users can switch away from Adaptive to a specific model at any time using /model, directly matching the story. Missing for 10: independent/hands-on corroboration of the /model switch in practice and more detail on which specific models are selectable.

   • [claimed-docs] “Adaptive analyzes your prompt and routes it to the model that will deliver the best result.”
   • [claimed-docs] “You can switch away from Adaptive to a specific model at any time with /model.”
   • [claimed-docs] “Adaptive is an intelligent model router that automatically selects the best AI model for each task.”
   • [probe] “official CLI documented at https://docs.devin.ai/cli/adaptive”

   proof ↗⚑ flag

   Jules—none0/10

   No evidence pack item mentions model selection settings or the ability to choose a specific underlying model; the only related mention is 'Priority access to the latest models, starting with Gemini 3 Pro' which describes access tiers, not user-controlled model switching.

   ⚑ flag

Visibility monitoring

1. developerWatch what a running agent is doing in real time, including its current status

   weight 3 · round to Devin
   Devin✓fullclaimed7/10

   Devin's docs describe real-time observability features like watching the terminal execute commands with live output logs, jumping in during interactive browsing, and taking over in Devin's IDE mid-task, all indicating a live view of the agent's current activity/status. Missing for 10: no explicit dedicated 'status dashboard' documentation and no independent/hands-on corroboration of the real-time monitoring UX beyond vendor docs.

   • [claimed-docs] “Devin’s terminal, where you can watch commands being executed and view output logs. You can also copy the shell output for debugging purpose…”
   • [claimed-docs] “You can jump in to help Devin navigate through browsing tasks via the Interactive Browser.”
   • [claimed-docs] “Delegating a more complex task via the web application and taking over in Devin’s IDE once it gives you a good first draft.”

   proof ↗⚑ flag

   Jules~partialcommunity5/10

   Jules provides notifications on task completion/input-needed and a plan-approval step, implying some status visibility, plus a diff/PR review flow, but there is no evidence of a live real-time activity feed or streaming log of the agent's current actions while running; community feedback even notes the absence of a stop/interrupt control mid-run, suggesting limited real-time observability. Missing for 10: evidence of a real-time execution log/console view, granular step-by-step status updates, and independent confirmation of live monitoring UX.

   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “You’ll be notified when the task completes or needs your input.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [community] “Jules quickly after a few messages/prompts just gets stuck in an endless loop like Gemini-cli does. The worst part is that there is no 'STOP…”

   proof ↗⚑ flag

2. developerGet notified when an agent completes a task or needs my input

   weight 2 · round to Jules
   Devin~partialclaimed6/10

   Devin supports Slack/Teams tagging where sessions and results appear in-thread (devin-docs-4, devin-docs-19), and scheduled sessions explicitly support 'notification preferences' (devin-docs-17), implying some completion/status alerts. However there's no dedicated first-party documentation of a generic notify-on-completion or needs-input alert system (e.g., email/push) outside Slack/Teams and scheduling contexts. Missing for 10: explicit documentation of a unified notification mechanism across all session types, and independent/hands-on confirmation that notifications reliably fire when input is needed.

   • [claimed-docs] “Tagging Devin on a Slack or Teams thread about a bug you’re discussing with coworkers”
   • [claimed-docs] “Tag Devin on Slack or Teams: Start sessions directly from conversations about bugs, feature requests, or questions.”
   • [claimed-docs] “Supports cron expressions for recurring schedules, one-time scheduling, notification preferences, and agent selection”
   • [claimed-docs] “You can jump in to help Devin navigate through browsing tasks via the Interactive Browser.”

   proof ↗⚑ flag

   Jules✓fullcommunity8/10

   First-party docs explicitly state users are notified when a task completes or needs input, and the workflow (plan approval, diff review, PR creation) reinforces oversight checkpoints where notifications matter. Community evidence corroborates the async, check-back-later usage pattern (e.g. reviewing PRs after time away), consistent with notification-driven workflows. Missing for 10: independent verification of notification channels (email/push/Slack), and no detail on notification reliability or configurability.

   • [claimed-docs] “You’ll be notified when the task completes or needs your input.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [community] “I've been actually kind-of enjoying using Jules as a way of 'coding' my side project using my phone. By the time I get home in the evening I…”
   • [community] “Jules was unable to complete the task in time. Please review the work done so far and provide feedback for Jules to continue.”

   proof ↗⚑ flag

Natural language task intake

1. developerDescribe a feature or bug in plain language and have it automatically turned into a scoped implementation task

   weight 3 · round to Devin
   Devin✓fullcommunity7/10

   Devin's docs describe accepting natural-language asks (tickets, Slack/Teams messages, delegated tasks) and turning them into sessions/implementation tasks, which is the core intent-to-spec workflow (devin-docs-1, devin-docs-4, devin-docs-5, devin-docs-19). Community commentary raises concerns about actual task success rates and correctness (devin-comm-2, devin-comm-4), tempering confidence in scoping quality. Missing for 10: independent hands-on evidence that the auto-generated task scope/plan is accurate or reviewable before execution, and no dedicated 'plan/spec preview' feature described.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Tagging Devin on a Slack or Teams thread about a bug you’re discussing with coworkers”
   • [claimed-docs] “Delegating a more complex task via the web application and taking over in Devin’s IDE once it gives you a good first draft.”
   • [claimed-docs] “Tag Devin on Slack or Teams: Start sessions directly from conversations about bugs, feature requests, or questions.”
   • [community] “From the graph at the end: 13.8% of issues resolved. Devin may need some additional help for awhile.”
   • [community] “Although the demos are impressive, they seem short and limited in scope... How often will the solution only be 90% correct, which is 100% no…”

   proof ↗⚑ flag

   Jules~partialcommunity6/10

   Jules docs clearly show a plain-language task submission flow that generates a reviewable/approvable plan before code changes, then produces a diff and PR (jules-docs-4, jules-docs-7, jules-docs-8), directly matching the intent-to-spec story. However community reports show inconsistent scoping quality — confusion in monorepos, endless loops, and needing heavy babysitting — indicating the automatic scoping doesn't always hold up in practice. Missing for 10: independent verification of plan/spec quality on complex codebases, and consistent evidence the generated plan reliably matches developer intent without back-and-forth correction.

   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [community] “I've tried using Jules for a side project, and the code quality it emits is much worse than GH Copilot, Gemini CLI, and Claude Code. It also…”
   • [community] “Jules quickly after a few messages/prompts just gets stuck in an endless loop like Gemini-cli does. The worst part is that there is no 'STOP…”
   • [community] “Do people really find Jules useful? I find it needs babysitting much more than Cursor.”

   proof ↗⚑ flag

2. product-managerConvert user feedback submissions into structured tasks with proposed scope

   weight 2 · round to Devin
   Devin~partialclaimed5/10

   Devin can ingest raw inputs like Linear/Jira tickets and Slack/Teams threads about bugs or feature requests and turn them into an actionable session (devin-docs-1, devin-docs-4, devin-docs-19), which covers the 'feedback-to-task' intake path, but there is no evidence of a distinct structured-task output with an explicit proposed scope/plan artifact for PM review before execution. Missing for 10: evidence of a scoping/planning deliverable (e.g., a spec doc or task breakdown with effort/scope estimate) generated prior to coding, and any PM-specific workflow or UI for reviewing/approving that scope.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Tagging Devin on a Slack or Teams thread about a bug you’re discussing with coworkers”
   • [claimed-docs] “Tag Devin on Slack or Teams: Start sessions directly from conversations about bugs, feature requests, or questions.”
   • [claimed-docs] “Delegating a more complex task via the web application and taking over in Devin’s IDE once it gives you a good first draft.”

   proof ↗⚑ flag

   Jules—none0/10

   Jules's documented workflow is a coding-agent pipeline (clone repo, generate a plan, produce diffs/PRs) driven by developer-specified tasks or GitHub issues, not a mechanism for ingesting unstructured user-feedback and converting it into a structured task with proposed scope for a PM. No evidence shows feedback intake, requirement structuring, or scope proposal features aimed at product managers.

   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Use the "jules" label in an issue to assign a task directly in GitHub.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”

   proof ↗⚑ flag

3. developerAttach a marked-up screenshot or mockup to a task so the agent implements the correct visual change

   weight 2 · round drawn
   Devin—none0/10

   No evidence in the pack mentions attaching screenshots, mockups, or marked-up images to a Devin task, nor any visual-diff/design-to-code workflow; documentation covers ticket delegation, terminal, Slack, MCP, and environment setup but nothing about image/mockup-based task intake.

   ⚑ flag

   Jules—none0/10

   No evidence Jules supports attaching images, screenshots, or mockups to a task — task submission is described only via text prompts, GitHub issue labels, or API messages. Missing for 10: any mention of image/screenshot attachment, multimodal input handling, or visual mockup interpretation.

   ⚑ flag

Plan approval

1. developerReview and approve an agent's implementation plan before any code changes are made

   weight 3 · round to Jules
   Devin—none0/10

   The evidence pack describes Devin's task delegation, IDE handoff, terminal visibility, and MCP integrations, but nowhere documents a plan-review/approval gate before Devin begins making code changes. Missing for 10: any doc mentioning an upfront plan proposal, explicit approval step, or 'plan mode' prior to execution.

   • [claimed-docs] “Delegating a more complex task via the web application and taking over in Devin’s IDE once it gives you a good first draft.”
   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”

   proof ↗⚑ flag

   Jules✓fullclaimed8/10

   Jules docs explicitly state the agent generates a plan upon task submission that users can review and approve before any code changes are made, directly matching the story. No independent/hands-on evidence specifically confirms or contradicts this plan-approval step, though other workflow claims (diffs, PRs) are corroborated by community mentions. Missing for 10: independent/hands-on confirmation of the plan-review step specifically, and more detail on what the plan interface looks like.

   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”

   proof ↗⚑ flag

2. engineering-leadApprove a task's scope and contract before an agent is allowed to modify the repository

   weight 2 · round to Jules
   Devin—none0/10

   The evidence pack shows delegation, review-after-the-fact, and IDE takeover workflows, but no documented mechanism where an engineering-lead must approve a task's scope/contract before Devin is allowed to start modifying the repository. Missing for 10: explicit pre-execution approval/gating workflow, scope/contract sign-off UI or API, evidence of blocking repo writes until lead approval.

   • [claimed-docs] “Delegating a more complex task via the web application and taking over in Devin’s IDE once it gives you a good first draft.”
   • [claimed-docs] “using /handoff to send longer tasks to cloud Devin”

   proof ↗⚑ flag

   Jules~partialclaimed6/10

   Jules generates a plan that the user can review and approve before any code changes are made, giving an engineering-lead a gate before modification, and it also produces diffs/PRs requiring approval before merge. However, this is a general 'plan approval' UX rather than a formal scope/contract sign-off workflow tailored for engineering-lead governance (e.g., no evidence of role-based approval gates, policy enforcement, or blocking unauthorized starts). missing for 10: explicit lead/role-based approval gating before task execution starts, evidence of enforceable contract/scope definitions beyond a plan preview, independent confirmation that the plan-approval step reliably blocks modification.

   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”

   proof ↗⚑ flag

Ticket driven tasking

1. developerAssign a coding task to an agent directly from an existing issue or ticket

   weight 3 · round drawn
   Devin✓fullclaimed8/10

   Docs explicitly state Devin can be asked to tackle Linear/Jira tickets, tagged on Slack/Teams threads about bugs to start sessions, and integrated via API/MCP to create sessions from external triggers, directly supporting assigning tasks from existing issues/tickets. Missing for 10: independent hands-on verification of ticket-to-task handoff and no direct product screenshot of ticket-linking UI in evidence.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Tag Devin on Slack or Teams: Start sessions directly from conversations about bugs, feature requests, or questions.”
   • [claimed-docs] “Tagging Devin on a Slack or Teams thread about a bug you’re discussing with coworkers”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”

   proof ↗⚑ flag

   Jules✓fullcommunity8/10

   Jules explicitly supports assigning tasks directly from GitHub issues via a 'jules' label, and API/community evidence corroborates task dispatch and PR creation workflows. Missing for 10: evidence of ticket-tracker integrations beyond GitHub (e.g., Jira/Linear) and independent hands-on confirmation of the label-to-task flow itself.

   • [claimed-docs] “Use the "jules" label in an issue to assign a task directly in GitHub.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [community] “its way better than the github thing in my experience it produces usable PRs”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round drawn
   Devin~partialprobed5/10

   Devin exposes a documented API (session creation, create_as_user_id) and an MCP server that surfaces session management, playbooks, knowledge, and scheduling (devin-docs-8,9,15,29), suggesting broad but not full parity with the web/IDE UI. There's no explicit claim or evidence that UI-only features like the Interactive Browser takeover, terminal handoff, VPN setup, or environment/emulation configuration are fully API-accessible, and no OpenAPI spec was found (devin-probe-2) to confirm completeness. Missing for 10: explicit vendor statement of API/UI parity, evidence that interactive/browser/VPN/environment features are API-controllable, and independent confirmation of parity.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “create sessions, manage playbooks and knowledge, set up schedules, and more”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

   Jules~partialprobed5/10

   Jules exposes a public API for creating tasks, sending messages, and enabling automation (PR creation) — core building blocks of the UI workflow — and a community user confirms dispatching tasks via the API from an external MCP client, showing real usage beyond the UI. However, there's no documented API coverage for plan approval, diff browsing/approval, or settings/API-key management, and no OpenAPI spec is discoverable (probes 404), so full UI/API parity is unconfirmed. Missing for 10: API endpoints for plan review/approval, diff/PR review parity, settings management, and a public API spec confirming full feature coverage.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”
   • [claimed-docs] “To send a message to the agent:”
   • [community] “Was able to build a personal MCP server that connects to the Jules API, letting me dispatch tasks to Jules, from Copilot Chat in VS Code.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://jules.google/openapi.json, https://jules.google/swagger.json, https://jules.google/api/opena…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round drawn
   Devin—none0/10

   No evidence of any data export feature, open-format export, or account/data portability tooling in Devin's docs; evidence covers task delegation, MCP, CLI, and platform support but nothing about exporting session data, knowledge, or playbooks in open formats for migration away from the product.

   ⚑ flag

   Jules—none0/10

   No evidence of any data export feature, open-format export, or account/data portability mechanism for Jules; the product works via GitHub repos/PRs but nothing indicates users can export Jules-specific data (task history, configs, etc.) and leave. missing for 10: any documentation of data export, open format support, or account portability/deletion workflow.

   ⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round drawn
   Devin—none0/10

   Devin is a closed-source commercial SaaS product; no evidence of any open-source license or public source repository is present in the evidence pack, and the axis is applicable since AI tooling could conceivably publish open-source components. missing for 10: any open-license source release, public repo link, or licensing statement.

   ⚑ flag

   Jules—none0/10

   Jules is a closed, proprietary Google product (cloud VM, API keys, usage limits); no evidence anywhere of source code being published or licensed openly, and probes for docs/openapi artifacts return 404s, further suggesting no open publishing.

   • [claimed-docs] “Jules needs access to your repositories in order to work.”
   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [probe] “PROBE llms.txt: HTTP 404 at https://jules.google/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://jules.google/docs.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://jules.google/openapi.json, https://jules.google/swagger.json, https://jules.google/api/opena…”

   proof ↗⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round drawn
   Devin—none0/10

   Devin is exclusively delivered as a cloud SaaS product (with local CLI/terminal front-ends that hand off to cloud sessions); no evidence anywhere in the docs or community discussion describes a self-hostable core engine or on-prem deployment option. Missing for 10: any documentation of self-hosted/on-prem deployment, container/binary distribution of the core agent, or licensing for self-hosting.

   • [claimed-docs] “Running Devin for Terminal in your local environment for quick fixes, code exploration, and interactive coding right from the command line”
   • [claimed-docs] “using /handoff to send longer tasks to cloud Devin”
   • [claimed-docs] “Configure it once, and every session boots into that known-good state.”

   proof ↗⚑ flag

   Jules—none0/10

   Jules is explicitly a cloud-based VM service (Google-hosted) with no evidence of any self-hosting option; community even notes cloud-only design as a drawback compared to local tools like Claude Code.

   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [community] “Jules runs in cloud-based VMs instead of on my local machine, making it much less useful than Claude Code. My projects have bespoke build sc…”
   • [community] “It's a shame Google picked the wrong system design for Jules. Claude Code's system design is clearly superior at this point.”

   proof ↗⚑ flag

Enterprise licensing

1. engineering-leadLicense an enterprise deployment with SSO and commercial support for organization-wide rollout

   weight 2 · round drawn
   Devin—none0/10

   No evidence pack items mention enterprise licensing, SSO, or commercial support offerings for Devin; only product feature docs and community sentiment are present. Missing for 10: SSO/SAML documentation, enterprise plan/pricing page, commercial support SLA details.

   ⚑ flag

   Jules—none0/10

   No evidence of enterprise licensing, SSO, or commercial support offerings; docs mention only individual API keys and usage-tier limits, and community discussion references a free beta plan, not enterprise deployment. missing for 10: SSO integration, enterprise licensing/contract terms, commercial support SLAs, org-wide admin/rollout tooling.

   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [claimed-docs] “Power users & agent-heavy workflows ... 300 ... 60”
   • [community] “Is Jules free of charge? Yes, for now, Jules is free of charge. Jules is in beta and available without payment while we learn from usage.”

   proof ↗⚑ flag

Model flexibility

1. engineering-leadBring my own LLM or API key so agents run on the model of my choice

   weight 2 · round drawn
   Devin—none0/10

   Docs describe an internal 'Adaptive' model router and a /model command to pick among Devin's supported models, but there is no evidence of bringing a custom LLM or your own API key to run agents on a model of the engineering lead's choice.

   • [claimed-docs] “Adaptive analyzes your prompt and routes it to the model that will deliver the best result.”
   • [claimed-docs] “You can switch away from Adaptive to a specific model at any time with /model.”
   • [claimed-docs] “Adaptive is an intelligent model router that automatically selects the best AI model for each task.”

   proof ↗⚑ flag

   Jules—none0/10

   No evidence that Jules supports bringing a custom LLM or third-party API key; it is tied to Google's own models (Gemini), with documentation only mentioning Jules's own API key for accessing Jules itself, not for configuring underlying model providers. missing for 10: any mention of BYO-LLM support, model selection options, or third-party API key configuration.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [claimed-docs] “Priority access to the latest models, starting with Gemini 3 Pro”

   proof ↗⚑ flag

Usage quotas

1. engineering-leadSee and manage plan-based daily task and concurrency limits for agent workflows

   weight 2 · round to Jules
   Devin—none0/10

   No evidence pack item mentions plan tiers, daily task quotas, concurrency limits, or an admin console/dashboard for managing such limits; docs cover running parallel Devins and API session creation but not visibility or management of plan-based limits. Missing for 10: pricing/plan documentation, admin UI for concurrency/task caps, usage dashboards or alerts for limit management.

   • [claimed-docs] “Run multiple Devins in parallel: Carve out independent tasks and run them simultaneously.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”

   proof ↗⚑ flag

   Jules~partialcommunity4/10

   Jules docs reference a usage-limits page with plan-tiered daily task numbers (e.g., 300/60) and community confirms daily task limits exist and change between releases (60→15 on free plan), showing plan-based limits are real and documented. However there's no evidence of an engineering-lead-facing dashboard or admin console to monitor team usage, set/adjust concurrency, or manage limits across a team — only a static limits reference page and anecdotal user experience of hitting caps. Missing for 10: team/org usage dashboard, per-user concurrency visibility, ability to configure or request limit changes, and any admin/lead-specific management UI.

   • [claimed-docs] “Power users & agent-heavy workflows ... 300 ... 60”
   • [community] “The daily task limit went down from 60 to 15 (on the free plan) with this release. Personally I wasn't close to exhausting the limit because…”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round drawn
   Devin—none0/10

   No evidence in the pack mentions data residency, regional storage options, or compliance controls for where session/data is stored; docs cover VPN connectivity, environments, and MCP but nothing about choosing a storage region. Missing for 10: any mention of data residency/region selection, compliance certifications (e.g., SOC2/GDPR data locality), or enterprise data storage controls.

   ⚑ flag

   Jules—none0/10

   No evidence in the pack mentions data residency, regional storage options, or any control over where Jules stores/processes data; docs only describe VM execution and GitHub integration without residency settings.

   ⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   Devin—none0/10

   No evidence in the pack addresses data-training opt-out, privacy controls, or any policy about excluding user data from model training; all docs cover feature capabilities, MCP, CLI, and environment support instead.

   ⚑ flag

   Jules—none0/10

   No evidence in the pack addresses data usage/training opt-out policies, privacy controls, or any statement about whether user code/data is used to train AI models; the docs focus on functionality (VM execution, PR creation, API) rather than data governance.

   ⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round drawn
   Devin—none0/10

   No evidence in the pack addresses data retention policies, deletion controls, or privacy configuration options for Devin; the docs cover features, MCP, CLI, and environments but not data governance/retention settings.

   ⚑ flag

   Jules—none0/10

   No evidence pack item mentions data retention policies, data deletion controls, or privacy settings for repositories/code processed by Jules; docs cover workflow, VM execution, and API usage but not retention/deletion controls.

   ⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Devin—none0/10

   No evidence pack item mentions telemetry opt-out, analytics settings, or usage-tracking controls for Devin; nothing addresses this privacy axis at all.

   ⚑ flag

   Jules—none0/10

   No evidence in the pack mentions telemetry, usage tracking, or an opt-out mechanism; the docs focus on repo access, VM execution, PR workflow, and API usage without addressing privacy/telemetry controls.

   ⚑ flag

Chat integration

1. developerTag an agent in a chat thread to discuss and delegate a bug or task

   weight 2 · round to Devin
   Devin✓fullclaimed8/10

   Docs explicitly describe tagging Devin in Slack/Teams threads to discuss and delegate bugs/tasks, starting sessions directly from conversations. Missing for 10: independent/hands-on corroboration of this specific chat-tagging workflow beyond vendor docs.

   • [claimed-docs] “Tagging Devin on a Slack or Teams thread about a bug you’re discussing with coworkers”
   • [claimed-docs] “Tag Devin on Slack or Teams: Start sessions directly from conversations about bugs, feature requests, or questions.”
   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”

   proof ↗⚑ flag

   Jules~partialcommunity4/10

   Jules supports delegating tasks via a GitHub issue label ('jules') and via API messages to the agent, which is a form of task delegation but not the classic 'tag an agent in a chat thread to discuss' interaction pattern; one community user built a custom MCP server to dispatch Jules tasks from VS Code Copilot Chat, showing it's possible but not a native feature. missing for 10: native chat-thread tagging/mention UI, evidence of back-and-forth discussion in a thread before delegation, first-party support for Slack/Teams-style @mentions.

   • [claimed-docs] “Use the "jules" label in an issue to assign a task directly in GitHub.”
   • [claimed-docs] “To send a message to the agent:”
   • [community] “Was able to build a personal MCP server that connects to the Jules API, letting me dispatch tasks to Jules, from Copilot Chat in VS Code.”

   proof ↗⚑ flag

Knowledge context

1. developerAdd a context file describing my codebase conventions so agents generate more relevant plans and code

   weight 3 · round to Devin
   Devin✓fullclaimed9/10

   Devin explicitly supports AGENTS.md files placed in the project root that Devin reads before coding, plus a separate 'Knowledge' feature for sharing codebase-level conventions/context to improve plans and code generation. Missing for 10: independent/hands-on confirmation that this measurably improves plan/code relevance beyond vendor docs.

   • [claimed-docs] “Just put an AGENTS.md file in your project root (or anywhere else). Devin will look for the file before it starts coding.”
   • [claimed-docs] “Knowledge is the best way to share codebase-level (vs. task-level) context that can help Devin when working in your codebase.”
   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki.”

   proof ↗⚑ flag

   Jules✓fullclaimed7/10

   Jules automatically reads an AGENTS.md file in the repo root to guide its plans and code generation, directly matching the story of adding a context file describing codebase conventions. Missing for 10: no independent/hands-on confirmation of how well conventions in AGENTS.md actually improve plan/code relevance, and no detail on supported format/scope beyond the single doc mention.

   • [claimed-docs] “Jules now automatically looks for a file named AGENTS.md in the root of your repository.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”

   proof ↗⚑ flag

2. developerQuery generated documentation for any public or private repository

   weight 1 · round to Devin
   Devin✓fullclaimed9/10

   Devin explicitly documents 'DeepWiki'-style repo indexing and MCP tools that 'explore and query documentation for any GitHub repository (public or private with authentication)', directly matching the story, backed by repo indexing docs enabling Ask Devin/DeepWiki features. Missing for 10: independent hands-on verification of documentation query quality/accuracy beyond vendor docs.

   • [claimed-docs] “These tools let you explore and query documentation for any GitHub repository (public or private with authentication)”
   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki.”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling.”
   • [claimed-docs] “create sessions, manage playbooks and knowledge, set up schedules, and more”

   proof ↗⚑ flag

   Jules—none0/10

   Evidence describes Jules as a task-based coding agent that clones repos, edits code, and creates PRs, but there is no mention of generating or letting users query documentation for a repository (public or private). missing for 10: any docs-generation feature, a documentation query/search interface, evidence of indexing repo docs for Q&A.

   • [claimed-docs] “Jules needs access to your repositories in order to work.”
   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”

   proof ↗⚑ flag

Project management integration

1. product-managerConnect issue trackers like Jira, Linear, ClickUp, or Monday.com so agents can manage tickets directly

   weight 3 · round to Devin
   Devin~partialclaimed6/10

   Devin explicitly supports Linear and Jira tickets natively (devin-docs-1) and its MCP extensibility (devin-docs-12, devin-docs-21) lets it connect to arbitrary tool servers including issue trackers, which could cover ClickUp/Monday.com. However, there's no first-party documentation of native ClickUp or Monday.com connectors, and no independent/hands-on evidence confirming ticket-management workflows actually work end-to-end via these integrations. missing for 10: explicit native ClickUp/Monday.com support docs, independent verification of issue-tracker workflows succeeding in practice.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “MCP (Model Context Protocol) lets you connect external tool servers to Devin CLI, giving the agent access to APIs, databases, issue trackers…”
   • [claimed-docs] “Extend Devin's reach with MCP integrations: Connect Devin to Datadog, Sentry, databases, Figma, Notion, Stripe, and hundreds of other tool”
   • [claimed-docs] “MCP tools are subject to the same permission system as built-in tools. You can control access at multiple levels”

   proof ↗⚑ flag

   Jules—none0/10

   Evidence only shows Jules integrating with GitHub (repos, issues, PRs) and provides an API for custom workflows, but there is no mention of Jira, Linear, ClickUp, Monday.com, or any issue-tracker integration beyond GitHub.

   ⚑ flag

Version control integration

1. developerConnect a GitHub repository so an agent can access the code and open pull requests against it

   weight 3 · round to Jules
   Devin~partialclaimed6/10

   Docs confirm repository indexing (public/private) and GitHub-repo access for context, plus 'Devin Review with Auto-Fix' responding to code review/CI, implying PR interaction, but there's no explicit documentation of a GitHub connection/auth flow or of Devin opening pull requests directly. missing for 10: explicit GitHub app/OAuth connection setup steps, explicit 'opens pull requests' documentation, independent/hands-on confirmation of PR creation.

   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki.”
   • [claimed-docs] “These tools let you explore and query documentation for any GitHub repository (public or private with authentication)”
   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures”
   • [claimed-docs] “Devin can connect to a VPN from inside its workspace, so sessions can reach internal services such as package registries, databases, and int…”

   proof ↗⚑ flag

   Jules✓fullcommunity9/10

   Jules is built around GitHub repo access: docs describe cloning repos, GitHub issue label assignment, generating diffs/plans, and creating PRs that can be merged on GitHub, corroborated by community reports of receiving usable PRs from Jules-driven tasks. Missing for 10: independent verification of the full connect-repo setup flow and edge-case reliability across repo types.

   • [claimed-docs] “Jules needs access to your repositories in order to work.”
   • [claimed-docs] “Use the "jules" label in an issue to assign a task directly in GitHub.”
   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [community] “its way better than the github thing in my experience it produces usable PRs”
   • [community] “I've moved infrastructure to Railway and taught the customer to use Jules. They make their own PRs now, Railway spins up an environment with…”

   proof ↗⚑ flag

2. developerGrant an agent access to my repositories with a one-click install, without complex setup

   weight 2 · round to Jules
   Devin~partialclaimed5/10

   Devin does support connecting to repos (indexing, GitHub-based session creation, VPN access to internal git hosts) and this is described as straightforward onboarding, but there's no evidence of a literal 'one-click install' flow like a GitHub App marketplace install — setup instead involves indexing, environment configuration, AGENTS.md, and knowledge onboarding steps. Missing for 10: explicit one-click/marketplace-style install flow, independent evidence of setup simplicity, and confirmation that repo access requires no additional configuration.

   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki.”
   • [claimed-docs] “Devin can connect to a VPN from inside its workspace, so sessions can reach internal services such as package registries, databases, and int…”
   • [claimed-docs] “Configure it once, and every session boots into that known-good state.”
   • [claimed-docs] “Just put an AGENTS.md file in your project root (or anywhere else). Devin will look for the file before it starts coding.”

   proof ↗⚑ flag

   Jules~partialcommunity6/10

   Jules integrates with GitHub (label-based task assignment, PR creation) and needs repo access to work, implying a connect/authorize flow, and community evidence confirms GitHub integration works well for many users. However, there's no explicit documentation of a literal 'one-click install' onboarding flow or app installation process described in the evidence pack. missing for 10: explicit one-click GitHub App install/authorization flow documentation, independent confirmation of setup simplicity.

   • [claimed-docs] “Jules needs access to your repositories in order to work.”
   • [claimed-docs] “Use the "jules" label in an issue to assign a task directly in GitHub.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [community] “I used Jules three times today, very impressive! It also handles coding-adjacent work. Good github integrations.”
   • [community] “its way better than the github thing in my experience it produces usable PRs”

   proof ↗⚑ flag

Ci remediation

1. engineering-leadHave failed CI workflows automatically diagnosed and fixed with a proposed pull request

   weight 3 · round to Devin
   Devin✓fullclaimed7/10

   Devin explicitly documents 'Devin Review with Auto-Fix' that responds to code review comments, fixes flagged bugs, and iterates on CI failures, directly matching the story of auto-diagnosing and fixing failed CI with a proposed PR. However, this is vendor-only documentation with no independent/hands-on corroboration of the CI-fix workflow specifically. Missing for 10: independent verification of CI-fix accuracy/success rate, details on PR proposal format, and evidence of real-world reliability beyond the single docs line.

   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures”

   proof ↗⚑ flag

   Jules~partialclaimed3/10

   Jules offers building blocks—an API for custom automation that can 'automate tasks like bug fixing' and auto-create PRs via automationMode—but there is no direct evidence of a native integration that detects a failed CI workflow and automatically diagnoses/fixes it with a PR; this would require custom API wiring, not an out-of-box feature. Missing for 10: documented CI-failure trigger/integration, evidence of automatic diagnosis of CI logs, and case studies of this specific end-to-end flow.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [claimed-docs] “Use the "jules" label in an issue to assign a task directly in GitHub.”

   proof ↗⚑ flag

2. developerTrigger an agent from CI/CD pipelines to fix a broken build or failing test

   weight 2 · round to Devin
   Devin~partialclaimed6/10

   Docs explicitly state Devin Review with Auto-Fix can iterate on CI failures and fix flagged bugs, and the Devin API lets you integrate Devin into applications/workflows and create sessions programmatically (including on behalf of users), which supports triggering it from automated pipelines. However, there's no explicit documented example of a CI/CD pipeline (e.g., GitHub Actions/Jenkins) directly invoking Devin on a failing build/test, and no independent/hands-on confirmation of this exact workflow. Missing for 10: explicit CI/CD trigger integration example, independent verification that pipeline-triggered fixes work reliably.

   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”

   proof ↗⚑ flag

   Jules~partialcommunity5/10

   Jules exposes a documented API for creating custom workflows and automating tasks like bug fixing (jules-docs-9, jules-docs-10, jules-docs-12), which could be wired into a CI/CD pipeline to trigger a fix, and community evidence shows users building custom integrations to dispatch Jules tasks programmatically (jules-comm-19). However, there is no first-party CI/CD-specific integration (e.g. GitHub Actions step, build-failure webhook) or documented example of triggering Jules to fix a broken build/failing test directly from a pipeline. missing for 10: explicit CI/CD pipeline integration/example, evidence of triggering on build/test failure events, hands-on confirmation of this specific workflow succeeding.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”
   • [claimed-docs] “To send a message to the agent:”
   • [community] “Was able to build a personal MCP server that connects to the Jules API, letting me dispatch tasks to Jules, from Copilot Chat in VS Code.”

   proof ↗⚑ flag

Diff review

1. developerConfigure an agent to automatically open a pull request when its task completes

   weight 2 · round to Jules
   Devin~partialclaimed5/10

   Devin is known to work within GitHub workflows and create PRs as part of its task completion (implied by its coding-agent nature and features like Auto-Fix responding to review comments/CI), but the evidence pack has no explicit documentation describing a configurable setting to auto-open a pull request upon task completion. missing for 10: explicit docs describing PR-creation-on-completion configuration, settings/toggle for auto-PR behavior, and independent confirmation of this specific workflow.

   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures”
   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Delegating a more complex task via the web application and taking over in Devin’s IDE once it gives you a good first draft.”

   proof ↗⚑ flag

   Jules✓fullcommunity8/10

   Jules natively creates a PR when a task completes (jules-docs-8), and the API exposes an `automationMode` field to configure automatic PR creation, defaulting to off unless explicitly configured (jules-docs-11), directly matching the story of configuring automatic PR-on-completion. Community reports corroborate that Jules produces usable PRs and users regularly review PRs it opens (jules-comm-6, jules-comm-12, jules-comm-16). missing for 10: independent hands-on verification of the automationMode toggle specifically, and more detail on configuration options/edge cases (e.g., partial completions, failed tasks) affecting whether a PR is always opened.

   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”
   • [community] “its way better than the github thing in my experience it produces usable PRs”
   • [community] “I've been actually kind-of enjoying using Jules as a way of 'coding' my side project using my phone. By the time I get home in the evening I…”
   • [community] “I've moved infrastructure to Railway and taught the customer to use Jules. They make their own PRs now, Railway spins up an environment with…”

   proof ↗⚑ flag

2. developerReview a diff of an agent's changes and approve it before it becomes a pull request

   weight 3 · round to Jules
   Devin~partialclaimed4/10

   Docs mention delegating a task and 'taking over in Devin's IDE once it gives you a good first draft' (devin-docs-5), implying some human checkpoint before final output, and Devin's terminal/log viewing lets you watch work in progress (devin-docs-6). However there is no explicit documentation of a diff-review/approve-before-PR gate — 'Devin Review with Auto-Fix' (devin-docs-20) is about post-PR review comment handling, not a pre-PR approval step. Missing for 10: explicit diff viewer with approve/reject before PR creation, documentation of a formal approval gate, and independent/hands-on confirmation of this workflow.

   • [claimed-docs] “Delegating a more complex task via the web application and taking over in Devin’s IDE once it gives you a good first draft.”
   • [claimed-docs] “Devin’s terminal, where you can watch commands being executed and view output logs. You can also copy the shell output for debugging purpose…”
   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures”

   proof ↗⚑ flag

   Jules✓fullcommunity9/10

   Jules explicitly provides a diff of changes for review and approval before it creates/publishes a PR, with plan approval and diff approval steps documented; community mentions confirm PRs are generated for review. Missing for 10: independent hands-on confirmation of the diff-approval UI flow itself (comments focus on PR quality/output rather than the diff-review step specifically).

   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [community] “I've been actually kind-of enjoying using Jules as a way of 'coding' my side project using my phone. By the time I get home in the evening I…”

   proof ↗⚑ flag

Pr review automation

1. ai-native userHave incoming issues automatically triaged with severity suggested and routed to the right owner

   weight 2 · round drawn
   Devin—none0/10

   Devin can be pointed at Linear/Jira tickets or tagged in Slack to work a bug, but nothing in the evidence describes automatic triage logic that assesses severity or routes issues to the correct owner — these are core, specific parts of the story. Missing for 10: severity classification/scoring, ownership/routing logic, any 'triage' feature or automated inbox-sorting workflow.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Tagging Devin on a Slack or Teams thread about a bug you’re discussing with coworkers”
   • [claimed-docs] “Tag Devin on Slack or Teams: Start sessions directly from conversations about bugs, feature requests, or questions.”

   proof ↗⚑ flag

   Jules—none0/10

   Jules is a coding agent that executes tasks assigned via label or API, but there is no evidence of automatic issue triage, severity classification, or routing to owners — the closest feature is manual label-based task assignment, not automated triage. Missing for 10: any mention of severity scoring, triage logic, or owner-routing automation.

   • [claimed-docs] “Use the "jules" label in an issue to assign a task directly in GitHub.”
   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”

   proof ↗⚑ flag

2. engineering-leadHave every pull request automatically reviewed with AI-generated inline comments

   weight 3 · round to Devin
   Devin~partialclaimed5/10

   Devin documents a 'Devin Review with Auto-Fix' feature that responds to code review comments, fixes flagged bugs, and iterates on CI failures, implying an automated review capability tied to PRs, but the evidence pack gives no detail on how review comments are generated, whether they are inline, or how coverage/setup works for 'every PR'. missing for 10: explicit documentation of inline AI-generated PR comments, configuration for gating every PR, and independent/hands-on confirmation of the review quality.

   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures”

   proof ↗⚑ flag

   Jules—none0/10

   Jules is documented as a task-execution agent that clones repos, makes code changes, and creates its own PRs for approval — not as a bot that automatically reviews incoming pull requests with inline comments. The only tangential mention is jules-docs-9's passing reference to 'automate tasks like ... code reviews' via API, but there is no documentation of an automatic inline-comment review gate applied to every PR.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”

   proof ↗⚑ flag

Readiness checks

1. engineering-leadAutomatically fix failing agent-readiness criteria in my repository

   weight 1 · round to Devin
   Devin~partialclaimed6/10

   Devin Review with Auto-Fix explicitly auto-fixes CI failures and review comments, which could cover agent-readiness gate checks if they're wired into CI/PR review, and AGENTS.md support shows Devin can consume agent-readiness config files. However there's no evidence of a dedicated 'agent-readiness criteria' scorecard or targeted remediation workflow distinct from generic CI/review auto-fix. Missing for 10: explicit agent-readiness scoring/criteria detection, evidence of automatically remediating specific agent-readiness failures (vs generic CI/review comments), and independent validation of this auto-fix loop working reliably.

   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures”
   • [claimed-docs] “Just put an AGENTS.md file in your project root (or anywhere else). Devin will look for the file before it starts coding.”
   • [claimed-docs] “Configure it once, and every session boots into that known-good state.”

   proof ↗⚑ flag

   Jules—none0/10

   Jules can be assigned generic bug-fix/coding tasks and reads an AGENTS.md file if present, but there is no evidence of it detecting or automatically remediating 'agent-readiness' quality-gate criteria (e.g., missing/invalid AGENTS.md, agent-compatibility checks) as a review gate — it only consumes such files, it doesn't audit or fix them as a compliance gate.

   • [claimed-docs] “Jules now automatically looks for a file named AGENTS.md in the root of your repository.”
   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [community] “I gave Jules a try on a new, very unorganized Python project. It failed the first time, I gave it the error message and it fixed it. I was s…”

   proof ↗⚑ flag

2. engineering-leadRun a readiness report that evaluates how ready my repository is for autonomous agents

   weight 2 · round drawn
   Devin—none0/10

   No evidence of any 'readiness report' or scoring feature that evaluates a repository's suitability for autonomous agents; docs cover onboarding features like AGENTS.md, environment setup, indexing, and knowledge, but nothing that generates a readiness assessment/report artifact for engineering leads.

   ⚑ flag

   Jules—none0/10

   No evidence of a readiness report, scorecard, or repository-assessment feature for autonomous agent suitability; evidence only covers task execution, PR creation, and API usage. Missing for 10: any readiness-scoring feature, repo audit/checklist, or report output evaluating agent-readiness.

   ⚑ flag

Security remediation

1. engineering-leadHave security alerts automatically validated and remediated with an opened pull request

   weight 2 · round to Devin
   Devin~partialclaimed5/10

   Devin's 'Review with Auto-Fix' auto-responds to code review comments, fixes flagged bugs and iterates on CI failures, and MCP integrations extend it to security-adjacent tools like Sentry and Datadog, implying it could react to alerts and open a PR fix. However, there is no explicit documentation of a security-scanner (SAST/dependency-alert) validation-and-remediation pipeline or evidence of an actual security-alert-to-PR workflow. Missing for 10: explicit security-scanner (e.g., CodeQL/Dependabot/Snyk) integration, evidence of validating true-positive vs false-positive alerts, and a documented end-to-end 'alert triggers PR' example.

   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures”
   • [claimed-docs] “Extend Devin's reach with MCP integrations: Connect Devin to Datadog, Sentry, databases, Figma, Notion, Stripe, and hundreds of other tool”

   proof ↗⚑ flag

   Jules—none0/10

   Jules is a general coding agent that can be assigned tasks (via GitHub label or API) and opens PRs with diffs for review, but there is no evidence of any security-alert scanning, vulnerability detection/validation, or a workflow that automatically triggers remediation from a security alert (e.g., Dependabot/CVE integration). The evidence pack shows generic task-to-PR flow, not a security-alert-specific pipeline.

   • [claimed-docs] “Jules needs access to your repositories in order to work.”
   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [claimed-docs] “Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.”
   • [claimed-docs] “Jules provides a diff of the changes. Quickly browse and approve code edits.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”

   proof ↗⚑ flag

Concurrent execution

1. engineering-leadRun many agent tasks concurrently to scale delivery throughput

   weight 3 · round to Devin
   Devin~partialclaimed7/10

   Devin docs explicitly document running multiple Devins in parallel on independent tasks and provide an API with create_as_user_id for programmatically spawning sessions on behalf of org members, directly supporting concurrent scaling of throughput. However, there's no first-party data on concurrency limits, cost/resource management at scale, or independent case studies validating throughput gains at team scale; missing for 10: quantified concurrency limits/scaling guidance, independent evidence of large-scale parallel usage, and cost/governance controls for many simultaneous sessions.

   • [claimed-docs] “Run multiple Devins in parallel: Carve out independent tasks and run them simultaneously.”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “create sessions, manage playbooks and knowledge, set up schedules, and more”

   proof ↗⚑ flag

   Jules~partialcommunity6/10

   Jules architecture (cloud VMs per task, API for creating tasks, GitHub issue labeling) supports dispatching many tasks in parallel, and usage-limit docs explicitly reference 'agent-heavy workflows' with numeric caps (e.g. 300/60), and community reports (jules-comm-11, jules-comm-16) describe handling large task volumes successfully. However, other evidence shows real caveats: daily task limits were cut from 60 to 15 on the free plan (jules-comm-8), tasks can get stuck in loops with no stop button (jules-comm-14), and users report needing heavy babysitting (jules-comm-18), undercutting smooth high-throughput scaling. Missing for 10: dedicated documentation of a concurrency/queue dashboard, enterprise-tier concurrency guarantees, and independent benchmarks confirming reliable parallel execution at scale.

   • [claimed-docs] “Power users & agent-heavy workflows ... 300 ... 60”
   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [community] “I've used this tool for a few months now and have been pretty impressed by it. It handles large quantities of tasks very well and is good at…”
   • [community] “I've moved infrastructure to Railway and taught the customer to use Jules. They make their own PRs now, Railway spins up an environment with…”
   • [community] “The daily task limit went down from 60 to 15 (on the free plan) with this release. Personally I wasn't close to exhausting the limit because…”
   • [community] “Jules quickly after a few messages/prompts just gets stuck in an endless loop like Gemini-cli does. The worst part is that there is no 'STOP…”
   • [community] “Do people really find Jules useful? I find it needs babysitting much more than Cursor.”

   proof ↗⚑ flag

2. engineering-leadCreate agent sessions on behalf of other users in my organization

   weight 2 · round to Devin
   Devin✓fullclaimed8/10

   Devin's API explicitly supports creating sessions on behalf of any user in the organization via the create_as_user_id parameter, directly matching the engineering-lead use case for org-wide session delegation. Missing for 10: independent/hands-on corroboration of this specific feature and details on permission scoping/audit controls for delegated sessions.

   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”

   proof ↗⚑ flag

   Jules—none0/10

   No evidence of any organization, team, or admin/delegation features allowing an engineering-lead to create sessions on behalf of other users; Jules docs only describe individual API keys, per-user GitHub connections, and personal task creation. Missing for 10: org/team management, delegated session creation, role-based admin controls, multi-user account provisioning.

   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “Jules needs access to your repositories in order to work.”

   proof ↗⚑ flag

Deployment flexibility

1. developerUse a managed cloud offering to run agents without operating my own backend infrastructure

   weight 2 · round to Jules
   Devin✓fullclaimed7/10

   Devin is delivered as a cloud-hosted agent (web app, API, session creation on behalf of users) so developers don't need to run their own backend/agent infrastructure — sessions run on Devin's managed cloud with its own configured environment, VPN access, and parallel execution. missing for 10: independent/hands-on confirmation of the managed infrastructure model, and explicit documentation of uptime/scaling/ops guarantees for the hosted service.

   • [claimed-docs] “using /handoff to send longer tasks to cloud Devin”
   • [claimed-docs] “Delegating a more complex task via the web application and taking over in Devin’s IDE once it gives you a good first draft.”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “Run multiple Devins in parallel: Carve out independent tasks and run them simultaneously.”
   • [claimed-docs] “Devin can connect to a VPN from inside its workspace, so sessions can reach internal services such as package registries, databases, and int…”
   • [claimed-docs] “Configure it once, and every session boots into that known-good state.”

   proof ↗⚑ flag

   Jules✓fullcommunity8/10

   Jules is explicitly a cloud-hosted agent service: tasks run in Google-managed VMs, integrated with GitHub, with an API and usage limits/plans, requiring no self-hosted backend. Community evidence corroborates real-world usage at scale (75% success on customer PRs, teams dispatching many tasks) though some users report reliability/UI issues. Missing for 10: no independent infra/SLA details, no discoverable OpenAPI/docs.md confirming API completeness, and mixed community reliability reports.

   • [claimed-docs] “Jules needs access to your repositories in order to work.”
   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [claimed-docs] “Jules creates a PR of the changes. Approve the PR, merge it to your branch, and publish it on GitHub.”
   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “Power users & agent-heavy workflows ... 300 ... 60”
   • [community] “I've moved infrastructure to Railway and taught the customer to use Jules. They make their own PRs now, Railway spins up an environment with…”
   • [community] “Was able to build a personal MCP server that connects to the Jules API, letting me dispatch tasks to Jules, from Copilot Chat in VS Code.”

   proof ↗⚑ flag

2. engineering-leadSelf-host agent infrastructure locally, in containers, or on my own VMs

   weight 2 · round drawn
   Devin—none0/10

   Devin is delivered as a cloud-hosted SaaS agent platform; all evidence points to cloud sessions, API access, VPN connectivity into customer networks, and a CLI/terminal client for local interaction, but nothing indicates the core agent infrastructure itself can be self-hosted on-prem, in customer-managed containers, or on customer VMs. Missing for 10: any documentation of a self-hosted/on-prem deployment option, container images for the agent runtime, or VM-based installation of Devin's backend.

   • [claimed-docs] “Devin can connect to a VPN from inside its workspace, so sessions can reach internal services such as package registries, databases, and int…”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “Running Devin for Terminal in your local environment for quick fixes, code exploration, and interactive coding right from the command line”

   proof ↗⚑ flag

   Jules—none0/10

   Jules is explicitly cloud-hosted: it runs tasks in Google's own VMs, and community feedback confirms it is not local (jules-comm-1). There is no evidence of any self-hosting, on-prem, container, or private-VM deployment option.

   • [claimed-docs] “Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.”
   • [community] “Jules runs in cloud-based VMs instead of on my local machine, making it much less useful than Claude Code. My projects have bespoke build sc…”

   proof ↗⚑ flag

Headless automation

1. developerRun an agent headlessly inside CI/CD pipelines and shell scripts

   weight 2 · round to Devin
   Devin~partialprobed6/10

   Devin's API is explicitly designed to 'integrate Devin into your applications, automate workflows, and build powerful tools' and supports creating sessions on behalf of users, which enables scripted/headless invocation, and a CLI is also documented. However, there is no explicit CI/CD pipeline example, exit-code/status handling, or GitHub Actions/shell-script integration guide in the evidence — the closest CI-related capability shown is 'Auto-Fix' reacting to CI failures rather than Devin running as a CI step itself. missing for 10: explicit CI/CD pipeline integration docs (e.g., GitHub Actions/shell script examples), documented headless/non-interactive CLI invocation with exit codes.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures”
   • [probe] “official CLI documented at https://docs.devin.ai/cli/adaptive”

   proof ↗⚑ flag

   Jules~partialcommunity5/10

   Jules exposes a public API (create tasks, send messages, optional automationMode) explicitly pitched for 'custom workflows' and embedding into other tools, which can be scripted/curled headlessly, and a community member built an MCP server hitting the API to dispatch tasks programmatically. However there's no documented CLI, no explicit CI/CD pipeline examples (e.g., GitHub Actions), and Jules always runs its own cloud VM rather than a lightweight headless process invokable inline in a shell script/pipeline step. missing for 10: official CLI or CI/CD pipeline integration docs (e.g. GitHub Actions step), examples of shell-script invocation, confirmation that API calls run synchronously enough for pipeline gating.

   • [claimed-docs] “You can use the API to create custom workflows, automate tasks like bug fixing and code reviews, and embed Jules's intelligence directly int…”
   • [claimed-docs] “In the Jules web app, go to the Settings page to create a new API key. You can have at most 3 API keys at a time.”
   • [claimed-docs] “The `automationMode` field is optional. By default, no PR will be automatically created.”
   • [claimed-docs] “To send a message to the agent:”
   • [community] “Was able to build a personal MCP server that connects to the Jules API, letting me dispatch tasks to Jules, from Copilot Chat in VS Code.”
   • [community] “I've moved infrastructure to Railway and taught the customer to use Jules. They make their own PRs now, Railway spins up an environment with…”

   proof ↗⚑ flag