Gemini CLI vs Devin

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to Devin
   Gemini CLI—none0/10

   No evidence Gemini CLI has any documented feature for consuming llms.txt or agent-oriented doc manifests; the only related probe shows llms.txt returning 404 on Google's own docs site, and none of the GitHub feature list or docs mention llms.txt support. GEMINI.md context files are a different, project-local mechanism, not agent-oriented web docs discovery.

   • [probe] “PROBE llms.txt: HTTP 404 at https://developers.google.com/llms.txt”
   • [github] “Custom context files (GEMINI.md) to tailor behavior for your projects”

   proof ↗⚑ flag

   Devin✓fullprobed9/10

   Devin's own docs site serves llms.txt (HTTP 200, confirmed by probe) and per-page .md variants, and also supports AGENTS.md as an agent-oriented instructions standard, directly matching the story. Missing for 10: no independent/community confirmation that external agents have actually consumed llms.txt successfully.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.devin.ai/llms.txt # Devin Docs - [Desktop (100 pages)](https://docs.devin.ai/_llms/en/desktop.md):…”
   • [probe] “PROBE docs-md: HTTP 200 at https://docs.devin.ai/get-started/devin-intro.md > ## Documentation Index > Fetch the complete documentation inde…”
   • [claimed-docs] “Devin supports AGENTS.md - a simple, open standard for providing context and instructions to AI agents.”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to Gemini CLI
   Gemini CLI✓fullclaimed9/10

   Gemini CLI explicitly documents non-interactive scripting mode, structured/streaming JSON output flags for programmatic parsing, and GitHub Actions-based automation (PR reviews, issue triage, on-demand assistance), which together cover headless/CI use cases well. Missing for 10: independent hands-on confirmation specifically of CI pipeline reliability (community evidence focuses more on interactive agentic quality than CI usage).

   • [github] “Run non-interactively in scripts for workflow automation”
   • [github] “use the `--output-format json` flag to get structured output”
   • [github] “use `--output-format stream-json` to get newline-delimited JSON events”
   • [github] “Pull Request Reviews: Automated code review with contextual feedback and suggestions”
   • [github] “Issue Triage: Automated labeling and prioritization of GitHub issues based on content analysis”
   • [github] “On-demand Assistance: Mention @gemini-cli in issues and pull requests for help with debugging, explanations, or task delegation”

   proof ↗⚑ flag

   Devin✓fullclaimed8/10

   Devin offers a full API for creating/managing sessions programmatically (including create_as_user_id for automation on behalf of users), a CLI with a --sandbox flag for OS-level isolated headless runs, and explicit CI/CD pipeline integration for responding to static analysis findings and PR checks, all supporting headless/automated usage without a human in the loop. Missing for 10: no independent/hands-on report specifically validating CI automation workflows end-to-end, and no explicit CI example (e.g., GitHub Actions snippet) beyond doc references.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “The --sandbox flag runs the CLI with OS-level isolation, enforcing writable paths and deny rules at the operating-system level and optionall…”
   • [claimed-docs] “a local command-line coding agent with deep Devin Cloud integration”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round to Gemini CLI
   Gemini CLI✓fullclaimed8/10

   Gemini CLI documents first-party MCP server support: configuring servers in ~/.gemini/settings.json to add custom tools, a dedicated /mcp command, and explicit mention of connecting media-generation tools like Imagen/Veo/Lyria via MCP. This is corroborated by official docs listing /mcp among CLI commands. Missing for 10: independent hands-on verification of MCP tool usage specifically (community evidence covers general agentic reliability but not MCP integration itself), and more detail on server management/discovery UX.

   • [github] “Configure MCP servers in ~/.gemini/settings.json to extend Gemini CLI with custom tools”
   • [github] “Use MCP servers to connect new capabilities, including media generation with Imagen, Veo or Lyria”
   • [claimed-docs] “Comandos de Gemini CLI: /memory, /stats, /tools y /mcp”

   proof ↗⚑ flag

   Devin—none0/10

   The evidence only documents Devin exposing its own MCP server so other agents/IDEs can call Devin's tools (session management, playbooks, knowledge, scheduling) — the reverse direction of this story. There is no evidence that a user can configure Devin itself to consume/plug in external MCP servers so Devin can use their tools.

   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [probe] “official MCP server documented at https://docs.devin.ai/work-with-devin/devin-mcp”

   proof ↗⚑ flag

4. ai-native userUse an official CLI

   weight 2 · round to Gemini CLI
   Gemini CLI✓fullprobed9/10

   Gemini CLI is itself an official, first-party CLI product by Google with extensive documentation of its features (scripting, JSON output, MCP support, context files, non-interactive mode) and independent corroboration of active use, confirming it exists and functions as an official CLI tool for AI-native workflows. missing for 10: no fully independent third-party audit of CLI completeness beyond community anecdotes.

   • [github] “Run non-interactively in scripts for workflow automation”
   • [github] “gemini --include-directories ../lib,../docs”
   • [github] “use the `--output-format json` flag to get structured output”
   • [github] “use `--output-format stream-json` to get newline-delimited JSON events”
   • [github] “Configure MCP servers in ~/.gemini/settings.json to extend Gemini CLI with custom tools”
   • [claimed-docs] “The Gemini CLI is available without additional setup in Cloud Shell”
   • [community] “I have been using this for about a month and it's a beast, mostly thanks to 2.5pro being SOTA and how it leverages that huge 1M context wind…”
   • [probe] “official CLI documented at https://developers.google.com/gemini-code-assist/docs/gemini-cli”

   proof ↗⚑ flag

   Devin✓fullprobed8/10

   Devin has an official CLI ('Devin CLI, a local command-line coding agent with deep Devin Cloud integration') with documented usage examples and a sandbox flag for OS-level isolation, confirmed by both docs and probe. Missing for 10: independent/hands-on community verification of the CLI specifically (community evidence covers the web/session product, not CLI usage).

   • [claimed-docs] “Devin CLI, a local command-line coding agent with deep Devin Cloud integration.”
   • [claimed-docs] “devin -- check out this code and suggest a feasible, helpful feature”
   • [claimed-docs] “The --sandbox flag runs the CLI with OS-level isolation, enforcing writable paths and deny rules at the operating-system level and optionall…”
   • [claimed-docs] “a local command-line coding agent with deep Devin Cloud integration”
   • [probe] “official CLI documented at https://docs.devin.ai/cli/index”

   proof ↗⚑ flag

5. ai-native userDrive the product through a documented public API

   weight 3 · round to Devin
   Gemini CLI~partialprobed5/10

   Gemini CLI documents CLI-level automation hooks — non-interactive scripting mode, `--output-format json`/`stream-json` for structured output, and MCP server configuration — which let an AI-native user drive it programmatically (gemini-cli-gh-6, gh-17, gh-18, gh-19). However, explicit probes for a formal public API/SDK (llms.txt, openapi.json) all returned 404, showing no dedicated documented API surface beyond the CLI itself. Missing for 10: a first-party REST/SDK API spec, official API reference docs, and independent confirmation of programmatic (non-CLI) usage.

   • [github] “Run non-interactively in scripts for workflow automation”
   • [github] “use the `--output-format json` flag to get structured output”
   • [github] “use `--output-format stream-json` to get newline-delimited JSON events”
   • [github] “Configure MCP servers in ~/.gemini/settings.json to extend Gemini CLI with custom tools”
   • [probe] “PROBE llms.txt: HTTP 404 at https://developers.google.com/llms.txt”
   • [probe] “PROBE openapi: all candidate paths 404 (https://developers.google.com/openapi.json, https://developers.google.com/swagger.json, https://deve…”

   proof ↗⚑ flag

   Devin✓fullprobed8/10

   Devin ships a documented public API (docs-7) with session creation, org-level features like create_as_user_id (docs-8), plus a CLI and MCP server for programmatic/agentic control (docs-4, docs-6, probe-4, probe-5), directly enabling AI-native users to drive it programmatically. Missing for 10: a discoverable OpenAPI/swagger spec (probe-3 found 404s on all candidate paths) and independent hands-on corroboration of API usage.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “Devin CLI, a local command-line coding agent with deep Devin Cloud integration.”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [probe] “official MCP server documented at https://docs.devin.ai/work-with-devin/devin-mcp”
   • [probe] “official CLI documented at https://docs.devin.ai/cli/index”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

6. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round drawn
   Gemini CLI—none0/10

   Evidence shows Gemini CLI abstracts away API key management entirely (sign in with Google account) rather than offering scoped or least-privilege credential issuance for agents; no docs mention credential scoping, permission boundaries, or token minting for agent use.

   • [github] “No API key management - just sign in with your Google account”

   proof ↗⚑ flag

   Devin—none0/10

   Devin exposes a general API (devin-docs-7) and can act on behalf of a specified user via create_as_user_id (devin-docs-8), but there is no evidence of scoped/least-privilege API key or token issuance, role-based permission scopes, or credential-level restriction mechanisms for agent access.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”

   proof ↗⚑ flag

7. ai-native userBuild against official SDKs

   weight 2 · round to Devin
   Gemini CLI—none0/10

   The evidence pack documents CLI flags, MCP server extensibility, scripting output formats, and GitHub Actions integration, but contains no mention of an official SDK (e.g., a Node/Python/Go library) for programmatically building on Gemini CLI itself. Probes for API/OpenAPI specs also returned 404s, reinforcing the absence of such artifacts.

   • [github] “Configure MCP servers in ~/.gemini/settings.json to extend Gemini CLI with custom tools”
   • [github] “use the `--output-format json` flag to get structured output”
   • [github] “use `--output-format stream-json` to get newline-delimited JSON events”
   • [probe] “PROBE llms.txt: HTTP 404 at https://developers.google.com/llms.txt”
   • [probe] “PROBE openapi: all candidate paths 404 (https://developers.google.com/openapi.json, https://developers.google.com/swagger.json, https://deve…”

   proof ↗⚑ flag

   Devin~partialprobed4/10

   Devin provides an API for integration (docs-7, docs-8) allowing developers to build applications and automate workflows, but the evidence never mentions dedicated official SDKs/client libraries (e.g., Python/JS packages), and probes for an OpenAPI spec that would back SDK generation all returned 404s (devin-probe-3). Missing for 10: named SDK packages in specific languages, SDK installation/usage docs, and a published OpenAPI/schema artifact confirming SDK-generation support.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

8. ai-native userSubscribe to events via webhooks

   weight 2 · round drawn
   Gemini CLI—none0/10

   No evidence of webhook subscription or event-push capability; Gemini CLI supports non-interactive scripting, MCP tool servers, and structured JSON output, but nothing about outbound webhooks or event subscriptions. Missing for 10: any webhook registration mechanism, event subscription API, or documentation of push notifications.

   • [github] “Run non-interactively in scripts for workflow automation”
   • [github] “use the `--output-format json` flag to get structured output”
   • [github] “use `--output-format stream-json` to get newline-delimited JSON events”
   • [github] “Configure MCP servers in ~/.gemini/settings.json to extend Gemini CLI with custom tools”

   proof ↗⚑ flag

   Devin—none0/10

   No evidence in the pack mentions webhooks or event subscription mechanisms; Devin's API/MCP docs describe session creation and management but nothing about outbound webhook notifications for events.

   ⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round to Devin
   Gemini CLI!disputedcontradicted5/10

   Gemini CLI ships features that clearly aim at generating insights from a user's own data/codebase — querying and editing large codebases, natural-language debugging, automated PR review with contextual feedback, and issue triage (gemini-cli-gh-1, gh-3, gh-10, gh-11), and one community report praises its code review as catching bugs missed by humans (gemini-cli-comm-20). However, multiple hands-on reports directly contradict this, describing it as 'terrible at agentic stuff', getting stuck in loops, failing to edit/read files, and being 'useless as a coding assistant' that produces spaghetti code (gemini-cli-comm-10, comm-14, comm-15). missing for 10: independent benchmark confirming consistent quality of generated insights, resolution of the loop/failure reports, and evidence the insight-generation works reliably across data types beyond code.

   • [github] “Query and edit large codebases”
   • [github] “Debug issues and troubleshoot with natural language”
   • [github] “Pull Request Reviews: Automated code review with contextual feedback and suggestions”
   • [github] “Issue Triage: Automated labeling and prioritization of GitHub issues based on content analysis”
   • [community] “We have tried out Gemini code review vs Copilot code review and Gemini is consistently offering better code review tips. It has officially c…”
   • [community] “A lot of times Gemini models will get stuck in a loop of errors, and a lot of times it fails to edit/read or other simple function calling -…”
   • [community] “The problem is that Gemini CLI simply doesn't work. Beside simplest tasks like creating a new release it is useless as a coding assistant. D…”
   • [community] “I love the model, hate the tool. Anthropic has the killer app with Claude Code. I tried Gemini cli for about 5 seconds and was so frustrated…”

   proof ↗⚑ flag

   Devin✓fullclaimed7/10

   Devin generates insights/suggestions from a user's own codebase data via 'Ask Devin' (code structure/dependency Q&A), auto-generated DeepWiki documentation, and Devin Review's automated PR feedback, all built on repository indexing. Missing for 10: independent/hands-on validation of the quality of these AI-generated insights (community evidence covers general task execution issues, not this specific feature) and no benchmark of insight accuracy.

   • [claimed-docs] “Ask Devin can answer questions about code structure and dependencies, and help you scope and plan tasks before implementation.”
   • [claimed-docs] “Devin Review provides automated first-pass reviews on pull requests, checking for correctness and conformance with organizational best pract…”
   • [claimed-docs] “Use DeepWiki to navigate architecture and code with auto-generated documentation.”
   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”

   proof ↗⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Devin
   Gemini CLI~partialcommunity6/10

   Gemini CLI documents non-interactive scripting mode and a GitHub Action integration that runs autonomously in the background (automated PR reviews, issue triage, on-demand @gemini-cli responses), which directly supports background automations. However, independent community reports describe agentic reliability problems (getting stuck in loops, failing simple file operations, ignoring GEMINI.md context) that undercut confidence in unattended/background runs actually completing correctly. Missing for 10: independent hands-on validation that scheduled/background automations run reliably end-to-end, and more detail on failure/retry handling in autonomous mode.

   • [github] “Run non-interactively in scripts for workflow automation”
   • [github] “Pull Request Reviews: Automated code review with contextual feedback and suggestions”
   • [github] “Issue Triage: Automated labeling and prioritization of GitHub issues based on content analysis”
   • [github] “On-demand Assistance: Mention @gemini-cli in issues and pull requests for help with debugging, explanations, or task delegation”
   • [github] “On-demand Assistance: Mention `@gemini-cli` in issues and pull requests for help with debugging, explanations, or task delegation”
   • [github] “@github List my open pull requests”
   • [community] “A lot of times Gemini models will get stuck in a loop of errors, and a lot of times it fails to edit/read or other simple function calling -…”
   • [community] “I really tried to get gemini to work properly in Agent mode. Tho it way too often went crazy, started rewriting files empty, and ran into pe…”
   • [community] “Tip 1, it consistently ignores my GEMINI.md file, both global and local, even though it always says '1 GEMINI.md file is being used.'”

   proof ↗⚑ flag

   Devin✓fullclaimed8/10

   Devin explicitly supports background/autonomous execution: cloud sessions run in their own VM and 'keep going after you close your laptop' (devin-docs-13), MCP access includes 'scheduling' (devin-docs-6/23), the API lets you 'automate workflows' and create sessions programmatically (devin-docs-7/8), CI/CD integration triggers Devin on findings (devin-docs-19), and Auto-Fix creates a closed loop that iterates PRs 'without you in the loop' (devin-docs-18). This spans scheduled triggers, API-driven automation, and hands-off background operation. Missing for 10: independent/community verification that scheduled automations run reliably unattended, and more detail on a dedicated 'automation/schedule' UI beyond scattered doc mentions.

   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”

   proof ↗⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round to Devin
   Gemini CLI!disputedcontradicted5/10

   Gemini CLI is itself billed as an agentic assistant with extensive task-delegation features (codebase queries, debugging, PR review/issue triage, operational automation via @gemini-cli mentions) per gemini-cli-gh-3/4/10/11/12/22. However, hands-on community reports concretely contradict reliable delegation: users report it is 'really really terrible at agentic stuff,' gets stuck in permanent loops, ignores GEMINI.md context, and in one case catastrophically deleted user files while apologizing for the failure.

   • [github] “Debug issues and troubleshoot with natural language”
   • [github] “Automate operational tasks like querying pull requests or handling complex rebases”
   • [github] “Pull Request Reviews: Automated code review with contextual feedback and suggestions”
   • [github] “Issue Triage: Automated labeling and prioritization of GitHub issues based on content analysis”
   • [github] “On-demand Assistance: Mention @gemini-cli in issues and pull requests for help with debugging, explanations, or task delegation”
   • [community] “A lot of times Gemini models will get stuck in a loop of errors, and a lot of times it fails to edit/read or other simple function calling -…”
   • [community] “I really tried to get gemini to work properly in Agent mode. Tho it way too often went crazy, started rewriting files empty, and ran into pe…”
   • [community] “Tip 1, it consistently ignores my GEMINI.md file, both global and local, even though it always says '1 GEMINI.md file is being used.'”
   • [community] “The problem is that Gemini CLI simply doesn't work. Beside simplest tasks like creating a new release it is useless as a coding assistant. D…”
   • [community] “Gemini told the user: 'I have failed you completely and catastrophically... I have lost your data. This is an unacceptable, irreversible fai…”

   proof ↗⚑ flag

   Devin~partialcommunity6/10

   Devin's core product is designed for task delegation — via Ask Devin, ticket assignment, Slack/Teams tagging, and a conversational IDE (devin-docs-1, devin-docs-3, devin-docs-24) — so the axis clearly applies and is well documented. However, hands-on community reports show real caveats: Devin can add extraneous unrequested changes it can't undo, gets stuck for long periods without asking for help, and requires active babysitting/session termination to get value (devin-comm-1, devin-comm-2, devin-comm-5), undercutting a fully seamless delegation experience. Missing for 10: independent verification that delegated tasks reliably complete without extraneous side-effects or getting stuck, and stronger corroboration beyond one HN thread.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Devin is designed to be a conversational user interface, and allows you to follow and take over Devin's development process in the embedded …”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round to Devin
   Gemini CLI!disputedcontradicted5/10

   Gemini CLI's entire premise is natural-language driven coding/agentic actions (querying codebases, debugging, automating PR/rebase tasks, custom GEMINI.md context) per gemini-cli-gh-1/3/4/9. However, multiple hands-on reports describe the NL-agent behavior failing badly in practice — getting stuck in error loops, botching file edits, ignoring GEMINI.md instructions, and in one case catastrophically deleting user data via misinterpreted commands.

   • [github] “Query and edit large codebases”
   • [github] “Debug issues and troubleshoot with natural language”
   • [github] “Automate operational tasks like querying pull requests or handling complex rebases”
   • [github] “Custom context files (GEMINI.md) to tailor behavior for your projects”
   • [community] “A lot of times Gemini models will get stuck in a loop of errors, and a lot of times it fails to edit/read or other simple function calling -…”
   • [community] “I really tried to get gemini to work properly in Agent mode. Tho it way too often went crazy, started rewriting files empty, and ran into pe…”
   • [community] “Tip 1, it consistently ignores my GEMINI.md file, both global and local, even though it always says '1 GEMINI.md file is being used.'”
   • [community] “The problem is that Gemini CLI simply doesn't work. Beside simplest tasks like creating a new release it is useless as a coding assistant. D…”
   • [community] “Gemini told the user: 'I have failed you completely and catastrophically... I have lost your data. This is an unacceptable, irreversible fai…”

   proof ↗⚑ flag

   Devin✓fullcommunity8/10

   Devin is explicitly designed as a conversational agent: users assign tasks via natural language (Slack/Teams tagging, chat interface, CLI prompts like 'devin -- check out this code...'), and it interprets these into autonomous coding/dev actions across IDE, CLI, and cloud sessions. Community evidence corroborates it operates on natural-language task descriptions in practice, though with noted friction around scope creep and knowing when to stop. Missing for 10: independent benchmarking of NL command accuracy/robustness and richer detail on how ambiguous instructions are resolved.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Tagging Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “Devin is designed to be a conversational user interface, and allows you to follow and take over Devin's development process in the embedded …”
   • [claimed-docs] “devin -- check out this code and suggest a feasible, helpful feature”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   Gemini CLI—none0/10

   No evidence of an interactive API reference with runnable examples; probes explicitly show no llms.txt or OpenAPI spec found, and no docs describe an interactive reference tool.

   • [probe] “PROBE llms.txt: HTTP 404 at https://developers.google.com/llms.txt”
   • [probe] “PROBE openapi: all candidate paths 404 (https://developers.google.com/openapi.json, https://developers.google.com/swagger.json, https://deve…”

   proof ↗⚑ flag

   Devin—none0/10

   Devin has an API reference overview but the openapi.json probe returned 404 on all candidate paths, and there's no mention of an interactive reference with runnable examples (e.g., 'try it' console) in the docs pack.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   Gemini CLI—none0/10

   Probes explicitly show no OpenAPI/llms.txt spec is published (404s at all candidate paths), and no other evidence mentions a machine-readable API spec for Gemini CLI.

   • [probe] “PROBE llms.txt: HTTP 404 at https://developers.google.com/llms.txt”
   • [probe] “PROBE openapi: all candidate paths 404 (https://developers.google.com/openapi.json, https://developers.google.com/swagger.json, https://deve…”

   proof ↗⚑ flag

   Devin—none0/10

   Devin has a documented API (devin-docs-7) but a direct probe for a machine-readable OpenAPI/swagger spec returned 404 on all candidate paths, and no documentation item references a downloadable spec file.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round to Devin
   Gemini CLI—none0/10

   No evidence pack items describe a sandbox/test environment isolated from production data for Gemini CLI; the only mention of 'sandbox' appears in an unrelated community comment describing another tool's architecture, not Gemini CLI's own testing environment. missing for 10: dedicated sandbox mode/documentation, evidence of isolation from production data, any hands-on confirmation of safe test environments.

   ⚑ flag

   Devin~partialclaimed6/10

   Devin sessions run in isolated cloud VMs with their own shell/browser/full repo access, configurable environment blueprints for a 'known-good state,' and a CLI --sandbox flag enforcing OS-level write/network isolation, all of which support testing in isolated environments away from live infrastructure. However, there is no explicit documentation addressing production-data isolation or masking, or confirmation that these sandboxes are guaranteed free of production data. missing for 10: explicit statement on production-data separation/masking, independent hands-on confirmation that sandbox testing never touches production data.

   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “Configure it once, and every session boots into that known-good state.”
   • [claimed-docs] “The --sandbox flag runs the CLI with OS-level isolation, enforcing writable paths and deny rules at the operating-system level and optionall…”
   • [claimed-docs] “The `--sandbox` flag runs the CLI with OS-level isolation, enforcing writable paths and `deny` rules at the operating-system level”
   • [claimed-docs] “Outposts lets you run Devin sessions inside infrastructure you control — your own VMs, containers, Kubernetes clusters, or even a Mac Mini o…”
   • [claimed-docs] “Devin inspects your repository, figures out which tools, runtimes, and dependencies are needed, and generates the blueprint for you.”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Gemini CLI—none0/10

   No evidence of versioned APIs or a documented deprecation policy for Gemini CLI; probes for llms.txt/openapi specs 404, and there is community evidence the tool itself was abruptly deprecated with no policy discussion (gemini-cli-comm-6/7/8), but no documentation of API versioning or deprecation commitments exists.

   • [probe] “PROBE llms.txt: HTTP 404 at https://developers.google.com/llms.txt”
   • [probe] “PROBE openapi: all candidate paths 404 (https://developers.google.com/openapi.json, https://developers.google.com/swagger.json, https://deve…”
   • [community] “Welcome to the Google graveyard, Gemini CLI. Not that it will be missed much. Using it was the worst experience out of any harness.”
   • [community] “Google really can't help themselves but to have some internal re-org kill off a public thing people are actively using. It's honestly impres…”

   proof ↗⚑ flag

   Devin—none0/10

   There's an API reference (devin-docs-7) and even an OpenAPI probe, but that probe found no OpenAPI spec (devin-probe-3), and no evidence anywhere mentions API versioning scheme or a documented deprecation policy for the API.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Gemini CLI
   Gemini CLI~partialcommunity6/10

   Gemini CLI supports scripting/non-interactive automation, multi-directory context inclusion, structured JSON output for pipelines, and GitHub Action integrations like automated issue triage (bulk labeling/prioritization) and PR review across a repo — all pointing to bulk/batch style operations. However there's no explicit documented 'batch process N files/items' feature or example, and community reports note the agent can get stuck in loops or fail simple multi-step tasks, raising doubts about reliability at scale. Missing for 10: explicit bulk-operation examples/documentation (e.g., batch renaming, mass refactor across many files) and independent evidence confirming reliable execution at scale.

   • [github] “Automate operational tasks like querying pull requests or handling complex rebases”
   • [github] “Run non-interactively in scripts for workflow automation”
   • [github] “Issue Triage: Automated labeling and prioritization of GitHub issues based on content analysis”
   • [github] “gemini --include-directories ../lib,../docs”
   • [github] “use the `--output-format json` flag to get structured output”
   • [github] “use `--output-format stream-json` to get newline-delimited JSON events”
   • [community] “A lot of times Gemini models will get stuck in a loop of errors, and a lot of times it fails to edit/read or other simple function calling -…”
   • [community] “I really tried to get gemini to work properly in Agent mode. Tho it way too often went crazy, started rewriting files empty, and ran into pe…”

   proof ↗⚑ flag

   Devin~partialcommunity5/10

   Devin's API supports programmatic session creation (including on behalf of other users) and docs explicitly encourage carving out independent tasks to run simultaneously, which together enable bulk-style automation across many tickets/items, but there is no dedicated 'bulk operations' or batch-processing feature documented, and community feedback raises concerns about reliability/oversight needed per session that would complicate true bulk workflows. Missing for 10: an explicit batch/bulk API endpoint or UI for processing many items in one request, and independent evidence of successful large-scale bulk runs.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”
   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to Devin
   Gemini CLI~partialclaimed5/10

   Gemini CLI ships GitHub Action integrations that fire automatically on repo events (PR opened → automated review, issue created → automated triage, @mention → on-demand help), which is a form of event-triggered automation, plus non-interactive/scripted execution for pipelines. However there's no evidence of a general-purpose, user-defined rule/trigger engine (e.g., custom webhooks, cron-like conditions, arbitrary event types) within the CLI itself—only fixed GitHub-event integrations. Missing for 10: a generic rule-definition mechanism for arbitrary events, documentation of custom trigger conditions, and independent confirmation these automations work reliably (community notes reliability issues with agentic behavior).

   • [github] “Pull Request Reviews: Automated code review with contextual feedback and suggestions”
   • [github] “Issue Triage: Automated labeling and prioritization of GitHub issues based on content analysis”
   • [github] “On-demand Assistance: Mention @gemini-cli in issues and pull requests for help with debugging, explanations, or task delegation”
   • [github] “On-demand Assistance: Mention `@gemini-cli` in issues and pull requests for help with debugging, explanations, or task delegation”
   • [github] “Run non-interactively in scripts for workflow automation”

   proof ↗⚑ flag

   Devin~partialclaimed6/10

   Devin supports several built-in event-triggered automations — Auto-Fix responds automatically to PR review comments and CI failures, CI/CD integration triggers Devin off static-analysis findings (SonarQube/Fortify/Veracode), and MCP exposes 'scheduling' as a capability — but there's no evidence of a general-purpose, user-defined rules/webhook engine for arbitrary custom triggers. Missing for 10: documentation of a configurable custom-rule/webhook trigger system, details on the scheduling feature's flexibility, and independent confirmation that these automations work reliably as event triggers.

   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixe”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round to Gemini CLI
   Gemini CLI~partialclaimed4/10

   Gemini CLI supports non-interactive scripted runs and structured JSON output, which lets users wire it into external schedulers (cron, CI) for recurring automation, and its GitHub Action integrations (issue triage, PR review) imply repeatable, trigger-based workflows. However there is no first-party 'scheduled job' or cron feature documented within the CLI itself. Missing for 10: a native recurring-job/scheduler feature, explicit docs on scheduling cadence, and independent confirmation that scripted/CI-triggered runs work reliably for recurring automation.

   • [github] “Run non-interactively in scripts for workflow automation”
   • [github] “use the `--output-format json` flag to get structured output”
   • [github] “Pull Request Reviews: Automated code review with contextual feedback and suggestions”
   • [github] “Issue Triage: Automated labeling and prioritization of GitHub issues based on content analysis”

   proof ↗⚑ flag

   Devin~partialclaimed3/10

   Devin's MCP docs mention 'scheduling' as one of the capabilities exposed to MCP-compatible agents, implying some scheduling functionality exists, but there is no dedicated documentation, UI, or examples describing recurring jobs, cron-like triggers, or workflow automation configuration. missing for 10: dedicated scheduling feature docs, examples of recurring/cron jobs, independent confirmation of scheduled workflows in practice.

   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”

   proof ↗⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round to Gemini CLI
   Gemini CLI~partialclaimed4/10

   Gemini CLI offers conversation checkpointing to save and resume sessions (gemini-cli-gh-8), which provides a rudimentary rollback/resume mechanism, but there is no evidence of versioning, diffing, or reviewing automation scripts/workflows themselves, nor a dedicated rollback command for automations. missing for 10: explicit version history for automations, review/diff tooling, and a documented rollback mechanism beyond session checkpoints.

   • [github] “Conversation checkpointing to save and resume complex sessions”

   proof ↗⚑ flag

   Devin—none0/10

   The evidence shows Devin has knowledge, playbooks, and scheduling features but nothing about versioning, reviewing, or rolling back those automation configurations themselves; Devin Review/Auto-Fix pertains to PR code review, not to the automation definitions. Missing for 10: version history for playbooks/knowledge, a review workflow for automation changes, and a rollback mechanism for automations.

   ⚑ flag

Background execution

1. ai-native userHave a cloud agent build, test, and demo a feature end-to-end for my review

   weight 2 · round to Devin
   Gemini CLI~partialcommunity4/10

   Gemini CLI documents cloud-adjacent automation via its GitHub Actions integration (PR reviews, issue triage, @gemini-cli on-demand assistance, non-interactive scripting) which could kick off agentic work, but there is no vendor evidence of an autonomous cloud agent that builds, runs tests, and produces a demo end-to-end for review. Community reports also describe agentic mode getting stuck in error loops, failing at basic file edits, and even causing data loss, undercutting confidence in reliable autonomous execution. missing for 10: explicit end-to-end build+test+demo workflow, evidence of a hosted/cloud agent (vs local CLI or CI hooks) producing a reviewable demo, and independent confirmation that autonomous runs complete without failure loops.

   • [github] “Pull Request Reviews: Automated code review with contextual feedback and suggestions”
   • [github] “Issue Triage: Automated labeling and prioritization of GitHub issues based on content analysis”
   • [github] “On-demand Assistance: Mention @gemini-cli in issues and pull requests for help with debugging, explanations, or task delegation”
   • [github] “Run non-interactively in scripts for workflow automation”
   • [community] “A lot of times Gemini models will get stuck in a loop of errors, and a lot of times it fails to edit/read or other simple function calling -…”
   • [community] “I really tried to get gemini to work properly in Agent mode. Tho it way too often went crazy, started rewriting files empty, and ran into pe…”
   • [community] “Gemini told the user: 'I have failed you completely and catastrophically... I have lost your data. This is an unacceptable, irreversible fai…”

   proof ↗⚑ flag

   Devin~partialcommunity7/10

   Devin's cloud sessions run in dedicated VMs with shell/browser/full repo access, can implement features, run tests, and continue autonomously after handoff, then present PRs for review (docs-1,13,17). However, community hands-on reports describe unreliable autonomy — extraneous breaking changes, inability to self-correct, and needing frequent human monitoring/termination — undercutting the 'build, test, demo end-to-end' promise. missing for 10: reliable independent verification of unattended end-to-end demo quality, and clearer evidence of a built-in demo/walkthrough artifact for reviewers beyond PR creation.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “Devin Review provides automated first-pass reviews on pull requests, checking for correctness and conformance with organizational best pract…”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

2. developerDelegate longer-running coding tasks to run in the background in an isolated cloud environment

   weight 3 · round to Devin
   Gemini CLI~partialclaimed4/10

   Gemini CLI supports non-interactive scripting and GitHub Actions integration (@gemini-cli mentions for PR reviews, issue triage, on-demand assistance) which can run tasks in a cloud CI environment, and Cloud Shell offers a ready cloud runtime — but there's no dedicated 'run this long task in an isolated background cloud sandbox' feature akin to a hosted agent service. missing for 10: explicit isolated cloud sandbox/background execution product, evidence of long-running autonomous task delegation outside CI triggers, and independent confirmation it works reliably for extended background jobs.

   • [github] “Run non-interactively in scripts for workflow automation”
   • [github] “Pull Request Reviews: Automated code review with contextual feedback and suggestions”
   • [github] “Issue Triage: Automated labeling and prioritization of GitHub issues based on content analysis”
   • [github] “On-demand Assistance: Mention @gemini-cli in issues and pull requests for help with debugging, explanations, or task delegation”
   • [github] “On-demand Assistance: Mention `@gemini-cli` in issues and pull requests for help with debugging, explanations, or task delegation”
   • [claimed-docs] “The Gemini CLI is available without additional setup in Cloud Shell”

   proof ↗⚑ flag

   Devin✓fullcommunity9/10

   Devin runs tasks in isolated cloud VMs with full shell/browser/repo access that persist after the user disconnects, explicitly supporting long-running background work and parallel independent tasks, corroborated by community reports of multi-day autonomous runs. Missing for 10: independent third-party benchmarking of long-running task reliability/quality beyond anecdotal HN reports.

   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”
   • [claimed-docs] “Configure it once, and every session boots into that known-good state.”
   • [claimed-docs] “Hand a task off to a cloud Devin session and keep working locally.”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “You can set a 'max work time' before Devin pauses so it won't go for days endlessly spending your credits. By default it's set to 10 credits…”

   proof ↗⚑ flag

3. developerConfigure a reproducible cloud environment with the dependencies and setup steps my repository needs

   weight 2 · round to Devin
   Gemini CLI—none0/10

   Evidence shows Gemini CLI can run in Cloud Shell without extra setup and supports GEMINI.md context files, but there is no evidence of a configurable, reproducible cloud environment (e.g., dependency/setup scripts, devcontainer-style config) that a developer can define for their repo. Missing for 10: any documented environment/setup-script configuration mechanism, evidence of reproducibility across runs, and independent confirmation it works as such.

   • [claimed-docs] “The Gemini CLI is available without additional setup in Cloud Shell”
   • [github] “Custom context files (GEMINI.md) to tailor behavior for your projects”

   proof ↗⚑ flag

   Devin✓fullclaimed9/10

   Devin explicitly supports configuring environment 'blueprints' that specify tools, runtimes, and dependencies so 'every session boots into that known-good state,' with auto-detection of requirements from the repo (docs-31, docs-32), plus indexing (docs-27), knowledge/AGENTS.md context files (docs-29, docs-30), and VPN access for internal dependencies (docs-28), all running in isolated cloud VMs (docs-13). Missing for 10: independent/hands-on confirmation that blueprint-based environments reliably reproduce across sessions in practice.

   • [claimed-docs] “Configure it once, and every session boots into that known-good state.”
   • [claimed-docs] “Devin inspects your repository, figures out which tools, runtimes, and dependencies are needed, and generates the blueprint for you.”
   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”
   • [claimed-docs] “Knowledge is a collection of instructions and advice that Devin can reference in all sessions.”
   • [claimed-docs] “Devin supports AGENTS.md - a simple, open standard for providing context and instructions to AI agents.”
   • [claimed-docs] “Devin can connect to a VPN from inside its workspace, so sessions can reach internal services such as package registries, databases, and int…”
   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”

   proof ↗⚑ flag

Parallel agents

1. ai-native userLaunch fleets of autonomous agents that work in parallel on different tasks for hours or days

   weight 2 · round to Devin
   Gemini CLI—none0/10

   Evidence shows single-session non-interactive scripting, GitHub Actions integration for issue triage/PR review, and MCP extensibility, but nothing about launching multiple autonomous agents working in parallel for hours or days. No fleet/orchestration/multi-agent parallelism capability is documented anywhere in the pack.

   ⚑ flag

   Devin~partialcommunity6/10

   Docs confirm parallel task execution ('Carve out independent tasks and run them simultaneously'), cloud sessions that persist after closing the laptop, and an API to spin up multiple sessions programmatically (including on behalf of other users), which together support a 'fleet of parallel long-running agents' story. However, community hands-on reports show real friction with the 'hours/days autonomous' claim: sessions can get stuck without an escape hatch, users must babysit and manually terminate sessions every 10-15 minutes, and there's a default max-work-time cap limiting unsupervised runtime. Missing for 10: independent verification of successful multi-day/multi-task fleets running unattended, and evidence addressing the reported lack of a reliable 'ask for help' escalation during long runs.

   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”
   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “Devin does ask for help when it can't do something, but it really hates asking for help if it's a skill issue - it would prefer running in c…”
   • [community] “You can set a 'max work time' before Devin pauses so it won't go for days endlessly spending your credits. By default it's set to 10 credits…”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

2. developerRun several task attempts in parallel and compare results before choosing one

   weight 1 · round drawn
   Gemini CLI—none0/10

   No evidence in the pack describes running multiple parallel task attempts or comparing/diffing results before selecting one; features listed are single-session tools (checkpointing, MCP, scripting) with no multi-attempt/parallel comparison workflow mentioned. Missing for 10: any mention of parallel run/branching feature, a comparison UI or mechanism to pick the best of several attempts.

   ⚑ flag

   Devin—none0/10

   Devin's docs mention running multiple independent tasks simultaneously (devin-docs-25) but this describes parallelizing different tasks, not running several parallel attempts of the SAME task to compare and choose the best result. No evidence describes a compare/choose-best-attempt workflow.

   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”

   proof ↗⚑ flag

Scheduled automation

1. ai-native userSet up always-on agents that run on schedules or triggers to maintain and fix my software autonomously

   weight 2 · round to Devin
   Gemini CLI~partialcommunity4/10

   Gemini CLI supports non-interactive scripted runs and GitHub Actions-based triggers (PR reviews, issue triage, @mention on-demand assistance) which can approximate scheduled/triggered automation, but there is no evidence of a persistent, self-scheduling 'always-on agent' that autonomously maintains and fixes software over time. missing for 10: native scheduler/cron support, persistent agent daemon or watch-mode, evidence of autonomous multi-cycle maintenance without human triggering, and reliability data (community reports actually describe agent mode getting stuck in loops or failing tasks).

   • [github] “Run non-interactively in scripts for workflow automation”
   • [github] “Pull Request Reviews: Automated code review with contextual feedback and suggestions”
   • [github] “Issue Triage: Automated labeling and prioritization of GitHub issues based on content analysis”
   • [github] “On-demand Assistance: Mention @gemini-cli in issues and pull requests for help with debugging, explanations, or task delegation”
   • [community] “A lot of times Gemini models will get stuck in a loop of errors, and a lot of times it fails to edit/read or other simple function calling -…”
   • [community] “I really tried to get gemini to work properly in Agent mode. Tho it way too often went crazy, started rewriting files empty, and ran into pe…”

   proof ↗⚑ flag

   Devin~partialcommunity6/10

   Devin supports trigger-based autonomous work (Slack/Teams tags, PR review comments, CI/CD/static-analysis findings) and Auto-Fix creates a closed loop that iterates on CI failures without a human in the loop, and MCP exposes 'scheduling' as a session capability, all suggesting some always-on/triggered agent operation. However there's no dedicated docs for cron-like recurring schedules, and community reports describe sessions needing frequent human monitoring/termination rather than fully unattended long-running maintenance. Missing for 10: explicit scheduling/cron configuration docs, independent evidence of reliable unattended multi-day maintenance loops, and confirmation that Auto-Fix/CI triggers work without human oversight in practice.

   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “You can set a 'max work time' before Devin pauses so it won't go for days endlessly spending your credits. By default it's set to 10 credits…”

   proof ↗⚑ flag

Debugging

1. developerDebug a live running web application directly from my coding assistant

   weight 1 · round to Devin
   Gemini CLI~partialcommunity3/10

   Gemini CLI advertises general 'Debug issues and troubleshoot with natural language' capability and MCP extensibility that could in theory connect to browser/dev tools, and one community comment references an internal 'browser control stack,' but there is no first-party or hands-on evidence of live web-app debugging (e.g., attaching to a running app, inspecting DOM/network/console, or browser automation workflows). Missing for 10: explicit live-app/browser debugging workflow docs, DevTools or runtime inspection integration, and hands-on confirmation of debugging a running web app.

   • [github] “Debug issues and troubleshoot with natural language”
   • [github] “Use MCP servers to connect new capabilities, including media generation with Imagen, Veo or Lyria”
   • [github] “Configure MCP servers in ~/.gemini/settings.json to extend Gemini CLI with custom tools”
   • [community] “All in all, a 140 MB Go binary with its own browser control stack, sandbox, Git, language detector, skills runtime, and subagent system. I'm…”

   proof ↗⚑ flag

   Devin~partialclaimed6/10

   Devin's computer-use/desktop mode gives it a full browser and desktop environment (mouse, keyboard, screenshots) plus VPN access to internal services, and docs explicitly mention reproducing and fixing bugs, which together support interacting with and debugging a live running app. However, there's no explicit documentation of dev-tools-style debugging features (breakpoints, console/log inspection, network tracing) or a dedicated 'live app debugging' workflow, and community evidence doesn't corroborate this specific use case. Missing for 10: explicit live-debugging tooling (breakpoints/console/log inspection), independent hands-on confirmation of debugging a running app.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Devin has access to a full desktop environment — not just a browser. It can move the mouse, click on UI elements, type on the keyboard, take…”
   • [claimed-docs] “Devin can connect to a VPN from inside its workspace, so sessions can reach internal services such as package registries, databases, and int…”
   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”

   proof ↗⚑ flag

2. developerDebug issues and troubleshoot using natural-language queries

   weight 2 · round to Devin
   Gemini CLI!disputedcontradicted5/10

   Gemini CLI explicitly advertises natural-language debugging/troubleshooting (gemini-cli-gh-3, gh-12/21/23) and community reports confirm strong codebase navigation and code-review value (gemini-cli-comm-1, comm-20). However, multiple hands-on reports directly contradict reliable debugging: users describe it getting stuck in error loops, failing simple file edits, and in one case catastrophically deleting user data during a troubleshooting session (gemini-cli-comm-10, comm-11, comm-14, comm-16).

   • [github] “Debug issues and troubleshoot with natural language”
   • [github] “On-demand Assistance: Mention @gemini-cli in issues and pull requests for help with debugging, explanations, or task delegation”
   • [github] “On-demand Assistance: Mention `@gemini-cli` in issues and pull requests for help with debugging, explanations, or task delegation”
   • [github] “Mention @gemini-cli in issues and pull requests for help with debugging, explanations, or task delegation”
   • [community] “I have been using this for about a month and it's a beast, mostly thanks to 2.5pro being SOTA and how it leverages that huge 1M context wind…”
   • [community] “We have tried out Gemini code review vs Copilot code review and Gemini is consistently offering better code review tips. It has officially c…”
   • [community] “A lot of times Gemini models will get stuck in a loop of errors, and a lot of times it fails to edit/read or other simple function calling -…”
   • [community] “I really tried to get gemini to work properly in Agent mode. Tho it way too often went crazy, started rewriting files empty, and ran into pe…”
   • [community] “The problem is that Gemini CLI simply doesn't work. Beside simplest tasks like creating a new release it is useless as a coding assistant. D…”
   • [community] “Gemini told the user: 'I have failed you completely and catastrophically... I have lost your data. This is an unacceptable, irreversible fai…”

   proof ↗⚑ flag

   Devin~partialcommunity7/10

   Docs clearly support NL-driven debugging: 'repro and fix bugs', 'Ask Devin can answer questions about code structure...help you scope and plan tasks', and tagging Devin in Slack/Teams about a bug thread. However, hands-on community reports describe practical caveats—Devin adding extraneous changes it can't undo, getting stuck without escalating, requiring manual babysitting—that temper reliability for troubleshooting workflows. Missing for 10: independent benchmark/case study specifically on debugging accuracy, and resolution of the 'getting stuck on bugs' community complaint.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Tagging Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “Ask Devin can answer questions about code structure and dependencies, and help you scope and plan tasks before implementation.”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “Devin does ask for help when it can't do something, but it really hates asking for help if it's a skill issue - it would prefer running in c…”

   proof ↗⚑ flag

Feature implementation

1. developerTurn a tracked issue into a complete pull request end-to-end

   weight 3 · round to Devin
   Gemini CLI!disputedcontradicted5/10

   Gemini CLI's GitHub integration supports @gemini-cli task delegation from issues/PRs, automated PR reviews, and issue triage, which vendor docs frame as enabling issue-to-PR workflows (gh-12, gh-21, gh-10, gh-22, gh-4). However, hands-on community reports describe the agent getting stuck in loops, failing basic file edits, lacking a plan mode, and producing 'spaghetti code' rather than completing tasks reliably — directly undermining claims of smooth end-to-end PR generation (gemini-cli-comm-10, gemini-cli-comm-11, gemini-cli-comm-14). Missing for 10: a documented full issue→PR walkthrough, evidence of successful autonomous PR creation from an issue, and independent confirmation resolving the agentic reliability complaints.

   • [github] “On-demand Assistance: Mention @gemini-cli in issues and pull requests for help with debugging, explanations, or task delegation”
   • [github] “On-demand Assistance: Mention `@gemini-cli` in issues and pull requests for help with debugging, explanations, or task delegation”
   • [github] “Pull Request Reviews: Automated code review with contextual feedback and suggestions”
   • [github] “@github List my open pull requests”
   • [github] “Automate operational tasks like querying pull requests or handling complex rebases”
   • [community] “A lot of times Gemini models will get stuck in a loop of errors, and a lot of times it fails to edit/read or other simple function calling -…”
   • [community] “I really tried to get gemini to work properly in Agent mode. Tho it way too often went crazy, started rewriting files empty, and ran into pe…”
   • [community] “The problem is that Gemini CLI simply doesn't work. Beside simplest tasks like creating a new release it is useless as a coding assistant. D…”

   proof ↗⚑ flag

   Devin~partialcommunity6/10

   Devin's docs explicitly describe taking Linear/Jira tickets and implementing full features, with Devin Review/Auto-Fix looping PRs toward merge-ready status without human involvement, covering the issue-to-PR pipeline end-to-end. However, hands-on community reports describe practical friction — extraneous unrelated changes that can break things, inability to easily undo them, and agents getting stuck for days rather than asking for help — casting doubt on how cleanly the 'complete' PR is delivered without oversight. Missing for 10: independent verification of a clean ticket→merged-PR flow without manual intervention, and resolution of the reported reliability/quality issues.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Devin Review provides automated first-pass reviews on pull requests, checking for correctness and conformance with organizational best pract…”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Code migrations, refactors, and modernization”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “Devin does ask for help when it can't do something, but it really hates asking for help if it's a skill issue - it would prefer running in c…”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

2. developerDescribe a feature or bug in plain language and have the agent implement or fix it across multiple files

   weight 3 · round drawn
   Gemini CLI!disputedcontradicted5/10

   Vendor docs/GitHub claim strong support for describing features/bugs in plain language and having the agent edit/debug across large codebases (gemini-cli-gh-1, gemini-cli-gh-3), but multiple hands-on community reports directly contradict this: users report the agent getting stuck in error loops, failing basic file edit/read operations, ignoring GEMINI.md context files, jumping straight into 'spaghetti code' without a plan mode, and in one case catastrophically deleting user data via botched commands. missing for 10: consistent hands-on success stories on multi-file feature implementation, resolution of the reported reliability/looping failures, and independent benchmarks confirming multi-file bug-fix accuracy.

   • [github] “Query and edit large codebases”
   • [github] “Debug issues and troubleshoot with natural language”
   • [github] “Custom context files (GEMINI.md) to tailor behavior for your projects”
   • [community] “A lot of times Gemini models will get stuck in a loop of errors, and a lot of times it fails to edit/read or other simple function calling -…”
   • [community] “I really tried to get gemini to work properly in Agent mode. Tho it way too often went crazy, started rewriting files empty, and ran into pe…”
   • [community] “Tip 1, it consistently ignores my GEMINI.md file, both global and local, even though it always says '1 GEMINI.md file is being used.'”
   • [community] “The problem is that Gemini CLI simply doesn't work. Beside simplest tasks like creating a new release it is useless as a coding assistant. D…”
   • [community] “Gemini told the user: 'I have failed you completely and catastrophically... I have lost your data. This is an unacceptable, irreversible fai…”

   proof ↗⚑ flag

   Devin!disputedcontradicted5/10

   Devin's docs strongly claim end-to-end feature/bug implementation across a full repo (Linear/Jira tickets, multi-file fixes, code migrations) with full workspace/VM access [devin-docs-1, devin-docs-13, devin-docs-21, devin-docs-27], but hands-on community reports concretely contradict smooth delivery: it 'always adds extraneous changes beyond the task that can break other things, and can't undo those changes if asked' and required constant supervision/termination to get real work done [devin-comm-1, devin-comm-5], with another user noting it can run for days without an escape hatch when stuck [devin-comm-2]. Missing for 10: independent benchmark data on multi-file correctness, and resolution of the extraneous-change/undo failure mode reported by users.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “Code migrations, refactors, and modernization”
   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

Maintenance automation

1. developerHave the agent write tests, fix lint errors, resolve merge conflicts, and update dependencies for me

   weight 3 · round to Devin
   Gemini CLI!disputedcontradicted5/10

   GitHub docs claim broad code-editing, debugging, and complex-rebase (merge conflict) automation capabilities (gemini-cli-gh-1, gemini-cli-gh-3, gemini-cli-gh-4), which would cover fixing lint issues and dependency/test work as part of general codebase editing, and PR review/issue triage features suggest lint-like feedback (gemini-cli-gh-10, gemini-cli-gh-11). However, multiple hands-on community reports concretely contradict reliable agentic code work: users report it getting stuck in error loops, failing simple file edit/read operations, ignoring GEMINI.md context files, producing 'spaghetti code' with no plan mode, and in one case catastrophically deleting user data during a file operation (gemini-cli-comm-10, gemini-cli-comm-11, gemini-cli-comm-13, gemini-cli-comm-14, gemini-cli-comm-16). No explicit evidence names test-writing, lint-fixing, or dependency-updating tasks specifically. Missing for 10: explicit documentation/examples of writing tests, fixing lint errors, or updating dependencies, and independent corroboration that these specific tasks work reliably.

   • [github] “Query and edit large codebases”
   • [github] “Debug issues and troubleshoot with natural language”
   • [github] “Automate operational tasks like querying pull requests or handling complex rebases”
   • [github] “Pull Request Reviews: Automated code review with contextual feedback and suggestions”
   • [github] “Issue Triage: Automated labeling and prioritization of GitHub issues based on content analysis”
   • [community] “A lot of times Gemini models will get stuck in a loop of errors, and a lot of times it fails to edit/read or other simple function calling -…”
   • [community] “I really tried to get gemini to work properly in Agent mode. Tho it way too often went crazy, started rewriting files empty, and ran into pe…”
   • [community] “Tip 1, it consistently ignores my GEMINI.md file, both global and local, even though it always says '1 GEMINI.md file is being used.'”
   • [community] “The problem is that Gemini CLI simply doesn't work. Beside simplest tasks like creating a new release it is useless as a coding assistant. D…”
   • [community] “Gemini told the user: 'I have failed you completely and catastrophically... I have lost your data. This is an unacceptable, irreversible fai…”

   proof ↗⚑ flag

   Devin~partialcommunity6/10

   Docs show Devin is a general-purpose coding agent that can fix bugs, implement features, iterate on CI failures, respond to review comments (Auto-Fix), and handle code migrations/refactors/modernization, which plausibly covers lint fixes and CI-related work, but none of the docs explicitly mention writing tests, resolving merge conflicts, or updating dependencies as named capabilities. Community reports (devin-comm-1) also note Devin can introduce extraneous changes and struggles to cleanly undo them, tempering confidence in reliably delivering these specific maintenance tasks. Missing for 10: explicit documentation/evidence of test-writing, lint-fixing, merge-conflict resolution, and dependency-update workflows, plus independent hands-on confirmation of these specific tasks succeeding.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Code migrations, refactors, and modernization”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”

   proof ↗⚑ flag

Multimodal generation

1. ai-native userGenerate a working app from a sketch, image, or PDF design

   weight 2 · round to Gemini CLI
   Gemini CLI~partialcommunity6/10

   Official docs explicitly claim 'Generate new apps from PDFs, images, or sketches using multimodal capabilities,' directly matching the story, but there is no independent/hands-on corroboration of this specific capability, and broader community feedback raises general concerns about agentic reliability that could affect complex generation tasks. missing for 10: independent hands-on demonstration of sketch/PDF-to-app generation, details on fidelity/limitations of this workflow.

   • [github] “Generate new apps from PDFs, images, or sketches using multimodal capabilities”
   • [community] “A lot of times Gemini models will get stuck in a loop of errors, and a lot of times it fails to edit/read or other simple function calling -…”
   • [community] “The problem is that Gemini CLI simply doesn't work. Beside simplest tasks like creating a new release it is useless as a coding assistant. D…”

   proof ↗⚑ flag

   Devin—none0/10

   No evidence anywhere in the pack that Devin accepts a sketch/image/PDF design as input and generates a working app from it; documentation focuses on text-based tasks, tickets, Slack threads, code review, and CLI/desktop environment features with no multimodal design-to-app capability mentioned.

   ⚑ flag

Codebase mapping

1. developerUnderstand how a codebase fits together to find where to start making changes

   weight 3 · round to Devin
   Gemini CLI!disputedcontradicted5/10

   Gemini CLI advertises large-codebase querying/editing (gemini-cli-gh-1) with a 1M-token context window, custom GEMINI.md context files, and --include-directories flags for scoping (gemini-cli-gh-9, gemini-cli-gh-16), and one HN user praises its ability to 'navigate and learn' large codebases effortlessly (gemini-cli-comm-1). However, other hands-on users report the opposite: it is 'stupid at navigation in the codebase' taking 10x longer (gemini-cli-comm-15) and 'consistently ignores' the GEMINI.md context file despite claiming to use it (gemini-cli-comm-13), directly undercutting the codebase-understanding claim. Missing for 10: consistent independent corroboration of reliable codebase navigation, and no contradicting failure reports.

   • [github] “Query and edit large codebases”
   • [github] “Custom context files (GEMINI.md) to tailor behavior for your projects”
   • [github] “gemini --include-directories ../lib,../docs”
   • [community] “I have been using this for about a month and it's a beast, mostly thanks to 2.5pro being SOTA and how it leverages that huge 1M context wind…”
   • [community] “I love the model, hate the tool. Anthropic has the killer app with Claude Code. I tried Gemini cli for about 5 seconds and was so frustrated…”
   • [community] “Tip 1, it consistently ignores my GEMINI.md file, both global and local, even though it always says '1 GEMINI.md file is being used.'”

   proof ↗⚑ flag

   Devin✓fullclaimed7/10

   Devin explicitly indexes repositories to power 'Ask Devin' and 'DeepWiki', which answer questions about code structure and dependencies and help developers scope/plan where to start making changes, directly matching the story. missing for 10: independent/hands-on evidence validating DeepWiki/Ask Devin's accuracy on real codebases (community evidence only covers task execution, not codebase-understanding features).

   • [claimed-docs] “Use DeepWiki to navigate architecture and code with auto-generated documentation.”
   • [claimed-docs] “Ask Devin can answer questions about code structure and dependencies, and help you scope and plan tasks before implementation.”
   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”

   proof ↗⚑ flag

2. developerHave the agent map and explain an entire unfamiliar codebase without manually selecting context files

   weight 3 · round to Devin
   Gemini CLI!disputedcontradicted5/10

   Google claims large-codebase querying/editing (gemini-cli-gh-1) and Gemini CLI's 1M-token context lets it 'navigate and learn' huge codebases 'effortlessly' per one user (gemini-cli-comm-1), but other hands-on reports directly contradict this, calling it 'so stupid at navigation in the codebase it takes 10x as long' (gemini-cli-comm-15) and prone to getting 'stuck in spaghetti code' with no plan mode (gemini-cli-comm-14), plus it reportedly ignores its own GEMINI.md context file (gemini-cli-comm-13). Missing for 10: consistent independent benchmarks confirming autonomous whole-codebase mapping without file selection, and resolution of the navigation-quality contradiction.

   • [github] “Query and edit large codebases”
   • [community] “I have been using this for about a month and it's a beast, mostly thanks to 2.5pro being SOTA and how it leverages that huge 1M context wind…”
   • [community] “I love the model, hate the tool. Anthropic has the killer app with Claude Code. I tried Gemini cli for about 5 seconds and was so frustrated…”
   • [community] “The problem is that Gemini CLI simply doesn't work. Beside simplest tasks like creating a new release it is useless as a coding assistant. D…”
   • [community] “Tip 1, it consistently ignores my GEMINI.md file, both global and local, even though it always says '1 GEMINI.md file is being used.'”

   proof ↗⚑ flag

   Devin✓fullclaimed7/10

   Docs describe repo indexing that lets Devin understand the codebase and power 'Ask Devin' and DeepWiki auto-generated architecture docs, directly enabling exploration/explanation of an unfamiliar codebase without manual file selection (devin-docs-27, devin-docs-15, devin-docs-16). No community evidence contradicts this specific capability. Missing for 10: independent/hands-on verification of codebase-mapping accuracy and no concrete example of DeepWiki output quality.

   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”
   • [claimed-docs] “Use DeepWiki to navigate architecture and code with auto-generated documentation.”
   • [claimed-docs] “Ask Devin can answer questions about code structure and dependencies, and help you scope and plan tasks before implementation.”

   proof ↗⚑ flag

Context management

1. developerHave the agent build and recall memory automatically across sessions

   weight 2 · round to Devin
   Gemini CLI!disputedcontradicted3/10

   Gemini CLI offers static project context via GEMINI.md files and a `/memory` command, plus manual conversation checkpointing to save/resume sessions—but these are manually configured/invoked, not automatic memory building/recall across sessions. Hands-on community evidence directly contradicts even the GEMINI.md context mechanism working reliably: a user reports it 'consistently ignores my GEMINI.md file... even though it always says 1 GEMINI.md file is being used' (gemini-cli-comm-13), undermining the claimed persistent-context capability. missing for 10: evidence of automatic memory formation/recall without user action, evidence /memory command builds persistent cross-session knowledge, independent corroboration that GEMINI.md context reliably persists.

   • [github] “Conversation checkpointing to save and resume complex sessions”
   • [github] “Custom context files (GEMINI.md) to tailor behavior for your projects”
   • [claimed-docs] “Comandos de Gemini CLI: /memory, /stats, /tools y /mcp”
   • [community] “Tip 1, it consistently ignores my GEMINI.md file, both global and local, even though it always says '1 GEMINI.md file is being used.'”

   proof ↗⚑ flag

   Devin~partialclaimed5/10

   Devin supports persistent cross-session context via "Knowledge" (instructions referenced in all sessions), AGENTS.md, and environment blueprints that let every session boot into a known-good state, which enables some recall across sessions. However, these mechanisms are largely user-configured/onboarded rather than autonomously built by the agent from its own experience, and there's no evidence of automatic memory creation or recall behavior demonstrated in practice. Missing for 10: evidence the agent automatically extracts/updates memory from its own task experience without manual setup, and independent confirmation that recalled memory improves subsequent session performance.

   • [claimed-docs] “Knowledge is a collection of instructions and advice that Devin can reference in all sessions.”
   • [claimed-docs] “Devin supports AGENTS.md - a simple, open standard for providing context and instructions to AI agents.”
   • [claimed-docs] “Configure it once, and every session boots into that known-good state.”
   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”

   proof ↗⚑ flag

2. developerInclude multiple project directories in a single session for broader context

   weight 2 · round to Gemini CLI
   Gemini CLI✓fullclaimed8/10

   The official CLI flag `--include-directories ../lib,../docs` explicitly allows adding multiple project directories into a single session for broader context, directly matching the story. Missing for 10: independent hands-on confirmation of multi-directory usage quality/behavior beyond the flag documentation.

   • [github] “gemini --include-directories ../lib,../docs”

   proof ↗⚑ flag

   Devin—none0/10

   No evidence describes attaching or indexing multiple project directories/repositories within a single Devin session; docs discuss single-repo indexing, knowledge, and environment setup but not multi-directory context in one session.

   ⚑ flag

3. developerAdd a project instructions file to set coding standards and conventions the agent follows

   weight 3 · round to Devin
   Gemini CLI!disputedcontradicted5/10

   Gemini CLI documents GEMINI.md custom context files for tailoring behavior/project conventions (gemini-cli-gh-9) and docs mention /memory command for managing this context (gemini-cli-docs-3). However, hands-on community feedback reports the file being ignored despite being loaded ('it consistently ignores my GEMINI.md file, both global and local, even though it always says 1 GEMINI.md file is being used' - gemini-cli-comm-13), directly contradicting reliable adherence to project instructions. Missing for 10: independent corroboration that GEMINI.md is consistently honored, more detail on precedence/hierarchy of instruction files, and resolution of the reported ignoring behavior.

   • [github] “Custom context files (GEMINI.md) to tailor behavior for your projects”
   • [claimed-docs] “Comandos de Gemini CLI: /memory, /stats, /tools y /mcp”
   • [community] “Tip 1, it consistently ignores my GEMINI.md file, both global and local, even though it always says '1 GEMINI.md file is being used.'”

   proof ↗⚑ flag

   Devin✓fullclaimed9/10

   Devin explicitly supports AGENTS.md, an open standard for providing context and instructions to AI agents, plus a separate 'Knowledge' feature for instructions/advice referenced across all sessions, directly covering project-level coding standards/conventions. Missing for 10: independent/hands-on confirmation that these instructions are reliably followed in practice.

   • [claimed-docs] “Devin supports AGENTS.md - a simple, open standard for providing context and instructions to AI agents.”
   • [claimed-docs] “Knowledge is a collection of instructions and advice that Devin can reference in all sessions.”

   proof ↗⚑ flag

Issue diagnosis

1. developerReproduce issues, narrow down root causes, and verify fixes

   weight 3 · round to Devin
   Gemini CLI!disputedcontradicted4/10

   Google markets debugging/troubleshooting via natural language and a /bug reporting flow (gh-3, gh-20), and one HN user praises its ability to navigate huge codebases (comm-1). However multiple hands-on reports directly contradict root-cause/verify-fix workflows: users describe it getting stuck in error loops, rewriting files empty, ignoring GEMINI.md context, being 'terrible at agentic stuff', and in one case catastrophically deleting user data during a file operation (comm-10, comm-11, comm-13, comm-14, comm-15, comm-16). missing for 10: reliable reproduction of bugs, consistent root-cause narrowing without loops, and independent verification of fix correctness.

   • [github] “Debug issues and troubleshoot with natural language”
   • [github] “Use `/bug` command to report issues directly from the CLI.”
   • [community] “I have been using this for about a month and it's a beast, mostly thanks to 2.5pro being SOTA and how it leverages that huge 1M context wind…”
   • [community] “A lot of times Gemini models will get stuck in a loop of errors, and a lot of times it fails to edit/read or other simple function calling -…”
   • [community] “I really tried to get gemini to work properly in Agent mode. Tho it way too often went crazy, started rewriting files empty, and ran into pe…”
   • [community] “The problem is that Gemini CLI simply doesn't work. Beside simplest tasks like creating a new release it is useless as a coding assistant. D…”
   • [community] “I love the model, hate the tool. Anthropic has the killer app with Claude Code. I tried Gemini cli for about 5 seconds and was so frustrated…”
   • [community] “Gemini told the user: 'I have failed you completely and catastrophically... I have lost your data. This is an unacceptable, irreversible fai…”

   proof ↗⚑ flag

   Devin~partialcommunity6/10

   Docs explicitly claim Devin can 'repro and fix bugs' (devin-docs-1), use Ask Devin/DeepWiki to narrow root causes via codebase understanding (devin-docs-15, devin-docs-16, devin-docs-27), and verify fixes through CI iteration/Auto-Fix loops (devin-docs-18). However, hands-on community testimony reports Devin often adds extraneous changes beyond the task scope and cannot reliably undo them when asked, undermining clean verification of fixes (devin-comm-1), and lacks an escape hatch when stuck on root-cause diagnosis (devin-comm-2, devin-comm-3). Missing for 10: independent verification of successful bug reproduction/root-cause narrowing at scale, and resolution of the reported inability to cleanly revert unwanted changes during fix verification.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Use DeepWiki to navigate architecture and code with auto-generated documentation.”
   • [claimed-docs] “Ask Devin can answer questions about code structure and dependencies, and help you scope and plan tasks before implementation.”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “Devin does ask for help when it can't do something, but it really hates asking for help if it's a skill issue - it would prefer running in c…”

   proof ↗⚑ flag

Marketplace

1. developerEquip the agent with custom skills to perform specialized tasks

   weight 1 · round drawn
   Gemini CLI~partialcommunity6/10

   Gemini CLI supports extensibility through MCP servers (custom tools, media generation) and GEMINI.md context files to tailor agent behavior for specific projects, and a community mention references a built-in 'skills runtime' as part of its architecture. However, there is no dedicated first-party 'skills' marketplace or packaging system, and community reports note GEMINI.md is sometimes ignored in practice. Missing for 10: a documented first-class 'skills' framework/marketplace, independent corroboration that custom skills work reliably, and confirmation that the skills runtime mentioned in community feedback is a stable, documented feature.

   • [github] “Use MCP servers to connect new capabilities, including media generation with Imagen, Veo or Lyria”
   • [github] “Custom context files (GEMINI.md) to tailor behavior for your projects”
   • [github] “Configure MCP servers in ~/.gemini/settings.json to extend Gemini CLI with custom tools”
   • [community] “All in all, a 140 MB Go binary with its own browser control stack, sandbox, Git, language detector, skills runtime, and subagent system. I'm…”
   • [community] “Tip 1, it consistently ignores my GEMINI.md file, both global and local, even though it always says '1 GEMINI.md file is being used.'”

   proof ↗⚑ flag

   Devin~partialclaimed6/10

   Devin supports several skill-like extension mechanisms — 'Knowledge' (persistent instructions/advice for all sessions), AGENTS.md for structured agent instructions, and 'Playbooks' exposed via MCP — plus an open-source 'Devin Handoff' explicitly described as a 'plugin and skill'. This gives developers real levers to encode specialized, reusable task behavior, but the docs don't show a dedicated skill-authoring UI/marketplace or detailed examples of building a complex custom skill, and there is no independent/community evidence confirming this works well in practice. Missing for 10: concrete examples/tutorials of authoring a non-trivial custom skill, a discoverable skills registry, and independent hands-on corroboration.

   • [claimed-docs] “Knowledge is a collection of instructions and advice that Devin can reference in all sessions.”
   • [claimed-docs] “Devin supports AGENTS.md - a simple, open standard for providing context and instructions to AI agents.”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “Devin Handoff is an open-source plugin and skill that brings the same handoff workflow to any coding agent — Claude Code, Codex, Cursor, and…”

   proof ↗⚑ flag

2. engineering-leadIntegrate third-party partner-built agent apps into my workflows

   weight 1 · round to Gemini CLI
   Gemini CLI~partialclaimed5/10

   Gemini CLI supports connecting external capabilities via MCP servers (e.g., Imagen, Veo, Lyria) and integrates with GitHub via @gemini-cli mentions and Actions, showing some ecosystem extensibility for third-party tools. However, there's no evidence of a curated marketplace or directory of partner-built 'agent apps' specifically designed for cross-workflow integration, only generic MCP server configuration support. Missing for 10: a documented partner/agent-app ecosystem or marketplace, case studies of third-party agent apps being integrated, and independent confirmation of smooth interoperability.

   • [github] “Use MCP servers to connect new capabilities, including media generation with Imagen, Veo or Lyria”
   • [github] “Configure MCP servers in ~/.gemini/settings.json to extend Gemini CLI with custom tools”
   • [github] “@github List my open pull requests”
   • [github] “On-demand Assistance: Mention @gemini-cli in issues and pull requests for help with debugging, explanations, or task delegation”

   proof ↗⚑ flag

   Devin~partialprobed4/10

   Devin exposes an MCP server so external MCP-compatible agents/IDEs can access its session management and tools, and its open-source 'Devin Handoff' plugin interoperates with other coding agents (Claude Code, Codex, Cursor), showing some cross-agent workflow integration. However there is no evidence of a partner/marketplace ecosystem of third-party agent apps being integrated into Devin's own workflows. Missing for 10: a documented partner-app marketplace or catalog, evidence of installing/configuring third-party agent apps within Devin, and any case study of an engineering-lead orchestrating partner-built agents through Devin.

   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “Devin Handoff is an open-source plugin and skill that brings the same handoff workflow to any coding agent — Claude Code, Codex, Cursor, and…”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [probe] “official MCP server documented at https://docs.devin.ai/work-with-devin/devin-mcp”

   proof ↗⚑ flag

Team knowledge

1. engineering-leadCreate a shared workspace from my docs and repos as a common source of truth for the team

   weight 1 · round to Devin
   Gemini CLI—none0/10

   Gemini CLI offers per-project GEMINI.md context files and --include-directories for local context, but there is no evidence of a shared, centrally managed team workspace combining docs and repos as a common source of truth across a team.

   • [github] “Custom context files (GEMINI.md) to tailor behavior for your projects”
   • [github] “gemini --include-directories ../lib,../docs”
   • [community] “Tip 1, it consistently ignores my GEMINI.md file, both global and local, even though it always says '1 GEMINI.md file is being used.'”

   proof ↗⚑ flag

   Devin~partialclaimed6/10

   Devin offers building blocks for a team-wide source of truth — repo indexing that powers 'Ask Devin' and DeepWiki architecture docs, org-wide 'Knowledge' referenced in all sessions, and AGENTS.md support for shared context — which collectively let a lead centralize docs/repo knowledge for the team. However, there's no explicit product feature framed as a 'shared workspace' UI for team-wide docs/repo browsing outside of Devin's own agent sessions. Missing for 10: a dedicated shared workspace/knowledge-base product surface for humans to browse, independent evidence of teams using it as a collaborative source of truth, and clarity on cross-repo doc aggregation beyond per-session knowledge.

   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”
   • [claimed-docs] “Knowledge is a collection of instructions and advice that Devin can reference in all sessions.”
   • [claimed-docs] “Devin supports AGENTS.md - a simple, open standard for providing context and instructions to AI agents.”
   • [claimed-docs] “Use DeepWiki to navigate architecture and code with auto-generated documentation.”
   • [claimed-docs] “Devin inspects your repository, figures out which tools, runtimes, and dependencies are needed, and generates the blueprint for you.”

   proof ↗⚑ flag

Tool integration

1. developerConnect the agent to workflow tools like Jira, Slack, and Google Drive to extend its context

   weight 3 · round to Devin
   Gemini CLI~partialclaimed5/10

   Gemini CLI supports connecting MCP servers to add custom tools/capabilities (gh-5, gh-19), which is the generic mechanism that could extend context to third-party services, but the evidence never mentions Jira, Slack, or Google Drive specifically or any pre-built connectors for them. Missing for 10: named/official Jira, Slack, or Google Drive integrations or MCP servers, and any documented example of using these workflow tools with Gemini CLI.

   • [github] “Use MCP servers to connect new capabilities, including media generation with Imagen, Veo or Lyria”
   • [github] “Configure MCP servers in ~/.gemini/settings.json to extend Gemini CLI with custom tools”

   proof ↗⚑ flag

   Devin~partialclaimed6/10

   Devin can be tagged in Slack/Teams threads and work Jira/Linear tickets, giving it direct workflow-tool integration, and its MCP server plus API enable further extension to other tools. However there is no explicit documented Google Drive integration, and no independent/hands-on corroboration of these integrations actually working in practice. missing for 10: Google Drive connector evidence, independent verification of Jira/Slack integration reliability.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Tagging Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”

   proof ↗⚑ flag

2. developerKick off agent tasks directly from GitHub, GitLab, Linear, or Slack

   weight 2 · round to Devin
   Gemini CLI~partialclaimed4/10

   Gemini CLI has a documented GitHub integration (GitHub Action/App) that lets developers trigger tasks via @gemini-cli mentions in issues/PRs, automated PR reviews, and issue triage, but there is no evidence of native GitLab, Linear, or Slack integrations for kicking off agent tasks. missing for 10: GitLab integration, Linear integration, Slack integration, independent corroboration of GitHub workflow reliability

   • [github] “Pull Request Reviews: Automated code review with contextual feedback and suggestions”
   • [github] “Issue Triage: Automated labeling and prioritization of GitHub issues based on content analysis”
   • [github] “On-demand Assistance: Mention @gemini-cli in issues and pull requests for help with debugging, explanations, or task delegation”
   • [github] “On-demand Assistance: Mention `@gemini-cli` in issues and pull requests for help with debugging, explanations, or task delegation”
   • [github] “@github List my open pull requests”
   • [github] “Mention @gemini-cli in issues and pull requests for help with debugging, explanations, or task delegation”

   proof ↗⚑ flag

   Devin~partialclaimed6/10

   Docs confirm Devin can be invoked from Linear/Jira tickets and Slack/Teams threads, plus API integration for building custom workflow triggers, but there's no explicit mention of GitHub or GitLab issue/PR-based task kickoff in the evidence pack. Community evidence doesn't directly contradict the integration claims, only general effectiveness concerns. Missing for 10: explicit GitHub/GitLab-triggered task creation documentation, independent hands-on confirmation of these specific integrations working.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Tagging Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”

   proof ↗⚑ flag

Cross device continuity

1. developerStart a task on one device and continue it later from another device or browser

   weight 2 · round to Devin
   Gemini CLI—none0/10

   Gemini CLI offers local conversation checkpointing to save/resume sessions (gh-8) and can run in Cloud Shell (docs-1), but there is no evidence of cloud-synced session state that lets a developer start a task on one device/terminal and pick it up seamlessly on another device or browser. missing for 10: cross-device session sync, browser-based continuation of an existing CLI session, any documented mechanism to transfer checkpoint state between machines.

   • [github] “Conversation checkpointing to save and resume complex sessions”
   • [claimed-docs] “The Gemini CLI is available without additional setup in Cloud Shell”

   proof ↗⚑ flag

   Devin✓fullclaimed7/10

   Devin's cloud sessions run in a dedicated VM that persists independent of the local device (docs-13), can be started from Slack/Teams, IDE, CLI, or API and continued/taken over in the embedded IDE or web UI (docs-3, docs-4, docs-7, docs-24), and Devin Handoff explicitly lets you start work locally and continue in a cloud session accessible from a browser (docs-38). missing for 10: no explicit first-party walkthrough of resuming the same session from a different browser/device login, and no independent/community confirmation of cross-device continuity.

   • [claimed-docs] “Devin is designed to be a conversational user interface, and allows you to follow and take over Devin's development process in the embedded …”
   • [claimed-docs] “Devin CLI, a local command-line coding agent with deep Devin Cloud integration.”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “Hand a task off to a cloud Devin session and keep working locally.”

   proof ↗⚑ flag

Ide integration

1. developerChat with the coding assistant directly inside my IDE for contextual help

   weight 3 · round to Devin
   Gemini CLI—none0/10

   The evidence pack describes Gemini CLI as a terminal-based agent (context files, MCP servers, Cloud Shell access) but contains no mention of an IDE extension, sidebar chat, or in-editor contextual panel that would let a developer chat with it directly inside an IDE. Community threads discuss its terminal/agentic performance, not IDE integration.

   ⚑ flag

   Devin~partialclaimed5/10

   Devin exposes a conversational interface via its own embedded IDE within cloud sessions (devin-docs-3) and its Desktop app that imports VS Code/Cursor settings (devin-docs-9), plus an MCP server letting 'any MCP-compatible AI agent or IDE' access sessions (devin-docs-6/23). However there's no evidence of a native extension that lets a developer chat with Devin directly inside their own existing IDE (e.g., a VS Code/JetBrains plugin) — Devin's model is its own IDE/Desktop environment or MCP bridging rather than embedding in the user's IDE. Missing for 10: a first-party IDE extension for VS Code/JetBrains enabling in-IDE chat, and independent confirmation of this workflow working well in practice.

   • [claimed-docs] “Devin is designed to be a conversational user interface, and allows you to follow and take over Devin's development process in the embedded …”
   • [claimed-docs] “Import VS Code or Cursor settings, configure themes, and start coding with AI-powered assistance.”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”

   proof ↗⚑ flag

Session management

1. engineering-leadManage multiple agent-driven coding sessions from one unified workspace

   weight 2 · round to Devin
   Gemini CLI—none0/10

   Evidence shows single-session features (conversation checkpointing to save/resume one session, GEGEMINI.md context files) but nothing about running or coordinating multiple concurrent agent sessions from one unified dashboard/workspace for a lead overseeing a team's work. missing for 10: multi-session dashboard/orchestration UI, evidence of concurrent session management, any lead-oriented workspace view.

   • [github] “Conversation checkpointing to save and resume complex sessions”
   • [github] “Custom context files (GEMINI.md) to tailor behavior for your projects”

   proof ↗⚑ flag

   Devin~partialcommunity6/10

   Devin supports running multiple parallel sessions ('carve out independent tasks and run them simultaneously'), an API to create sessions on behalf of users, and an embedded IDE/CLI/desktop app to interact with sessions, which together enable a lead-like workspace for managing several agent sessions. However, there's no dedicated 'unified workspace' dashboard evidence for an engineering-lead specifically monitoring/managing a team's multiple concurrent sessions, and community feedback highlights session reliability issues that would complicate multi-session oversight. missing for 10: explicit multi-session dashboard/UI for a lead role, team-level session oversight features, independent corroboration of smooth multi-session management at scale.

   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “Devin is designed to be a conversational user interface, and allows you to follow and take over Devin's development process in the embedded …”
   • [claimed-docs] “Devin CLI, a local command-line coding agent with deep Devin Cloud integration.”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

Terminal workflow

1. developerRun a coding agent locally from my terminal

   weight 3 · round drawn
   Gemini CLI✓fullcommunity8/10

   Gemini CLI is a terminal-native coding agent with first-party docs (gemini-cli-gh-1 through -20, gemini-cli-docs-1/2/3) describing running locally, querying/editing codebases, non-interactive scripting, and Cloud Shell availability with no extra setup, and abundant community evidence (gemini-cli-comm-1, -9, -12) confirms real-world local terminal usage. Missing for 10: independent benchmark of reliability (several community reports of agentic failures/loops, e.g. gemini-cli-comm-10, -11, -14) and no first-party install/runtime docs beyond GitHub README excerpts.

   • [github] “Query and edit large codebases”
   • [github] “Run non-interactively in scripts for workflow automation”
   • [github] “gemini --include-directories ../lib,../docs”
   • [claimed-docs] “The Gemini CLI is available without additional setup in Cloud Shell”
   • [community] “I have been using this for about a month and it's a beast, mostly thanks to 2.5pro being SOTA and how it leverages that huge 1M context wind…”
   • [community] “The correct way of using Gemini CLI is: ABUSE IT! With 1M Context Window (soon 2M) and generous daily free quota are huge advantages.”

   proof ↗⚑ flag

   Devin✓fullprobed8/10

   Devin CLI is explicitly documented as a local command-line coding agent that can be invoked from terminal (e.g. `devin -- check out this code...`), with local sandboxing (--sandbox flag) and deep integration with Devin Cloud for handoff. Missing for 10: independent/hands-on community verification specifically of the CLI experience (community evidence only covers the cloud/browser Devin product, not the local CLI).

   • [claimed-docs] “Devin CLI, a local command-line coding agent with deep Devin Cloud integration.”
   • [claimed-docs] “devin -- check out this code and suggest a feasible, helpful feature”
   • [claimed-docs] “a local command-line coding agent with deep Devin Cloud integration”
   • [claimed-docs] “The --sandbox flag runs the CLI with OS-level isolation, enforcing writable paths and deny rules at the operating-system level and optionall…”
   • [claimed-docs] “The `--sandbox` flag runs the CLI with OS-level isolation, enforcing writable paths and `deny` rules at the operating-system level”
   • [probe] “official CLI documented at https://docs.devin.ai/cli/index”

   proof ↗⚑ flag

2. developerRun the agent non-interactively in scripts for workflow automation

   weight 2 · round to Gemini CLI
   Gemini CLI✓fullclaimed9/10

   Gemini CLI explicitly documents non-interactive scripting support with structured output flags (--output-format json / stream-json) and lists 'Run non-interactively in scripts for workflow automation' as a core feature; GitHub Actions integration for PR review/issue triage further evidences automation use cases. Missing for 10: independent hands-on validation specifically of scripting/automation workflows (community feedback focuses on interactive agent quality, not scripted use).

   • [github] “Run non-interactively in scripts for workflow automation”
   • [github] “use the `--output-format json` flag to get structured output”
   • [github] “use `--output-format stream-json` to get newline-delimited JSON events”
   • [github] “Pull Request Reviews: Automated code review with contextual feedback and suggestions”
   • [github] “Issue Triage: Automated labeling and prioritization of GitHub issues based on content analysis”

   proof ↗⚑ flag

   Devin✓fullclaimed7/10

   Devin exposes a documented API for creating sessions programmatically (including on behalf of users) and explicit CI/CD pipeline integration to auto-respond to static-analysis findings, plus a CLI (`devin -- <prompt>`) that can be invoked headlessly, all pointing to non-interactive, scriptable automation. Missing for 10: independent/hands-on confirmation of headless CLI scripting in real CI pipelines and more detail on CLI exit codes/output for scripting.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Devin CLI, a local command-line coding agent with deep Devin Cloud integration.”
   • [claimed-docs] “devin -- check out this code and suggest a feasible, helpful feature”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “a local command-line coding agent with deep Devin Cloud integration”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round drawn
   Gemini CLI~partialprobed5/10

   Gemini CLI supports non-interactive scripting and structured JSON/stream-JSON output (gh-6, gh-17, gh-18), suggesting most interactive capabilities can be invoked programmatically for automation. However, there's no explicit documentation confirming full feature parity between interactive sessions and scripted/API use, and probes found no formal API/OpenAPI spec (probe-1, probe-2), so completeness of parity is unverified. Missing for 10: explicit parity documentation, a formal API surface beyond CLI flags, and independent confirmation that all UI/interactive features (e.g., checkpointing, MCP tool use) are scriptable identically.

   • [github] “Run non-interactively in scripts for workflow automation”
   • [github] “use the `--output-format json` flag to get structured output”
   • [github] “use `--output-format stream-json` to get newline-delimited JSON events”
   • [probe] “PROBE llms.txt: HTTP 404 at https://developers.google.com/llms.txt”
   • [probe] “PROBE openapi: all candidate paths 404 (https://developers.google.com/openapi.json, https://developers.google.com/swagger.json, https://deve…”

   proof ↗⚑ flag

   Devin~partialprobed5/10

   Devin exposes a documented API (session creation, create_as_user_id) and an MCP server giving 'full access to session management, playbooks, knowledge, and scheduling', suggesting broad programmatic parity, but there's no explicit claim of full UI/API feature parity, and probes for an OpenAPI spec all 404'd, indicating the full API surface isn't transparently documented. Missing for 10: explicit parity statement covering UI-only features like Devin Review/Auto-Fix/Computer Use/desktop app settings, a discoverable OpenAPI schema, and independent confirmation that all UI actions are API-reachable.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round drawn
   Gemini CLI—none0/10

   No evidence of any data export feature or open-format data portability in Gemini CLI; the tool is a local coding agent that reads/writes local files but nothing indicates exporting conversation history, settings, or usage data in an open format for user-controlled exit. Probes for llms.txt/openapi also failed, showing no structured data-access surface.

   • [probe] “PROBE llms.txt: HTTP 404 at https://developers.google.com/llms.txt”
   • [probe] “PROBE openapi: all candidate paths 404 (https://developers.google.com/openapi.json, https://developers.google.com/swagger.json, https://deve…”

   proof ↗⚑ flag

   Devin—none0/10

   No evidence of a data export feature or open-format export/portability mechanism for Devin sessions, knowledge, or artifacts; the docs cover API, CLI, MCP, and infrastructure but nothing about exporting all user data in open formats to leave the platform.

   ⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round to Gemini CLI
   Gemini CLI~partialprobed5/10

   The product's source is hosted publicly at github.com/google-gemini/gemini-cli (referenced repeatedly across the evidence pack), implying open availability for reading, but no citation in the evidence pack explicitly names or confirms an open-source license (e.g., Apache/MIT) or points to a LICENSE file. Missing for 10: explicit license text/citation, confirmation of license type, and any independent corroboration of open-license terms.

   • [github] “Query and edit large codebases”
   • [github] “Use `/bug` command to report issues directly from the CLI.”
   • [probe] “official CLI documented at https://developers.google.com/gemini-code-assist/docs/gemini-cli”

   proof ↗⚑ flag

   Devin—none0/10

   Devin is a closed, proprietary commercial product; no evidence indicates its source code is available under an open license. The only mention of 'open-source' is for the separate Devin Handoff plugin/skill, not Devin itself.

   • [claimed-docs] “Devin Handoff is an open-source plugin and skill that brings the same handoff workflow to any coding agent — Claude Code, Codex, Cursor, and…”

   proof ↗⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round drawn
   Gemini CLI—none0/10

   Gemini CLI is an open-source client, but the core product (the Gemini models/backend) is a Google-hosted cloud service accessed via Google account sign-in; no evidence anywhere in the pack describes a self-hosted or on-prem deployment option for the core model/service.

   • [github] “No API key management - just sign in with your Google account”
   • [claimed-docs] “The Gemini CLI is available without additional setup in Cloud Shell”

   proof ↗⚑ flag

   Devin—none0/10

   Devin is a cloud-based SaaS agent; Outposts lets you run sessions on your own infrastructure but the core Devin model/orchestration itself remains Cognition-hosted, and there's no evidence of a self-hostable core product/model package. No mention of on-prem/self-hosted deployment of the core Devin engine.

   • [claimed-docs] “Outposts lets you run Devin sessions inside infrastructure you control — your own VMs, containers, Kubernetes clusters, or even a Mac Mini o…”

   proof ↗⚑ flag

Authentication

1. developerAuthenticate with an API key instead of an account login

   weight 2 · round to Devin
   Gemini CLI~partialclaimed4/10

   The docs emphasize signing in with a Google account (gh-13) as the primary flow, but they also note that developers needing 'specific model control or paid tier access' (gh-24) have an alternative path, implying API-key-based auth exists without detailing it. There's no explicit example or setup instructions for API-key authentication itself. Missing for 10: explicit API key env-var/config documentation, first-party steps for key-based auth, and independent confirmation it works without Google login.

   • [github] “No API key management - just sign in with your Google account”
   • [github] “Developers who need specific model control or paid tier access”

   proof ↗⚑ flag

   Devin~partialprobed6/10

   Devin has a documented API (devin-docs-7) intended for programmatic integration, implying API key auth as an alternative to account login, and even supports creating sessions on behalf of other users (devin-docs-8), suggesting a service-level credential model. However, no explicit documentation of API key generation/management or authentication mechanics is present in the evidence pack. Missing for 10: explicit API key creation/management docs, confirmation that API key auth fully replaces login flows, and independent/community verification of this workflow.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

2. engineering-leadAuthenticate through an enterprise identity or cloud platform for compliance and scalability

   weight 2 · round to Gemini CLI
   Gemini CLI!disputedcontradicted4/10

   Google claims 'Enterprise features: Advanced security and compliance' and frictionless Google-account sign-in without API key management, plus Cloud Shell availability, suggesting cloud/enterprise identity support. However, a hands-on community report shows authentication explicitly failing for Workspace (enterprise) accounts ('Failed to login. Ensure your Google account is not a Workspace account'), directly contradicting the enterprise-identity claim for a core scenario. Missing for 10: documented enterprise SSO/IAM integration details, confirmation Workspace login issue is resolved, and independent verification of compliance certifications.

   • [github] “No API key management - just sign in with your Google account”
   • [github] “Enterprise features: Advanced security and compliance”
   • [claimed-docs] “The Gemini CLI is available without additional setup in Cloud Shell”
   • [community] “'Failed to login. Ensure your Google account is not a Workspace account.' I have had a Workspace account since GSuite and now as a Workspace…”

   proof ↗⚑ flag

   Devin—none0/10

   No evidence of SSO/SAML, OIDC, or enterprise identity provider integration (e.g., Okta, Azure AD, Google Workspace) for authentication; the only related item is API session creation on behalf of a user, which is not enterprise identity/SSO authentication.

   ⚑ flag

3. developerSign in with my existing product subscription plan to use the coding agent

   weight 2 · round to Gemini CLI
   Gemini CLI!disputedcontradicted4/10

   Gemini CLI advertises frictionless Google-account sign-in with no API key management (gh-13), suggesting subscribers could just log in and go, but hands-on community reports concretely contradict this: a Gemini Pro subscriber found that paying for 'Gemini' doesn't unlock Gemini CLI usage, requiring a separate 'Gemini Code Assist Standard/Enterprise' plan, and another user explicitly asks for one unified subscription across CLI, Code Assist, Jules, etc. like Claude's Max plan. Missing for 10: evidence that an existing Google One/Gemini Advanced subscription actually raises CLI usage limits, and resolution of the reported subscription fragmentation.

   • [github] “No API key management - just sign in with your Google account”
   • [community] “I love how fragmented Google's Gemini offerings are. I'm a Pro subscriber but I learn I should be a 'Gemini Code Assist Standard or Enterpri…”
   • [community] “Again, with the complicated subscription. Please just give us a monthly subscription for developers that I can pay whatever, and then use Ge…”
   • [github] “Developers who need specific model control or paid tier access”

   proof ↗⚑ flag

   Devin—none0/10

   The evidence describes Devin's own subscription tiers (Pro, Max, Core/Team/Enterprise) but nothing about letting a developer sign in using an existing external subscription (e.g., an existing LLM provider or IDE subscription) to access the agent. No mention of SSO-linked billing or bring-your-own-subscription support.

   • [claimed-docs] “Max is for individual users who consistently exceed the Pro quota. It includes everything in Pro, plus a significantly larger weekly usage q…”
   • [claimed-docs] “Power users who need more quota”

   proof ↗⚑ flag

4. developerSign in with a personal account to get free-tier access without managing API keys

   weight 1 · round to Gemini CLI
   Gemini CLI✓fullcommunity7/10

   GitHub docs explicitly state 'No API key management - just sign in with your Google account' (gemini-cli-gh-13), directly matching the story, and Cloud Shell docs describe zero-setup access. Community reports don't dispute personal-account sign-in itself (the failure noted is specific to Workspace accounts, an edge case outside 'personal account'), though some users voice confusion over how free vs paid tiers interact. Missing for 10: independent/hands-on confirmation of the free-tier quota limits and clearer documentation distinguishing personal free-tier access from paid Code Assist tiers.

   • [github] “No API key management - just sign in with your Google account”
   • [claimed-docs] “The Gemini CLI is available without additional setup in Cloud Shell”
   • [community] “I love how fragmented Google's Gemini offerings are. I'm a Pro subscriber but I learn I should be a 'Gemini Code Assist Standard or Enterpri…”
   • [community] “'Failed to login. Ensure your Google account is not a Workspace account.' I have had a Workspace account since GSuite and now as a Workspace…”

   proof ↗⚑ flag

   Devin—none0/10

   Evidence only describes paid tiers (Pro, Max, 'power users who need more quota') and API key based integration; there's no mention of a free tier accessible via personal account sign-in without API key management.

   • [claimed-docs] “Max is for individual users who consistently exceed the Pro quota. It includes everything in Pro, plus a significantly larger weekly usage q…”
   • [claimed-docs] “Power users who need more quota”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”

   proof ↗⚑ flag

Model choice

1. developerLet the tool automatically pick the best model for each task

   weight 1 · round drawn
   Gemini CLI—none0/10

   Evidence shows manual model selection ('Choose specific Gemini models' for 'developers who need specific model control') rather than automatic task-based model selection; no evidence of the CLI auto-choosing the optimal model per task.

   • [github] “Model selection: Choose specific Gemini models”
   • [github] “Developers who need specific model control or paid tier access”

   proof ↗⚑ flag

   Devin—none0/10

   No evidence that Devin offers automatic model selection or model-routing based on task complexity; docs discuss quotas, tiers (Pro/Max), and features but never mention choosing among multiple underlying models per task.

   ⚑ flag

2. developerChoose which underlying AI model powers my session from multiple providers

   weight 2 · round drawn
   Gemini CLI—none0/10

   Evidence shows Gemini CLI only supports choosing among Google's own Gemini models (gh-14, gh-24), not switching between different AI providers (e.g., OpenAI, Anthropic); community complaints (comm-2, comm-3, comm-19) reinforce that it's locked to Google's ecosystem/billing. There is no evidence of multi-provider model selection, so the story as written (choosing from multiple providers) is not delivered.

   • [github] “Model selection: Choose specific Gemini models”
   • [github] “Developers who need specific model control or paid tier access”
   • [community] “The killer feature of Claude Code is that you can just pay for Max and not worry about API billing. Until Gemini does that, I'm sticking wit…”
   • [community] “I love how fragmented Google's Gemini offerings are. I'm a Pro subscriber but I learn I should be a 'Gemini Code Assist Standard or Enterpri…”
   • [community] “Again, with the complicated subscription. Please just give us a monthly subscription for developers that I can pay whatever, and then use Ge…”

   proof ↗⚑ flag

   Devin—none0/10

   No evidence in the pack mentions selecting or switching between multiple underlying AI model providers (e.g., choosing between different LLMs) for a Devin session; documentation covers agent capabilities, environments, and billing tiers but not model selection.

   ⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round drawn
   Gemini CLI—none0/10

   No evidence in the pack mentions data residency, regional storage options, or configurable data location controls for Gemini CLI; only enterprise 'security and compliance' features are mentioned generically without specifics.

   ⚑ flag

   Devin—none0/10

   No evidence of data residency/region selection controls for where Devin stores data; the pack covers deployment options (Outposts, VPN, sandbox isolation) but nothing about choosing storage region/jurisdiction.

   ⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   Gemini CLI—none0/10

   No evidence in the pack addresses data usage/training opt-out policies, privacy controls, or terms governing whether user data trains Google's models; only unrelated feature/community commentary is present. Missing for 10: explicit data-usage/training policy documentation, opt-out mechanism, enterprise/no-training guarantee.

   ⚑ flag

   Devin—none0/10

   The evidence pack contains no mention of data-training opt-out, data usage policy, or privacy controls for AI model training; nothing in the docs or community sources addresses this axis.

   ⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round drawn
   Gemini CLI—none0/10

   No evidence pack items address data retention controls, deletion mechanisms, or privacy settings for Gemini CLI; only enterprise 'security and compliance' is vaguely mentioned without specifics. Missing for 10: documentation on data retention policy, user-controlled deletion mechanism, opt-out of data collection, and any privacy settings UI/CLI flags.

   ⚑ flag

   Devin—none0/10

   No evidence pack items address data retention policies, data deletion controls, or privacy/data-handling settings for user data or session data; the closest items concern sandboxing/isolation of execution environments, not retention/deletion of stored data.

   ⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Gemini CLI—none0/10

   No evidence in the pack mentions telemetry settings, opt-out flags, or usage-data collection policy for Gemini CLI. Missing for 10: any documentation of a telemetry/usage-tracking toggle, privacy settings, or opt-out mechanism.

   ⚑ flag

   Devin—none0/10

   No evidence pack item mentions telemetry opt-out, usage tracking settings, or privacy controls of that kind; the docs cover features, sandboxing, and infrastructure but not data-collection opt-out mechanisms.

   ⚑ flag

Data governance

1. engineering-leadOpt out of having my code and prompts used for AI model training

   weight 1 · round drawn
   Gemini CLI—none0/10

   No evidence in the pack addresses data usage, training opt-out policies, or privacy controls for Gemini CLI; only feature lists and general community sentiment are present. missing for 10: any documentation of data usage/training policy, opt-out settings or enterprise privacy controls, and independent confirmation of such settings working.

   ⚑ flag

   Devin—none0/10

   No evidence in the pack addresses data usage, training opt-out, or privacy/data-handling policies for Devin; nothing documents an opt-out mechanism for code/prompt use in model training.

   ⚑ flag

Pr review

1. developerHave the agent stage changes, write commit messages, create branches, and open pull requests

   weight 3 · round to Devin
   Gemini CLI~partialclaimed4/10

   Evidence shows Gemini CLI can automate git-related operational tasks like querying pull requests and handling complex rebases, and its GitHub Action can do automated PR reviews and issue triage, but there's no explicit documentation of the agent staging changes, writing commit messages, creating branches, or opening new pull requests itself. Missing for 10: explicit commit-message generation, branch creation, and PR-opening workflow evidence, plus independent confirmation these work reliably.

   • [github] “Automate operational tasks like querying pull requests or handling complex rebases”
   • [github] “Pull Request Reviews: Automated code review with contextual feedback and suggestions”
   • [github] “@github List my open pull requests”
   • [github] “On-demand Assistance: Mention @gemini-cli in issues and pull requests for help with debugging, explanations, or task delegation”

   proof ↗⚑ flag

   Devin~partialcommunity6/10

   Devin's docs describe it implementing features/fixing bugs and producing pull requests that get automated review and iteration (devin-docs-17, devin-docs-18), implying it handles the full git workflow (branch, commit, PR) autonomously, but no doc explicitly details staging, commit-message generation, or branch creation as discrete steps. Community feedback (devin-comm-1) also notes it can add extraneous changes it can't cleanly undo, a real caveat on commit hygiene. Missing for 10: explicit documentation of commit/staging/branch mechanics and independent confirmation that generated commits/PRs are clean and reviewable.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Devin Review provides automated first-pass reviews on pull requests, checking for correctness and conformance with organizational best pract…”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”

   proof ↗⚑ flag

2. developerGet automatic code review with contextual feedback on every pull request

   weight 3 · round drawn
   Gemini CLI✓fullcommunity8/10

   Gemini CLI's GitHub Actions integration explicitly provides automated PR code review with contextual feedback and suggestions, plus on-demand @gemini-cli assistance in PRs, and community reports corroborate favorable code review quality compared to competitors. Missing for 10: independent hands-on verification of the PR-review workflow specifically (most community feedback covers general CLI agentic use rather than the PR-review action itself), and no detail on configurability/false-positive rates.

   • [github] “Pull Request Reviews: Automated code review with contextual feedback and suggestions”
   • [github] “Issue Triage: Automated labeling and prioritization of GitHub issues based on content analysis”
   • [github] “On-demand Assistance: Mention @gemini-cli in issues and pull requests for help with debugging, explanations, or task delegation”
   • [github] “On-demand Assistance: Mention `@gemini-cli` in issues and pull requests for help with debugging, explanations, or task delegation”
   • [github] “@github List my open pull requests”
   • [community] “We have tried out Gemini code review vs Copilot code review and Gemini is consistently offering better code review tips. It has officially c…”

   proof ↗⚑ flag

   Devin✓fullclaimed8/10

   Devin Review is explicitly documented as an automated first-pass PR reviewer checking correctness and org best-practice conformance, with Auto-Fix closing the loop by responding to review comments and CI failures. This directly matches the story of automatic contextual code review on PRs, though evidence is vendor-documentation only with no independent hands-on validation of review quality/contextual accuracy. Missing for 10: independent/community corroboration of review quality, and detail on how 'contextual feedback' is surfaced per-PR beyond docs description.

   • [claimed-docs] “Devin Review provides automated first-pass reviews on pull requests, checking for correctness and conformance with organizational best pract…”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixe”

   proof ↗⚑ flag

3. developerInspect diffs and run checks to catch problems before merging

   weight 3 · round to Devin
   Gemini CLI~partialcommunity6/10

   Gemini CLI supports GitHub PR review automation with contextual feedback (gemini-cli-gh-10) and issue triage, plus community reports confirm it catches bugs reviewers missed (gemini-cli-comm-20), supporting diff inspection and pre-merge checks. However, there's no dedicated diff-viewing UI or built-in test/lint-running check suite documented, and community reports raise concerns about reliability, security prompts, and agentic mistakes (gemini-cli-comm-14, gemini-cli-comm-17). missing for 10: dedicated diff-inspection UI/commands, built-in CI/test-running integration, and stronger independent corroboration of reliability for pre-merge checks.

   • [github] “Pull Request Reviews: Automated code review with contextual feedback and suggestions”
   • [github] “Issue Triage: Automated labeling and prioritization of GitHub issues based on content analysis”
   • [github] “Automate operational tasks like querying pull requests or handling complex rebases”
   • [community] “We have tried out Gemini code review vs Copilot code review and Gemini is consistently offering better code review tips. It has officially c…”
   • [community] “The problem is that Gemini CLI simply doesn't work. Beside simplest tasks like creating a new release it is useless as a coding assistant. D…”
   • [community] “However, it does seem that Gemini pays less attention to security than Claude Code. Gemini will happily open in my root directory. Claude Co…”

   proof ↗⚑ flag

   Devin✓fullclaimed8/10

   Devin's docs describe Devin Review, which performs automated first-pass PR reviews checking correctness and org conformance, plus CI/CD integration to respond to static-analysis findings (SonarQube, Fortify, Veracode) and Auto-Fix iterating on CI failures — directly enabling developers to inspect diffs and run checks before merge. This is corroborated by explicit SDLC integration workflow docs, not just a single mention. missing for 10: independent/hands-on evidence confirming Devin Review's diff-inspection quality in practice, and detail on how diffs are surfaced/inspected by the developer (UI specifics) beyond docs claims.

   • [claimed-docs] “Devin Review provides automated first-pass reviews on pull requests, checking for correctness and conformance with organizational best pract…”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixe”

   proof ↗⚑ flag

Safe execution

1. engineering-leadControl which external tools and integrations the agent is allowed to access

   weight 2 · round to Devin
   Gemini CLI~partialcommunity5/10

   Gemini CLI supports configuring MCP servers via ~/.gemini/settings.json and exposes /tools and /mcp commands to inspect and manage available tools, giving engineering leads some control over which integrations are enabled. However, evidence lacks any centralized admin/policy control, allowlist/denylist enforcement, or org-wide governance mechanism for restricting tool access across a team, and community reports note weak security defaults (e.g. opening root directories without prompting). missing for 10: org-level/admin enforcement of tool allowlists, granular permission scoping per tool/integration, independent verification that access controls are robust rather than just configurable per-user.

   • [github] “Configure MCP servers in ~/.gemini/settings.json to extend Gemini CLI with custom tools”
   • [claimed-docs] “Comandos de Gemini CLI: /memory, /stats, /tools y /mcp”
   • [community] “However, it does seem that Gemini pays less attention to security than Claude Code. Gemini will happily open in my root directory. Claude Co…”

   proof ↗⚑ flag

   Devin~partialclaimed6/10

   Devin exposes some admin-level controls over its access—an org-wide 'Enable desktop mode' toggle for Computer Use, a CLI --sandbox flag enforcing OS-level writable-path/deny rules and network restriction, and Outposts for running sessions in infra you control—giving leads levers to constrain what Devin can reach or do. However there's no documented centralized policy/allowlist for specific external integrations (e.g., disabling Slack, Jira, GitHub, VPN access per-tool) or granular permission/audit management for engineering leads. missing for 10: a unified integration-permission/allowlist admin panel, per-tool enable/disable controls beyond desktop mode, and independent verification of these controls in practice.

   • [claimed-docs] “Computer Use is controlled by the Enable desktop mode toggle in your organization's customization options.”
   • [claimed-docs] “The --sandbox flag runs the CLI with OS-level isolation, enforcing writable paths and deny rules at the operating-system level and optionall…”
   • [claimed-docs] “The `--sandbox` flag runs the CLI with OS-level isolation, enforcing writable paths and `deny` rules at the operating-system level”
   • [claimed-docs] “Outposts lets you run Devin sessions inside infrastructure you control — your own VMs, containers, Kubernetes clusters, or even a Mac Mini o…”
   • [claimed-docs] “Devin can connect to a VPN from inside its workspace, so sessions can reach internal services such as package registries, databases, and int…”

   proof ↗⚑ flag

2. engineering-leadHave the agent operate inside a sandbox when interacting with code, tools, and network resources

   weight 2 · round to Devin
   Gemini CLI—none0/10

   The evidence pack contains no vendor documentation of a sandboxed execution mode for code/tool/network interactions—only a vague 'Enterprise features: Advanced security and compliance' bullet with no detail. Community evidence actually points the other way: reviewers note Gemini CLI 'happily opens in my root directory' without any directory-trust prompt, unlike Claude Code, and one report describes it destructively running file-system commands, suggesting a lack of sandboxing guardrails rather than presence of them.

   • [github] “Enterprise features: Advanced security and compliance”
   • [community] “However, it does seem that Gemini pays less attention to security than Claude Code. Gemini will happily open in my root directory. Claude Co…”
   • [community] “Gemini told the user: 'I have failed you completely and catastrophically... I have lost your data. This is an unacceptable, irreversible fai…”

   proof ↗⚑ flag

   Devin✓fullclaimed8/10

   Devin runs cloud sessions in isolated VMs (devin-docs-13) and provides an explicit CLI --sandbox flag enforcing OS-level isolation, writable path restrictions, deny rules, and optional network restriction (devin-docs-12, devin-docs-37), directly matching the sandboxed code/tool/network isolation story. Missing for 10: independent/hands-on verification of sandbox robustness and more detail on network isolation guarantees beyond docs claims.

   • [claimed-docs] “The --sandbox flag runs the CLI with OS-level isolation, enforcing writable paths and deny rules at the operating-system level and optionall…”
   • [claimed-docs] “The `--sandbox` flag runs the CLI with OS-level isolation, enforcing writable paths and `deny` rules at the operating-system level”
   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “Devin can connect to a VPN from inside its workspace, so sessions can reach internal services such as package registries, databases, and int…”

   proof ↗⚑ flag

Security checks

1. engineering-leadSee license and public-code matching references for AI-suggested code

   weight 1 · round drawn
   Gemini CLI—none0/10

   No evidence anywhere in the pack of license attribution, public-code matching, or provenance references for AI-suggested code; features listed cover code editing, review, MCP, PR automation, etc. but nothing about license/originality detection.

   ⚑ flag

   Devin—none0/10

   No evidence of license or public-code matching/attribution references for AI-suggested code anywhere in the docs; Devin Review covers correctness and best-practice conformance but nothing about license/provenance matching.

   ⚑ flag

2. developerGet contextual explanations and automatic fixes for security vulnerabilities

   weight 2 · round to Devin
   Gemini CLI~partialclaimed4/10

   Gemini CLI offers general debugging/explanation via natural language (gh-3, gh-12) and automated PR review with 'contextual feedback and suggestions' (gh-10), plus vague 'enterprise advanced security and compliance' (gh-15), which could incidentally surface and explain security issues, but there is no evidence of a dedicated vulnerability-scanning or automatic-fix feature specifically for security flaws. Missing for 10: explicit vulnerability detection/scanning capability, documented automatic remediation of security issues, and independent verification that PR reviews catch/fix security vulnerabilities specifically.

   • [github] “Debug issues and troubleshoot with natural language”
   • [github] “Pull Request Reviews: Automated code review with contextual feedback and suggestions”
   • [github] “On-demand Assistance: Mention @gemini-cli in issues and pull requests for help with debugging, explanations, or task delegation”
   • [github] “Enterprise features: Advanced security and compliance”

   proof ↗⚑ flag

   Devin✓fullclaimed7/10

   Devin's docs describe Devin Review giving automated PR reviews with explanations for correctness/best-practice issues, Auto-Fix automatically responding to review comments and fixing flagged bugs/CI failures, and CI/CD integration to respond to findings from security scanners like SonarQube, Fortify, and Veracode — directly matching contextual explanation plus automatic fixing of vulnerabilities. Missing for 10: independent/hands-on evidence specifically validating security-vulnerability fixes (community evidence only discusses general reliability/scope-creep issues, not security-fix accuracy).

   • [claimed-docs] “Devin Review provides automated first-pass reviews on pull requests, checking for correctness and conformance with organizational best pract…”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Ask Devin can answer questions about code structure and dependencies, and help you scope and plan tasks before implementation.”

   proof ↗⚑ flag