Codex vs Devin

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to Devin
   Codex~partialprobed4/10

   A dedicated llms.txt file is absent (404 at platform.openai.com/llms.txt), but Codex does publish machine-readable markdown docs (learn.chatgpt.com/docs/codex/cli.md) confirmed reachable by probe, which is an agent-friendly doc format an AI agent could be pointed at. Missing for 10: a standard llms.txt manifest, evidence of agents actually being pointed at these docs, and confirmation across all doc pages (docs/codex.md also 404s).

   • [probe] “PROBE llms.txt: HTTP 404 at https://platform.openai.com/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://platform.openai.com/docs/codex.md”
   • [probe] “official CLI documented at https://learn.chatgpt.com/docs/codex/cli”

   proof ↗⚑ flag

   Devin✓fullprobed9/10

   Devin's own docs site serves llms.txt (HTTP 200, confirmed by probe) and per-page .md variants, and also supports AGENTS.md as an agent-oriented instructions standard, directly matching the story. Missing for 10: no independent/community confirmation that external agents have actually consumed llms.txt successfully.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.devin.ai/llms.txt # Devin Docs - [Desktop (100 pages)](https://docs.devin.ai/_llms/en/desktop.md):…”
   • [probe] “PROBE docs-md: HTTP 200 at https://docs.devin.ai/get-started/devin-intro.md > ## Documentation Index > Fetch the complete documentation inde…”
   • [claimed-docs] “Devin supports AGENTS.md - a simple, open standard for providing context and instructions to AI agents.”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round drawn
   Codex✓fullclaimed8/10

   Codex CLI explicitly documents non-interactive execution via `codex exec` for use in repeatable workflows, scripts, and CI/CD pipelines (codex-docs-19, codex-docs-32), and permissions/sandbox controls can be configured for unattended runs (codex-docs-17, codex-docs-39). Missing for 10: no independent case study or CI provider (e.g. GitHub Actions) integration example, and no explicit exit-code/output-format spec for CI parsing.

   • [claimed-docs] “Run a non-interactive command in a repeatable workflow.”
   • [claimed-docs] “Compose with scripts and CI: Use Codex interactively or call codex exec from repeatable workflows and pipelines.”
   • [claimed-docs] “Choose when Codex can edit files or run commands without asking, and inspect the active sandbox and writable roots before you continue.”
   • [claimed-docs] “Set the boundaries for each run — /permissions: Choose when Codex can edit files or run commands without asking, and inspect the active sand…”

   proof ↗⚑ flag

   Devin✓fullclaimed8/10

   Devin offers a full API for creating/managing sessions programmatically (including create_as_user_id for automation on behalf of users), a CLI with a --sandbox flag for OS-level isolated headless runs, and explicit CI/CD pipeline integration for responding to static analysis findings and PR checks, all supporting headless/automated usage without a human in the loop. Missing for 10: no independent/hands-on report specifically validating CI automation workflows end-to-end, and no explicit CI example (e.g., GitHub Actions snippet) beyond doc references.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “The --sandbox flag runs the CLI with OS-level isolation, enforcing writable paths and deny rules at the operating-system level and optionall…”
   • [claimed-docs] “a local command-line coding agent with deep Devin Cloud integration”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round to Codex
   Codex✓fullclaimed9/10

   Codex CLI explicitly supports adding local/remote MCP servers via `codex mcp add`, inspecting available tools before use, and viewing active servers via `/mcp`; this configuration is shared across ChatGPT desktop app, CLI, and IDE extension. Docs also describe using MCP to connect to third-party tools like browsers or Figma. Missing for 10: independent hands-on verification of MCP tool usage in a real session beyond first-party docs.

   • [claimed-docs] “Add local or remote MCP servers, authenticate when needed, and inspect the tools available to the current session before Codex uses them.”
   • [claimed-docs] “The ChatGPT desktop app, Codex CLI, and IDE extension share this configuration. Once you configure your MCP servers, you can switch among th…”
   • [claimed-docs] “Connect external tools with MCP — codex mcp: Add local or remote MCP servers, authenticate when needed, and inspect the tools available to t…”
   • [claimed-docs] “Model Context Protocol (MCP) connects models to tools and context. Use it to give ChatGPT or Codex access to third-party documentation, or t…”
   • [claimed-docs] “codex mcp add <server-name> --env VAR1=VALUE1 --env VAR2=VALUE2 -- <stdio server-command>”
   • [claimed-docs] “In the `codex` TUI, use `/mcp` to see your active MCP servers.”

   proof ↗⚑ flag

   Devin—none0/10

   The evidence only documents Devin exposing its own MCP server so other agents/IDEs can call Devin's tools (session management, playbooks, knowledge, scheduling) — the reverse direction of this story. There is no evidence that a user can configure Devin itself to consume/plug in external MCP servers so Devin can use their tools.

   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [probe] “official MCP server documented at https://docs.devin.ai/work-with-devin/devin-mcp”

   proof ↗⚑ flag

4. ai-native userConnect an agent via an official MCP server

   weight 3 · round to Devin
   Codex~partialclaimed5/10

   Codex explicitly supports running itself as an MCP server (codex mcp-server) so other MCP clients can connect, but OpenAI's own docs mark this interface 'experimental' and now 'deprecated', pointing users to a newer 'Codex app server' as the recommended replacement. This is a genuine server-mode capability (not just Codex-as-MCP-client), but the deprecation and lack of independent hands-on confirmation of the replacement's stability keep it from a full verdict. Missing for 10: independent corroboration that the current 'Codex app server' MCP mode works reliably in production, and clearer first-party documentation of its interface now that the original is deprecated.

   • [github] “Codex MCP Server Interface [experimental]: a JSON-RPC API that runs over the Model Context Protocol (MCP) transport to control a local Codex…”
   • [claimed-docs] “codex mcp-server is deprecated. Use the Codex app server instead. ... This page documents the deprecated command for existing integrations. …”
   • [claimed-docs] “Add local or remote MCP servers, authenticate when needed, and inspect the tools available to the current session before Codex uses them.”

   proof ↗⚑ flag

   Devin✓fullprobed8/10

   Devin ships an official documented MCP server (devin-mcp) that gives any MCP-compatible agent or IDE full access to session management, playbooks, knowledge, and scheduling, confirmed both in docs and via probe. Missing for 10: independent/hands-on third-party corroboration of the MCP server working in practice, and detail on setup/auth specifics.

   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [probe] “official MCP server documented at https://docs.devin.ai/work-with-devin/devin-mcp”

   proof ↗⚑ flag

5. ai-native userUse an official CLI

   weight 2 · round to Codex
   Codex✓fullprobed9/10

   Codex ships an official, well-documented CLI (npm install -g @openai/codex) with rich agentic capabilities: local repo editing, exec/non-interactive scripting, MCP support, subagents, image input, sandbox/permissions control, cloud task delegation, and shell completions — all first-party documented and confirmed via GitHub repo and docs. Missing for 10: independent hands-on benchmarking specifically of CLI workflows (community evidence focuses mostly on model quality/UX rather than CLI mechanics) and some Linux-specific gaps noted by users.

   • [github] “Codex CLI is a coding agent from OpenAI that runs locally on your computer.”
   • [github] “npm install -g @openai/codex”
   • [claimed-docs] “Inspect code, make changes, run commands, and automate repeatable work without leaving your terminal.”
   • [claimed-docs] “Run a non-interactive command in a repeatable workflow.”
   • [claimed-docs] “Compose with scripts and CI: Use Codex interactively or call codex exec from repeatable workflows and pipelines.”
   • [claimed-docs] “Connect external tools with MCP — codex mcp: Add local or remote MCP servers, authenticate when needed, and inspect the tools available to t…”
   • [claimed-docs] “Split up a larger investigation — subagents: Ask Codex to delegate focused work to specialized agents, then bring their findings back into t…”
   • [claimed-docs] “Choose when Codex can edit files or run commands without asking, and inspect the active sandbox and writable roots before you continue.”
   • [claimed-docs] “Install the Codex CLI with the standalone installer for macOS and Linux.”
   • [probe] “official CLI documented at https://learn.chatgpt.com/docs/codex/cli”

   proof ↗⚑ flag

   Devin✓fullprobed8/10

   Devin has an official CLI ('Devin CLI, a local command-line coding agent with deep Devin Cloud integration') with documented usage examples and a sandbox flag for OS-level isolation, confirmed by both docs and probe. Missing for 10: independent/hands-on community verification of the CLI specifically (community evidence covers the web/session product, not CLI usage).

   • [claimed-docs] “Devin CLI, a local command-line coding agent with deep Devin Cloud integration.”
   • [claimed-docs] “devin -- check out this code and suggest a feasible, helpful feature”
   • [claimed-docs] “The --sandbox flag runs the CLI with OS-level isolation, enforcing writable paths and deny rules at the operating-system level and optionall…”
   • [claimed-docs] “a local command-line coding agent with deep Devin Cloud integration”
   • [probe] “official CLI documented at https://docs.devin.ai/cli/index”

   proof ↗⚑ flag

6. ai-native userDrive the product through a documented public API

   weight 3 · round to Devin
   Codex~partialcommunity5/10

   Codex documents multiple programmatic entry points — an MCP server interface for JSON-RPC control (though explicitly marked deprecated/experimental in favor of an undocumented 'app server'), a non-interactive `codex exec` mode for scripts/CI, and 'API key' usage — but these come with real caveats: API-key use 'requires additional setup', the flagship gpt-5.3-codex model was reportedly not yet available via API, and the primary MCP server route is deprecated rather than a stable first-class API. missing for 10: a single stable, non-deprecated documented public API surface, confirmation that the current model is API-accessible, and independent corroboration that third parties successfully drive Codex via this API.

   • [github] “You can also use Codex with an API key, but this requires additional setup.”
   • [github] “Codex MCP Server Interface [experimental]: a JSON-RPC API that runs over the Model Context Protocol (MCP) transport to control a local Codex…”
   • [claimed-docs] “codex mcp-server is deprecated. Use the Codex app server instead. ... This page documents the deprecated command for existing integrations. …”
   • [claimed-docs] “Run a non-interactive command in a repeatable workflow.”
   • [claimed-docs] “Compose with scripts and CI: Use Codex interactively or call codex exec from repeatable workflows and pipelines.”
   • [community] “gpt-5.3-codex isn't available on the API yet — 'We are working to safely enable API access soon.'”

   proof ↗⚑ flag

   Devin✓fullprobed8/10

   Devin ships a documented public API (docs-7) with session creation, org-level features like create_as_user_id (docs-8), plus a CLI and MCP server for programmatic/agentic control (docs-4, docs-6, probe-4, probe-5), directly enabling AI-native users to drive it programmatically. Missing for 10: a discoverable OpenAPI/swagger spec (probe-3 found 404s on all candidate paths) and independent hands-on corroboration of API usage.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “Devin CLI, a local command-line coding agent with deep Devin Cloud integration.”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [probe] “official MCP server documented at https://docs.devin.ai/work-with-devin/devin-mcp”
   • [probe] “official CLI documented at https://docs.devin.ai/cli/index”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

7. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round to Codex
   Codex~partialclaimed5/10

   OpenAI's platform docs describe RBAC and project/org-scoped API keys/custom roles, and Codex can authenticate via an API key (codex-gh-4), so scoped credentials are technically available to a Codex-using account. However, none of the evidence ties this RBAC/API-key scoping specifically to configuring or restricting a Codex agent's own permissions — missing for 10: Codex-specific docs on issuing least-privilege keys for agent sessions, guidance on scoping credentials per-task/per-repo, and independent confirmation that this RBAC applies to Codex's own execution rather than just general API access.

   • [claimed-docs] “Role-based access control (RBAC) lets you decide who can do what across your organization and projects—both through the API and in the Dashb…”
   • [github] “You can also use Codex with an API key, but this requires additional setup.”
   • [claimed-docs] “Create a model response — request/response reference with runnable code samples selectable per language: HTTP, Python, TypeScript, Go, Ruby,…”

   proof ↗⚑ flag

   Devin—none0/10

   Devin exposes a general API (devin-docs-7) and can act on behalf of a specified user via create_as_user_id (devin-docs-8), but there is no evidence of scoped/least-privilege API key or token issuance, role-based permission scopes, or credential-level restriction mechanisms for agent access.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”

   proof ↗⚑ flag

8. ai-native userBuild against official SDKs

   weight 2 · round to Codex
   Codex~partialcommunity6/10

   Codex is a coding agent, but the evidence shows a genuine SDK-adjacent surface: the underlying OpenAI Responses API has an official OpenAPI spec and multi-language code samples (Python, TypeScript, Go, Ruby, Java, HTTP, CLI), and Codex integrates via CLI/MCP for programmatic extension. However, there is no evidence of an official Codex-specific SDK (as opposed to the general OpenAI API SDK), and API access for the Codex model itself is explicitly noted as not yet available. missing for 10: a dedicated Codex SDK/library distinct from the general OpenAI Responses API, confirmation that Codex agent capabilities (not just chat completions) are exposed via SDK, independent developer corroboration of building against these SDKs.

   • [claimed-docs] “Create a model response — request/response reference with runnable code samples selectable per language: HTTP, Python, TypeScript, Go, Ruby,…”
   • [github] “A machine-readable description of the OpenAI REST API, authored in OpenAPI 3.1.”
   • [community] “gpt-5.3-codex isn't available on the API yet — 'We are working to safely enable API access soon.'”
   • [claimed-docs] “Add local or remote MCP servers, authenticate when needed, and inspect the tools available to the current session before Codex uses them.”

   proof ↗⚑ flag

   Devin~partialprobed4/10

   Devin provides an API for integration (docs-7, docs-8) allowing developers to build applications and automate workflows, but the evidence never mentions dedicated official SDKs/client libraries (e.g., Python/JS packages), and probes for an OpenAPI spec that would back SDK generation all returned 404s (devin-probe-3). Missing for 10: named SDK packages in specific languages, SDK installation/usage docs, and a published OpenAPI/schema artifact confirming SDK-generation support.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

9. ai-native userSubscribe to events via webhooks

   weight 2 · round drawn
   Codex—none0/10

   No evidence in the pack mentions webhooks or event subscription capabilities for Codex; the product exposes MCP servers, CLI, and cloud task integrations but nothing about outbound webhook events for AI-native consumers.

   ⚑ flag

   Devin—none0/10

   No evidence in the pack mentions webhooks or event subscription mechanisms; Devin's API/MCP docs describe session creation and management but nothing about outbound webhook notifications for events.

   ⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round to Devin
   Codex~partialclaimed6/10

   Codex generates AI-driven insights and suggestions specifically about code: it produces prioritized review findings, diffs, and summaries during automated reviews and delegated tasks (codex-docs-5, codex-docs-10, codex-docs-41, codex-docs-45), and can delegate to subagents for deeper investigation (codex-docs-35). However, this is scoped to code/repository data rather than general business or product data insights. Missing for 10: evidence of insight generation over non-code data sources, dashboards, or analytics-style summaries beyond code review findings.

   • [claimed-docs] “Inspect the summary and diff, request a follow-up, or open a pull request when the result is ready.”
   • [claimed-docs] “Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized findings without modifying your wo…”
   • [claimed-docs] “Review changes before they ship: Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized f…”
   • [claimed-docs] “Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized findings without modifying your wo…”
   • [claimed-docs] “Split up a larger investigation — subagents: Ask Codex to delegate focused work to specialized agents, then bring their findings back into t…”

   proof ↗⚑ flag

   Devin✓fullclaimed7/10

   Devin generates insights/suggestions from a user's own codebase data via 'Ask Devin' (code structure/dependency Q&A), auto-generated DeepWiki documentation, and Devin Review's automated PR feedback, all built on repository indexing. Missing for 10: independent/hands-on validation of the quality of these AI-generated insights (community evidence covers general task execution issues, not this specific feature) and no benchmark of insight accuracy.

   • [claimed-docs] “Ask Devin can answer questions about code structure and dependencies, and help you scope and plan tasks before implementation.”
   • [claimed-docs] “Devin Review provides automated first-pass reviews on pull requests, checking for correctness and conformance with organizational best pract…”
   • [claimed-docs] “Use DeepWiki to navigate architecture and code with auto-generated documentation.”
   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”

   proof ↗⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Devin
   Codex✓fullclaimed7/10

   Codex cloud supports delegating longer tasks that run in isolated cloud environments in parallel, triggered from GitHub, GitLab, Linear, or Slack, and returning results (diff/PR) when ready — a clear background-automation workflow, and the CLI also supports non-interactive/repeatable workflows for scripted automation. Missing for 10: no documentation of scheduled/cron-style recurring triggers, and no independent/hands-on confirmation that long unattended background runs work reliably (community commentary focuses on interactive model quality/UX rather than background automation specifically).

   • [claimed-docs] “Run tasks in isolated cloud environments, work in parallel, and start work from the web, GitHub, GitLab, Linear, or Slack.”
   • [claimed-docs] “Start work in Codex cloud from GitHub pull requests, GitLab merge requests and issues, Linear issues, or Slack channels and threads.”
   • [claimed-docs] “Run tasks in parallel without tying up your local machine.”
   • [claimed-docs] “Delegate a longer task and return when it is ready.”
   • [claimed-docs] “Inspect the summary and diff, request a follow-up, or open a pull request when the result is ready.”
   • [claimed-docs] “Run a non-interactive command in a repeatable workflow.”

   proof ↗⚑ flag

   Devin✓fullclaimed8/10

   Devin explicitly supports background/autonomous execution: cloud sessions run in their own VM and 'keep going after you close your laptop' (devin-docs-13), MCP access includes 'scheduling' (devin-docs-6/23), the API lets you 'automate workflows' and create sessions programmatically (devin-docs-7/8), CI/CD integration triggers Devin on findings (devin-docs-19), and Auto-Fix creates a closed loop that iterates PRs 'without you in the loop' (devin-docs-18). This spans scheduled triggers, API-driven automation, and hands-off background operation. Missing for 10: independent/community verification that scheduled automations run reliably unattended, and more detail on a dedicated 'automation/schedule' UI beyond scattered doc mentions.

   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”

   proof ↗⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round to Codex
   Codex✓fullclaimed8/10

   Codex documents explicit task delegation to its built-in agent, both for long-running cloud tasks ('Delegate a longer task and return when it is ready') and for sub-agent delegation within a session ('Ask Codex to delegate focused work to specialized agents, then bring their findings back into the main terminal session'), backed by detailed CLI/cloud docs. Missing for 10: independent hands-on verification specifically of the subagent delegation flow (community evidence discusses general agent quality/UX but not this feature directly).

   • [claimed-docs] “Delegate a longer task and return when it is ready.”
   • [claimed-docs] “Ask Codex to delegate focused work to specialized agents, then bring their findings back into the main terminal session.”
   • [claimed-docs] “Split up a larger investigation — subagents: Ask Codex to delegate focused work to specialized agents, then bring their findings back into t…”
   • [claimed-docs] “Move work to Codex cloud — codex cloud: Browse active and completed chats, submit work to a configured environment, and apply the result to …”
   • [github] “Codex CLI is a coding agent from OpenAI that runs locally on your computer.”

   proof ↗⚑ flag

   Devin~partialcommunity6/10

   Devin's core product is designed for task delegation — via Ask Devin, ticket assignment, Slack/Teams tagging, and a conversational IDE (devin-docs-1, devin-docs-3, devin-docs-24) — so the axis clearly applies and is well documented. However, hands-on community reports show real caveats: Devin can add extraneous unrequested changes it can't undo, gets stuck for long periods without asking for help, and requires active babysitting/session termination to get value (devin-comm-1, devin-comm-2, devin-comm-5), undercutting a fully seamless delegation experience. Missing for 10: independent verification that delegated tasks reliably complete without extraneous side-effects or getting stuck, and stronger corroboration beyond one HN thread.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Devin is designed to be a conversational user interface, and allows you to follow and take over Devin's development process in the embedded …”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round drawn
   Codex✓fullcommunity8/10

   Codex CLI, IDE extension, cloud, and web surfaces are all operated by natural-language prompts/chats — e.g. starting tasks from prompts, resuming chats, delegating subagents, pasting images into the composer, and non-interactive `codex exec` for scripted natural-language instructions — all documented as the primary interaction mode across surfaces. Community threads corroborate heavy real-world use of this conversational/agentic workflow, even amid quality complaints about model performance. missing for 10: independent benchmarking specifically of natural-language command comprehension/robustness (community evidence is about overall agent quality/speed, not NL parsing specifically).

   • [claimed-docs] “Delegate a longer task and return when it is ready.”
   • [claimed-docs] “Start Codex in a repository to explore unfamiliar code, plan a change, edit files, and run your local development tools.”
   • [claimed-docs] “Ask Codex to delegate focused work to specialized agents, then bring their findings back into the main terminal session.”
   • [claimed-docs] “Run a non-interactive command in a repeatable workflow.”
   • [claimed-docs] “`codex resume`: Reopen a recent chat from the current repository, or search across local chats when you need to return to older work.”
   • [claimed-docs] “Bring visual context into the prompt — codex --image: Pass an error screenshot, architecture diagram, or design reference with the first pro…”
   • [github] “Codex CLI is a coding agent from OpenAI that runs locally on your computer.”
   • [community] “Genuinely excited to try this out. I've started using Codex much more heavily in the past two months and honestly, it's been shockingly good…”

   proof ↗⚑ flag

   Devin✓fullcommunity8/10

   Devin is explicitly designed as a conversational agent: users assign tasks via natural language (Slack/Teams tagging, chat interface, CLI prompts like 'devin -- check out this code...'), and it interprets these into autonomous coding/dev actions across IDE, CLI, and cloud sessions. Community evidence corroborates it operates on natural-language task descriptions in practice, though with noted friction around scope creep and knowing when to stop. Missing for 10: independent benchmarking of NL command accuracy/robustness and richer detail on how ambiguous instructions are resolved.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Tagging Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “Devin is designed to be a conversational user interface, and allows you to follow and take over Devin's development process in the embedded …”
   • [claimed-docs] “devin -- check out this code and suggest a feasible, helpful feature”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round to Codex
   Codex~partialclaimed5/10

   The evidence shows OpenAI's general API reference (developers.openai.com) has runnable, per-language code samples with live examples, which an AI-native user could explore. However this is the general OpenAI Responses API reference, not a Codex-specific interactive API reference, and Codex itself is documented as a CLI/agent product rather than an API with its own dedicated reference docs. Missing for 10: a Codex-specific API reference page, evidence of interactivity beyond code-sample selection (e.g., live sandbox execution), and any Codex-specific documentation of this reference.

   • [claimed-docs] “Create a model response — request/response reference with runnable code samples selectable per language: HTTP, Python, TypeScript, Go, Ruby,…”
   • [github] “You can also use Codex with an API key, but this requires additional setup.”

   proof ↗⚑ flag

   Devin—none0/10

   Devin has an API reference overview but the openapi.json probe returned 404 on all candidate paths, and there's no mention of an interactive reference with runnable examples (e.g., 'try it' console) in the docs pack.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round to Codex
   Codex~partialclaimed4/10

   OpenAI publishes a machine-readable OpenAPI 3.1 spec for its REST API (codex-gh-9) and Codex can be used via that API (codex-gh-4), but the evidence never confirms this spec explicitly covers or is dedicated to Codex-specific endpoints, nor is there a direct 'download spec' link tied to Codex docs. missing for 10: a Codex-specific OpenAPI/spec file, explicit download instructions, or confirmation the general OpenAI OpenAPI spec includes Codex CLI/agent endpoints.

   • [github] “A machine-readable description of the OpenAI REST API, authored in OpenAPI 3.1.”
   • [github] “You can also use Codex with an API key, but this requires additional setup.”
   • [claimed-docs] “Create a model response — request/response reference with runnable code samples selectable per language: HTTP, Python, TypeScript, Go, Ruby,…”

   proof ↗⚑ flag

   Devin—none0/10

   Devin has a documented API (devin-docs-7) but a direct probe for a machine-readable OpenAPI/swagger spec returned 404 on all candidate paths, and no documentation item references a downloadable spec file.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round drawn
   Codex~partialcommunity6/10

   Codex offers isolated cloud task environments and CLI sandbox controls (writable roots, permission gating) that keep agent actions contained rather than acting directly on a live system, which functions as a sandbox layer for testing changes. However, there's no explicit documentation of test-vs-production data separation, and a community report raises unresolved concerns about the sandbox reading sensitive filesystem data without asking. Missing for 10: explicit production-data isolation guarantees, first-party documentation addressing the raised sandbox-safety concern, and independent verification that isolated environments never touch real prod data.

   • [claimed-docs] “Run tasks in isolated cloud environments, work in parallel, and start work from the web, GitHub, GitLab, Linear, or Slack.”
   • [claimed-docs] “Configure the dependencies, tools, variables, and setup steps each repository needs.”
   • [claimed-docs] “Choose when Codex can edit files or run commands without asking, and inspect the active sandbox and writable roots before you continue.”
   • [community] “Does that version of Codex still read sensitive data on your file system without even asking? Just curious. [links to github.com/openai/code…”

   proof ↗⚑ flag

   Devin~partialclaimed6/10

   Devin sessions run in isolated cloud VMs with their own shell/browser/full repo access, configurable environment blueprints for a 'known-good state,' and a CLI --sandbox flag enforcing OS-level write/network isolation, all of which support testing in isolated environments away from live infrastructure. However, there is no explicit documentation addressing production-data isolation or masking, or confirmation that these sandboxes are guaranteed free of production data. missing for 10: explicit statement on production-data separation/masking, independent hands-on confirmation that sandbox testing never touches production data.

   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “Configure it once, and every session boots into that known-good state.”
   • [claimed-docs] “The --sandbox flag runs the CLI with OS-level isolation, enforcing writable paths and deny rules at the operating-system level and optionall…”
   • [claimed-docs] “The `--sandbox` flag runs the CLI with OS-level isolation, enforcing writable paths and `deny` rules at the operating-system level”
   • [claimed-docs] “Outposts lets you run Devin sessions inside infrastructure you control — your own VMs, containers, Kubernetes clusters, or even a Mac Mini o…”
   • [claimed-docs] “Devin inspects your repository, figures out which tools, runtimes, and dependencies are needed, and generates the blueprint for you.”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round to Codex
   Codex~partialclaimed3/10

   There is a documented OpenAPI 3.1 spec and API reference (codex-gh-9, codex-docs-29) and one concrete example of a deprecation notice (codex mcp-server deprecated in favor of the Codex app server, codex-docs-23), showing some practice of versioning and deprecation. However, there is no comprehensive, documented deprecation policy (timelines, notice periods, version numbering scheme) covering the Codex/OpenAI API generally. Missing for 10: an explicit deprecation policy document, API version numbering scheme, and independent corroboration of adherence to it.

   • [github] “A machine-readable description of the OpenAI REST API, authored in OpenAPI 3.1.”
   • [claimed-docs] “Create a model response — request/response reference with runnable code samples selectable per language: HTTP, Python, TypeScript, Go, Ruby,…”
   • [claimed-docs] “codex mcp-server is deprecated. Use the Codex app server instead. ... This page documents the deprecated command for existing integrations. …”

   proof ↗⚑ flag

   Devin—none0/10

   There's an API reference (devin-docs-7) and even an OpenAPI probe, but that probe found no OpenAPI spec (devin-probe-3), and no evidence anywhere mentions API versioning scheme or a documented deprecation policy for the API.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Codex
   Codex~partialclaimed6/10

   Codex supports running multiple cloud tasks in parallel across repos (codex-docs-1, codex-docs-3, codex-docs-6) and delegating focused work to specialized sub-agents within a session (codex-docs-13), which gives some bulk/parallel automation capability. However, there's no explicit evidence of a bulk operation primitive (e.g., batch-apply an action across many files/items/tickets in one command) — the parallelism described is task-level (multiple independent runs) rather than a documented 'operate over N items at once' feature. Missing for 10: explicit bulk/batch API or CLI verb for acting across many items in one invocation, and independent confirmation of large-scale parallel throughput in practice.

   • [claimed-docs] “Run tasks in isolated cloud environments, work in parallel, and start work from the web, GitHub, GitLab, Linear, or Slack.”
   • [claimed-docs] “Run tasks in parallel without tying up your local machine.”
   • [claimed-docs] “Delegate a longer task and return when it is ready.”
   • [claimed-docs] “Ask Codex to delegate focused work to specialized agents, then bring their findings back into the main terminal session.”
   • [claimed-docs] “Run a non-interactive command in a repeatable workflow.”

   proof ↗⚑ flag

   Devin~partialcommunity5/10

   Devin's API supports programmatic session creation (including on behalf of other users) and docs explicitly encourage carving out independent tasks to run simultaneously, which together enable bulk-style automation across many tickets/items, but there is no dedicated 'bulk operations' or batch-processing feature documented, and community feedback raises concerns about reliability/oversight needed per session that would complicate true bulk workflows. Missing for 10: an explicit batch/bulk API endpoint or UI for processing many items in one request, and independent evidence of successful large-scale bulk runs.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”
   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to Devin
   Codex—none0/10

   Codex supports triggering tasks from external events (GitHub/GitLab/Linear/Slack) and running non-interactive workflows, but there is no evidence of a user-defined rules engine that lets users specify arbitrary trigger conditions and automated actions (e.g., 'on X event, do Y') — this is closer to integration hooks than a rules/automation framework. missing for 10: evidence of a rules/trigger definition interface, conditional logic configuration, or event-to-action mapping system that users can author themselves.

   • [claimed-docs] “Run tasks in isolated cloud environments, work in parallel, and start work from the web, GitHub, GitLab, Linear, or Slack.”
   • [claimed-docs] “Start work in Codex cloud from GitHub pull requests, GitLab merge requests and issues, Linear issues, or Slack channels and threads.”
   • [claimed-docs] “Run a non-interactive command in a repeatable workflow.”

   proof ↗⚑ flag

   Devin~partialclaimed6/10

   Devin supports several built-in event-triggered automations — Auto-Fix responds automatically to PR review comments and CI failures, CI/CD integration triggers Devin off static-analysis findings (SonarQube/Fortify/Veracode), and MCP exposes 'scheduling' as a capability — but there's no evidence of a general-purpose, user-defined rules/webhook engine for arbitrary custom triggers. Missing for 10: documentation of a configurable custom-rule/webhook trigger system, details on the scheduling feature's flexibility, and independent confirmation that these automations work reliably as event triggers.

   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixe”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round to Devin
   Codex—none0/10

   The evidence shows Codex can run in CI/scripts (codex exec), be triggered from GitHub/GitLab/Slack, and run cloud tasks, but there is no mention of a native recurring/scheduled job or cron-like trigger mechanism within Codex itself. Automation is triggered by external events or manual invocation, not scheduled recurrence.

   • [claimed-docs] “Compose with scripts and CI: Use Codex interactively or call codex exec from repeatable workflows and pipelines.”
   • [claimed-docs] “Run a non-interactive command in a repeatable workflow.”
   • [claimed-docs] “Start work in Codex cloud from GitHub pull requests, GitLab merge requests and issues, Linear issues, or Slack channels and threads.”
   • [claimed-docs] “Move work to Codex cloud — codex cloud: Browse active and completed chats, submit work to a configured environment, and apply the result to …”

   proof ↗⚑ flag

   Devin~partialclaimed3/10

   Devin's MCP docs mention 'scheduling' as one of the capabilities exposed to MCP-compatible agents, implying some scheduling functionality exists, but there is no dedicated documentation, UI, or examples describing recurring jobs, cron-like triggers, or workflow automation configuration. missing for 10: dedicated scheduling feature docs, examples of recurring/cron jobs, independent confirmation of scheduled workflows in practice.

   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”

   proof ↗⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round to Codex
   Codex~partialclaimed4/10

   Codex's CLI includes a dedicated review command that inspects diffs/commits without modifying the working tree (codex-docs-10, codex-doces-41/45), and it operates within git repos so changes are inherently versioned and revertible via git; skills/plugins can be packaged as reusable automations (codex-docs-20/42). However, there is no documented mechanism to version, review, or roll back the automations/skills/workflows themselves (e.g., skill version history, rollback of a plugin config, audit trail for automation changes) — only code diffs are reviewed. Missing for 10: explicit versioning of skills/automations, a rollback UI/command for automation configs, and independent evidence of this workflow in practice.

   • [claimed-docs] “Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized findings without modifying your wo…”
   • [claimed-docs] “Review changes before they ship: Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized f…”
   • [claimed-docs] “Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized findings without modifying your wo…”
   • [claimed-docs] “Package repeatable instructions as skills, then add plugins to connect Codex to your team's tools and data without leaving the CLI.”
   • [claimed-docs] “Use skills and plugins: Package repeatable instructions as skills, then add plugins to connect Codex to your team's tools and data without l…”

   proof ↗⚑ flag

   Devin—none0/10

   The evidence shows Devin has knowledge, playbooks, and scheduling features but nothing about versioning, reviewing, or rolling back those automation configurations themselves; Devin Review/Auto-Fix pertains to PR code review, not to the automation definitions. Missing for 10: version history for playbooks/knowledge, a review workflow for automation changes, and a rollback mechanism for automations.

   ⚑ flag

Background execution

1. ai-native userHave a cloud agent build, test, and demo a feature end-to-end for my review

   weight 2 · round to Codex
   Codex✓fullcommunity8/10

   Codex cloud lets users delegate tasks that run in isolated cloud environments, inspect summaries/diffs, request follow-ups, and open pull requests for review, effectively building/testing/demoing changes end-to-end for user review (codex-docs-1,5,6,7,37). Community commentary corroborates real-world agentic task completion, though with performance/reliability caveats. Missing for 10: independent hands-on verification specifically of the cloud (not CLI) workflow's demo/test artifacts, and no explicit mention of a 'demo' step (e.g., live preview) beyond diff/PR review.

   • [claimed-docs] “Run tasks in isolated cloud environments, work in parallel, and start work from the web, GitHub, GitLab, Linear, or Slack.”
   • [claimed-docs] “Inspect the summary and diff, request a follow-up, or open a pull request when the result is ready.”
   • [claimed-docs] “Delegate a longer task and return when it is ready.”
   • [claimed-docs] “Start and review work from the web or Codex CLI.”
   • [claimed-docs] “Move work to Codex cloud — codex cloud: Browse active and completed chats, submit work to a configured environment, and apply the result to …”
   • [community] “Often Claude Code Opus 4.6, on hard enough problems, can do the impression of acting fast without really making progress. Then you spin the …”
   • [community] “Genuinely excited to try this out. I've started using Codex much more heavily in the past two months and honestly, it's been shockingly good…”

   proof ↗⚑ flag

   Devin~partialcommunity7/10

   Devin's cloud sessions run in dedicated VMs with shell/browser/full repo access, can implement features, run tests, and continue autonomously after handoff, then present PRs for review (docs-1,13,17). However, community hands-on reports describe unreliable autonomy — extraneous breaking changes, inability to self-correct, and needing frequent human monitoring/termination — undercutting the 'build, test, demo end-to-end' promise. missing for 10: reliable independent verification of unattended end-to-end demo quality, and clearer evidence of a built-in demo/walkthrough artifact for reviewers beyond PR creation.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “Devin Review provides automated first-pass reviews on pull requests, checking for correctness and conformance with organizational best pract…”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

2. developerDelegate longer-running coding tasks to run in the background in an isolated cloud environment

   weight 3 · round to Devin
   Codex✓fullclaimed7/10

   OpenAI's docs describe a dedicated Codex cloud mode that runs tasks in isolated cloud environments, in parallel, triggered from web/GitHub/GitLab/Linear/Slack, with configurable repo setup and a workflow to inspect diffs/PRs on completion, plus a CLI command (`codex cloud`) to submit and later pull results locally — squarely matching the story of delegating longer background tasks to an isolated cloud environment. missing for 10: independent or hands-on community corroboration specifically validating the cloud/background execution feature (community evidence in the pack discusses CLI/app UX and model quality, not the cloud delegation flow itself).

   • [claimed-docs] “Run tasks in isolated cloud environments, work in parallel, and start work from the web, GitHub, GitLab, Linear, or Slack.”
   • [claimed-docs] “Start work in Codex cloud from GitHub pull requests, GitLab merge requests and issues, Linear issues, or Slack channels and threads.”
   • [claimed-docs] “Run tasks in parallel without tying up your local machine.”
   • [claimed-docs] “Configure the dependencies, tools, variables, and setup steps each repository needs.”
   • [claimed-docs] “Inspect the summary and diff, request a follow-up, or open a pull request when the result is ready.”
   • [claimed-docs] “Delegate a longer task and return when it is ready.”
   • [claimed-docs] “Move work to Codex cloud — codex cloud: Browse active and completed chats, submit work to a configured environment, and apply the result to …”
   • [github] “If you are looking for the cloud-based agent from OpenAI, Codex Web, go to chatgpt.com/codex.”
   • [github] “If you are looking for the <em>cloud-based agent</em> from OpenAI, <strong>Codex Web</strong>, go to <a href="https://chatgpt.com/codex">cha…”

   proof ↗⚑ flag

   Devin✓fullcommunity9/10

   Devin runs tasks in isolated cloud VMs with full shell/browser/repo access that persist after the user disconnects, explicitly supporting long-running background work and parallel independent tasks, corroborated by community reports of multi-day autonomous runs. Missing for 10: independent third-party benchmarking of long-running task reliability/quality beyond anecdotal HN reports.

   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”
   • [claimed-docs] “Configure it once, and every session boots into that known-good state.”
   • [claimed-docs] “Hand a task off to a cloud Devin session and keep working locally.”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “You can set a 'max work time' before Devin pauses so it won't go for days endlessly spending your credits. By default it's set to 10 credits…”

   proof ↗⚑ flag

3. developerConfigure a reproducible cloud environment with the dependencies and setup steps my repository needs

   weight 2 · round to Devin
   Codex~partialclaimed6/10

   Codex Cloud docs state you can configure the dependencies, tools, variables, and setup steps each repository needs for isolated cloud environments, directly matching the story. However, there is no detail on how reproducibility is guaranteed (e.g., container images, caching, version pinning) or independent hands-on confirmation of this setup workflow. Missing for 10: concrete configuration file/schema details, reproducibility guarantees, and independent verification of the setup working as documented.

   • [claimed-docs] “Configure the dependencies, tools, variables, and setup steps each repository needs.”
   • [claimed-docs] “Run tasks in isolated cloud environments, work in parallel, and start work from the web, GitHub, GitLab, Linear, or Slack.”
   • [claimed-docs] “Delegate a longer task and return when it is ready.”

   proof ↗⚑ flag

   Devin✓fullclaimed9/10

   Devin explicitly supports configuring environment 'blueprints' that specify tools, runtimes, and dependencies so 'every session boots into that known-good state,' with auto-detection of requirements from the repo (docs-31, docs-32), plus indexing (docs-27), knowledge/AGENTS.md context files (docs-29, docs-30), and VPN access for internal dependencies (docs-28), all running in isolated cloud VMs (docs-13). Missing for 10: independent/hands-on confirmation that blueprint-based environments reliably reproduce across sessions in practice.

   • [claimed-docs] “Configure it once, and every session boots into that known-good state.”
   • [claimed-docs] “Devin inspects your repository, figures out which tools, runtimes, and dependencies are needed, and generates the blueprint for you.”
   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”
   • [claimed-docs] “Knowledge is a collection of instructions and advice that Devin can reference in all sessions.”
   • [claimed-docs] “Devin supports AGENTS.md - a simple, open standard for providing context and instructions to AI agents.”
   • [claimed-docs] “Devin can connect to a VPN from inside its workspace, so sessions can reach internal services such as package registries, databases, and int…”
   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”

   proof ↗⚑ flag

Parallel agents

1. ai-native userLaunch fleets of autonomous agents that work in parallel on different tasks for hours or days

   weight 2 · round drawn
   Codex~partialcommunity6/10

   Codex Cloud supports running multiple tasks in parallel in isolated cloud environments, triggered from GitHub/GitLab/Linear/Slack, and delegating longer tasks to return to later, which covers parallel/async agent work. However, there is no explicit evidence of orchestrating large 'fleets' of many simultaneous agents, no stated duration limits confirming multi-day autonomous runs, and community feedback highlights usage-limit throttling that would constrain sustained parallel/long-running fleets. missing for 10: evidence of fleet-scale orchestration (many concurrent agents), confirmed multi-day autonomous run duration, and independent confirmation that parallel tasks aren't throttled by usage limits.

   • [claimed-docs] “Run tasks in isolated cloud environments, work in parallel, and start work from the web, GitHub, GitLab, Linear, or Slack.”
   • [claimed-docs] “Run tasks in parallel without tying up your local machine.”
   • [claimed-docs] “Delegate a longer task and return when it is ready.”
   • [claimed-docs] “Configure the dependencies, tools, variables, and setup steps each repository needs.”
   • [community] “Codex is my favorite UX for anything as it edits the files and I can use the proper tooling to adjust and test stuff... However lately the l…”
   • [community] “The main issue I have with Codex is that the best model is insanely slow, except at nights and weekends when Silicon Valley goes to bed... I…”

   proof ↗⚑ flag

   Devin~partialcommunity6/10

   Docs confirm parallel task execution ('Carve out independent tasks and run them simultaneously'), cloud sessions that persist after closing the laptop, and an API to spin up multiple sessions programmatically (including on behalf of other users), which together support a 'fleet of parallel long-running agents' story. However, community hands-on reports show real friction with the 'hours/days autonomous' claim: sessions can get stuck without an escape hatch, users must babysit and manually terminate sessions every 10-15 minutes, and there's a default max-work-time cap limiting unsupervised runtime. Missing for 10: independent verification of successful multi-day/multi-task fleets running unattended, and evidence addressing the reported lack of a reliable 'ask for help' escalation during long runs.

   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”
   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “Devin does ask for help when it can't do something, but it really hates asking for help if it's a skill issue - it would prefer running in c…”
   • [community] “You can set a 'max work time' before Devin pauses so it won't go for days endlessly spending your credits. By default it's set to 10 credits…”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

2. developerRun several task attempts in parallel and compare results before choosing one

   weight 1 · round to Codex
   Codex~partialclaimed6/10

   Docs confirm Codex cloud can run tasks in parallel in isolated cloud environments without tying up the local machine, and results can be inspected (summary/diff) before choosing to follow up or open a PR — this covers running multiple attempts and reviewing outcomes. However, there's no explicit documentation of a dedicated 'compare multiple attempts side-by-side' UI/workflow, and no independent/community evidence confirming this parallel-comparison workflow works well in practice. missing for 10: explicit side-by-side comparison UI documentation, independent hands-on confirmation of comparing parallel attempts.

   • [claimed-docs] “Run tasks in isolated cloud environments, work in parallel, and start work from the web, GitHub, GitLab, Linear, or Slack.”
   • [claimed-docs] “Run tasks in parallel without tying up your local machine.”
   • [claimed-docs] “Inspect the summary and diff, request a follow-up, or open a pull request when the result is ready.”
   • [claimed-docs] “Delegate a longer task and return when it is ready.”

   proof ↗⚑ flag

   Devin—none0/10

   Devin's docs mention running multiple independent tasks simultaneously (devin-docs-25) but this describes parallelizing different tasks, not running several parallel attempts of the SAME task to compare and choose the best result. No evidence describes a compare/choose-best-attempt workflow.

   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”

   proof ↗⚑ flag

Scheduled automation

1. ai-native userSet up always-on agents that run on schedules or triggers to maintain and fix my software autonomously

   weight 2 · round to Devin
   Codex~partialclaimed5/10

   Codex cloud supports starting tasks from external triggers (GitHub/GitLab issues & PRs, Linear issues, Slack messages) and running them in parallel isolated environments, which covers the 'triggers' half of the story, but there's no evidence of a true schedule/cron-based always-on agent that proactively maintains a repo without an external event. Missing for 10: explicit scheduled/cron execution, evidence of continuous unattended monitoring/maintenance loops, and independent confirmation these triggers reliably run autonomous fixes end-to-end.

   • [claimed-docs] “Run tasks in isolated cloud environments, work in parallel, and start work from the web, GitHub, GitLab, Linear, or Slack.”
   • [claimed-docs] “Start work in Codex cloud from GitHub pull requests, GitLab merge requests and issues, Linear issues, or Slack channels and threads.”
   • [claimed-docs] “Run tasks in parallel without tying up your local machine.”
   • [claimed-docs] “Delegate a longer task and return when it is ready.”

   proof ↗⚑ flag

   Devin~partialcommunity6/10

   Devin supports trigger-based autonomous work (Slack/Teams tags, PR review comments, CI/CD/static-analysis findings) and Auto-Fix creates a closed loop that iterates on CI failures without a human in the loop, and MCP exposes 'scheduling' as a session capability, all suggesting some always-on/triggered agent operation. However there's no dedicated docs for cron-like recurring schedules, and community reports describe sessions needing frequent human monitoring/termination rather than fully unattended long-running maintenance. Missing for 10: explicit scheduling/cron configuration docs, independent evidence of reliable unattended multi-day maintenance loops, and confirmation that Auto-Fix/CI triggers work without human oversight in practice.

   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “You can set a 'max work time' before Devin pauses so it won't go for days endlessly spending your credits. By default it's set to 10 credits…”

   proof ↗⚑ flag

Debugging

1. developerDebug issues and troubleshoot using natural-language queries

   weight 2 · round drawn
   Codex~partialcommunity7/10

   Codex CLI docs show clear natural-language debugging workflows: exploring unfamiliar code, running local tools, passing error screenshots for context, and running dedicated code review that reports prioritized findings (codex-docs-8, codex-docs-9, codex-docs-10, codex-docs-12). However, community evidence shows mixed real-world reliability on agentic/coding tasks and no independent confirmation specifically validating debugging accuracy. Missing for 10: hands-on validation of debugging/troubleshooting accuracy, and independent case studies showing successful root-cause diagnosis via NL queries.

   • [claimed-docs] “Inspect code, make changes, run commands, and automate repeatable work without leaving your terminal.”
   • [claimed-docs] “Start Codex in a repository to explore unfamiliar code, plan a change, edit files, and run your local development tools.”
   • [claimed-docs] “Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized findings without modifying your wo…”
   • [claimed-docs] “Pass an error screenshot, architecture diagram, or design reference with the first prompt, or paste an image into the interactive composer.”
   • [community] “Having used codex a fair bit I find it really struggles with … almost anything. However using the equivalent chat gpt model is fantastic.”
   • [community] “Often Claude Code Opus 4.6, on hard enough problems, can do the impression of acting fast without really making progress. Then you spin the …”

   proof ↗⚑ flag

   Devin~partialcommunity7/10

   Docs clearly support NL-driven debugging: 'repro and fix bugs', 'Ask Devin can answer questions about code structure...help you scope and plan tasks', and tagging Devin in Slack/Teams about a bug thread. However, hands-on community reports describe practical caveats—Devin adding extraneous changes it can't undo, getting stuck without escalating, requiring manual babysitting—that temper reliability for troubleshooting workflows. Missing for 10: independent benchmark/case study specifically on debugging accuracy, and resolution of the 'getting stuck on bugs' community complaint.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Tagging Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “Ask Devin can answer questions about code structure and dependencies, and help you scope and plan tasks before implementation.”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “Devin does ask for help when it can't do something, but it really hates asking for help if it's a skill issue - it would prefer running in c…”

   proof ↗⚑ flag

Feature implementation

1. developerTurn a tracked issue into a complete pull request end-to-end

   weight 3 · round to Codex
   Codex✓fullclaimed8/10

   Codex explicitly supports starting work from a tracked issue (GitHub, GitLab, Linear) in cloud environments, running the task, inspecting the diff/summary, and opening a pull request when done, covering the full issue-to-PR loop. missing for 10: independent hands-on confirmation of a full issue-to-merged-PR workflow succeeding end-to-end, and detail on how issue context/acceptance criteria are actually parsed.

   • [claimed-docs] “Run tasks in isolated cloud environments, work in parallel, and start work from the web, GitHub, GitLab, Linear, or Slack.”
   • [claimed-docs] “Start work in Codex cloud from GitHub pull requests, GitLab merge requests and issues, Linear issues, or Slack channels and threads.”
   • [claimed-docs] “Inspect the summary and diff, request a follow-up, or open a pull request when the result is ready.”
   • [claimed-docs] “Delegate a longer task and return when it is ready.”

   proof ↗⚑ flag

   Devin~partialcommunity6/10

   Devin's docs explicitly describe taking Linear/Jira tickets and implementing full features, with Devin Review/Auto-Fix looping PRs toward merge-ready status without human involvement, covering the issue-to-PR pipeline end-to-end. However, hands-on community reports describe practical friction — extraneous unrelated changes that can break things, inability to easily undo them, and agents getting stuck for days rather than asking for help — casting doubt on how cleanly the 'complete' PR is delivered without oversight. Missing for 10: independent verification of a clean ticket→merged-PR flow without manual intervention, and resolution of the reported reliability/quality issues.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Devin Review provides automated first-pass reviews on pull requests, checking for correctness and conformance with organizational best pract…”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Code migrations, refactors, and modernization”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “Devin does ask for help when it can't do something, but it really hates asking for help if it's a skill issue - it would prefer running in c…”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

2. developerDescribe a feature or bug in plain language and have the agent implement or fix it across multiple files

   weight 3 · round to Codex
   Codex✓fullcommunity8/10

   Codex CLI and cloud docs describe the core loop of natural-language task description leading to autonomous file inspection, editing, running local tools, and producing a diff/PR (codex-docs-30, codex-docs-9, codex-docs-6, codex-docs-5), and community commentary corroborates it does real multi-file edits ('it edits the files and I can use the proper tooling', 'shockingly good... no worse than average L3-L4 engs') alongside some negative UX complaints that don't dispute the core capability. Missing for 10: independent benchmark/case-study evidence specifically confirming complex multi-file refactors across large codebases, and some community reports of it 'struggling with almost anything' create mild quality tension without rising to a concrete dispute.

   • [claimed-docs] “Work against your local repository: Let Codex inspect files, make edits, and run the tools already installed on your machine.”
   • [claimed-docs] “Start Codex in a repository to explore unfamiliar code, plan a change, edit files, and run your local development tools.”
   • [claimed-docs] “Delegate a longer task and return when it is ready.”
   • [claimed-docs] “Inspect the summary and diff, request a follow-up, or open a pull request when the result is ready.”
   • [github] “Codex CLI is a coding agent from OpenAI that runs locally on your computer.”
   • [community] “Codex is my favorite UX for anything as it edits the files and I can use the proper tooling to adjust and test stuff... However lately the l…”
   • [community] “Genuinely excited to try this out. I've started using Codex much more heavily in the past two months and honestly, it's been shockingly good…”
   • [community] “Having used codex a fair bit I find it really struggles with … almost anything. However using the equivalent chat gpt model is fantastic.”

   proof ↗⚑ flag

   Devin!disputedcontradicted5/10

   Devin's docs strongly claim end-to-end feature/bug implementation across a full repo (Linear/Jira tickets, multi-file fixes, code migrations) with full workspace/VM access [devin-docs-1, devin-docs-13, devin-docs-21, devin-docs-27], but hands-on community reports concretely contradict smooth delivery: it 'always adds extraneous changes beyond the task that can break other things, and can't undo those changes if asked' and required constant supervision/termination to get real work done [devin-comm-1, devin-comm-5], with another user noting it can run for days without an escape hatch when stuck [devin-comm-2]. Missing for 10: independent benchmark data on multi-file correctness, and resolution of the extraneous-change/undo failure mode reported by users.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “Code migrations, refactors, and modernization”
   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

Maintenance automation

1. developerHave the agent write tests, fix lint errors, resolve merge conflicts, and update dependencies for me

   weight 3 · round drawn
   Codex~partialcommunity6/10

   Codex CLI/cloud docs describe a general-purpose coding agent that can inspect code, edit files, run local dev tools, automate repeatable work, and review diffs before PRs — capabilities broad enough to plausibly cover writing tests, fixing lint issues, resolving conflicts, and updating dependencies (codex-docs-8, codex-docs-9, codex-docs-30, codex-docs-41). However, none of the docs explicitly name test-writing, lint-fixing, merge-conflict resolution, or dependency updates as supported workflows, and community feedback is mixed on real-world reliability for complex agentic tasks (codex-comm-3, codex-comm-13). missing for 10: explicit documentation/examples of test generation, lint-fix automation, merge-conflict resolution, and dependency-update workflows, plus hands-on confirmation these specific tasks succeed.

   • [claimed-docs] “Inspect code, make changes, run commands, and automate repeatable work without leaving your terminal.”
   • [claimed-docs] “Start Codex in a repository to explore unfamiliar code, plan a change, edit files, and run your local development tools.”
   • [claimed-docs] “Work against your local repository: Let Codex inspect files, make edits, and run the tools already installed on your machine.”
   • [claimed-docs] “Review changes before they ship: Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized f…”
   • [claimed-docs] “Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized findings without modifying your wo…”
   • [community] “Codex is my favorite UX for anything as it edits the files and I can use the proper tooling to adjust and test stuff... However lately the l…”
   • [community] “Having used codex a fair bit I find it really struggles with … almost anything. However using the equivalent chat gpt model is fantastic.”

   proof ↗⚑ flag

   Devin~partialcommunity6/10

   Docs show Devin is a general-purpose coding agent that can fix bugs, implement features, iterate on CI failures, respond to review comments (Auto-Fix), and handle code migrations/refactors/modernization, which plausibly covers lint fixes and CI-related work, but none of the docs explicitly mention writing tests, resolving merge conflicts, or updating dependencies as named capabilities. Community reports (devin-comm-1) also note Devin can introduce extraneous changes and struggles to cleanly undo them, tempering confidence in reliably delivering these specific maintenance tasks. Missing for 10: explicit documentation/evidence of test-writing, lint-fixing, merge-conflict resolution, and dependency-update workflows, plus independent hands-on confirmation of these specific tasks succeeding.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Code migrations, refactors, and modernization”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”

   proof ↗⚑ flag

Multimodal generation

1. ai-native userGenerate a working app from a sketch, image, or PDF design

   weight 2 · round to Codex
   Codex~partialclaimed4/10

   Codex supports passing images (error screenshots, architecture diagrams, design references) into prompts, which is a partial building block for generating apps from a sketch/image, but there's no evidence of dedicated PDF-to-app workflows, multi-page design ingestion, or documented end-to-end 'sketch/image to working app' generation feature. missing for 10: explicit PDF design ingestion, dedicated image/design-to-app pipeline or template, independent hands-on demonstration of generating a full app from a design artifact.

   • [claimed-docs] “Pass an error screenshot, architecture diagram, or design reference with the first prompt, or paste an image into the interactive composer.”

   proof ↗⚑ flag

   Devin—none0/10

   No evidence anywhere in the pack that Devin accepts a sketch/image/PDF design as input and generates a working app from it; documentation focuses on text-based tasks, tickets, Slack threads, code review, and CLI/desktop environment features with no multimodal design-to-app capability mentioned.

   ⚑ flag

Codebase mapping

1. developerUnderstand how a codebase fits together to find where to start making changes

   weight 3 · round to Devin
   Codex~partialclaimed6/10

   Codex CLI docs explicitly mention exploring unfamiliar code and planning changes within a repository, and it can inspect code, run local dev tools, and review diffs/commits — supporting codebase orientation. However, there's no dedicated codebase-mapping/visualization feature, no evidence of dependency-graph or architecture-summary generation, and community feedback focuses on agentic task execution rather than comprehension aids. Missing for 10: dedicated codebase-map/architecture-overview feature, independent hands-on evidence of effectively onboarding to unfamiliar large codebases, and richer navigation/search tooling beyond terminal chat resume.

   • [claimed-docs] “Start Codex in a repository to explore unfamiliar code, plan a change, edit files, and run your local development tools.”
   • [claimed-docs] “Inspect code, make changes, run commands, and automate repeatable work without leaving your terminal.”
   • [claimed-docs] “Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized findings without modifying your wo…”
   • [claimed-docs] “Reopen a recent chat from the current repository, or search across local chats when you need to return to older work.”

   proof ↗⚑ flag

   Devin✓fullclaimed7/10

   Devin explicitly indexes repositories to power 'Ask Devin' and 'DeepWiki', which answer questions about code structure and dependencies and help developers scope/plan where to start making changes, directly matching the story. missing for 10: independent/hands-on evidence validating DeepWiki/Ask Devin's accuracy on real codebases (community evidence only covers task execution, not codebase-understanding features).

   • [claimed-docs] “Use DeepWiki to navigate architecture and code with auto-generated documentation.”
   • [claimed-docs] “Ask Devin can answer questions about code structure and dependencies, and help you scope and plan tasks before implementation.”
   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”

   proof ↗⚑ flag

2. developerHave the agent map and explain an entire unfamiliar codebase without manually selecting context files

   weight 3 · round to Devin
   Codex~partialclaimed5/10

   Codex CLI docs explicitly state it can be started in a repository 'to explore unfamiliar code, plan a change, edit files, and run your local development tools' (codex-docs-9), implying the agent autonomously navigates the codebase rather than requiring manual file selection, and codex-gh-1 confirms it runs as an autonomous coding agent locally. However, there's no detailed documentation of how it builds a whole-codebase map/summary, no explicit 'explain codebase' feature, and no independent hands-on evidence confirming this works well on large unfamiliar repos. Missing for 10: dedicated codebase-mapping/summarization feature documentation, evidence of handling very large repos, and independent user reports validating this specific capability.

   • [claimed-docs] “Start Codex in a repository to explore unfamiliar code, plan a change, edit files, and run your local development tools.”
   • [github] “Codex CLI is a coding agent from OpenAI that runs locally on your computer.”

   proof ↗⚑ flag

   Devin✓fullclaimed7/10

   Docs describe repo indexing that lets Devin understand the codebase and power 'Ask Devin' and DeepWiki auto-generated architecture docs, directly enabling exploration/explanation of an unfamiliar codebase without manual file selection (devin-docs-27, devin-docs-15, devin-docs-16). No community evidence contradicts this specific capability. Missing for 10: independent/hands-on verification of codebase-mapping accuracy and no concrete example of DeepWiki output quality.

   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”
   • [claimed-docs] “Use DeepWiki to navigate architecture and code with auto-generated documentation.”
   • [claimed-docs] “Ask Devin can answer questions about code structure and dependencies, and help you scope and plan tasks before implementation.”

   proof ↗⚑ flag

Context management

1. developerHave the agent build and recall memory automatically across sessions

   weight 2 · round to Devin
   Codex~partialclaimed3/10

   Codex CLI supports `codex resume` to reopen or search past local chats in a repository, giving a limited form of session recall, but this requires manual user action rather than automatic memory building/recall across sessions. Missing for 10: evidence of automatic persistent memory (learned facts, preferences, or context) that Codex builds unprompted and recalls without explicit resume/search commands, and any cross-session synthesis beyond raw chat transcripts.

   • [claimed-docs] “Reopen a recent chat from the current repository, or search across local chats when you need to return to older work.”
   • [claimed-docs] “`codex resume`: Reopen a recent chat from the current repository, or search across local chats when you need to return to older work.”

   proof ↗⚑ flag

   Devin~partialclaimed5/10

   Devin supports persistent cross-session context via "Knowledge" (instructions referenced in all sessions), AGENTS.md, and environment blueprints that let every session boot into a known-good state, which enables some recall across sessions. However, these mechanisms are largely user-configured/onboarded rather than autonomously built by the agent from its own experience, and there's no evidence of automatic memory creation or recall behavior demonstrated in practice. Missing for 10: evidence the agent automatically extracts/updates memory from its own task experience without manual setup, and independent confirmation that recalled memory improves subsequent session performance.

   • [claimed-docs] “Knowledge is a collection of instructions and advice that Devin can reference in all sessions.”
   • [claimed-docs] “Devin supports AGENTS.md - a simple, open standard for providing context and instructions to AI agents.”
   • [claimed-docs] “Configure it once, and every session boots into that known-good state.”
   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”

   proof ↗⚑ flag

2. developerInclude multiple project directories in a single session for broader context

   weight 2 · round drawn
   Codex—none0/10

   No evidence in the pack describes Codex supporting multiple project directories or repositories being combined in a single session/context; documentation focuses on single-repository sessions, cloud tasks, and per-repository setup steps.

   ⚑ flag

   Devin—none0/10

   No evidence describes attaching or indexing multiple project directories/repositories within a single Devin session; docs discuss single-repo indexing, knowledge, and environment setup but not multi-directory context in one session.

   ⚑ flag

3. developerAdd a project instructions file to set coding standards and conventions the agent follows

   weight 3 · round to Devin
   Codex—none0/10

   The evidence pack covers Codex's CLI, cloud, MCP, and review features but contains no mention of a project-level instructions/config file (e.g., AGENTS.md or similar) for setting coding standards or conventions the agent should follow. This is a plausible and common capability for coding agents, but nothing in the pack documents or demonstrates it.

   ⚑ flag

   Devin✓fullclaimed9/10

   Devin explicitly supports AGENTS.md, an open standard for providing context and instructions to AI agents, plus a separate 'Knowledge' feature for instructions/advice referenced across all sessions, directly covering project-level coding standards/conventions. Missing for 10: independent/hands-on confirmation that these instructions are reliably followed in practice.

   • [claimed-docs] “Devin supports AGENTS.md - a simple, open standard for providing context and instructions to AI agents.”
   • [claimed-docs] “Knowledge is a collection of instructions and advice that Devin can reference in all sessions.”

   proof ↗⚑ flag

Issue diagnosis

1. developerReproduce issues, narrow down root causes, and verify fixes

   weight 3 · round to Codex
   Codex✓fullclaimed7/10

   Codex CLI docs explicitly describe exploring unfamiliar code and running local dev tools to investigate issues, passing error screenshots for context, delegating focused investigation to subagents, and running dedicated reviews against uncommitted changes/commits/base branches to verify fixes before committing — covering reproduce, narrow-down, and verify steps. Missing for 10: no explicit 'reproduce a bug' walkthrough or first-hand/independent account of successfully diagnosing and fixing a real bug end-to-end with Codex.

   • [claimed-docs] “Start Codex in a repository to explore unfamiliar code, plan a change, edit files, and run your local development tools.”
   • [claimed-docs] “Pass an error screenshot, architecture diagram, or design reference with the first prompt, or paste an image into the interactive composer.”
   • [claimed-docs] “Ask Codex to delegate focused work to specialized agents, then bring their findings back into the main terminal session.”
   • [claimed-docs] “Work against your local repository: Let Codex inspect files, make edits, and run the tools already installed on your machine.”
   • [claimed-docs] “Review changes before they ship: Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized f…”
   • [claimed-docs] “Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized findings without modifying your wo…”
   • [claimed-docs] “Run a non-interactive command in a repeatable workflow.”

   proof ↗⚑ flag

   Devin~partialcommunity6/10

   Docs explicitly claim Devin can 'repro and fix bugs' (devin-docs-1), use Ask Devin/DeepWiki to narrow root causes via codebase understanding (devin-docs-15, devin-docs-16, devin-docs-27), and verify fixes through CI iteration/Auto-Fix loops (devin-docs-18). However, hands-on community testimony reports Devin often adds extraneous changes beyond the task scope and cannot reliably undo them when asked, undermining clean verification of fixes (devin-comm-1), and lacks an escape hatch when stuck on root-cause diagnosis (devin-comm-2, devin-comm-3). Missing for 10: independent verification of successful bug reproduction/root-cause narrowing at scale, and resolution of the reported inability to cleanly revert unwanted changes during fix verification.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Use DeepWiki to navigate architecture and code with auto-generated documentation.”
   • [claimed-docs] “Ask Devin can answer questions about code structure and dependencies, and help you scope and plan tasks before implementation.”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “Devin does ask for help when it can't do something, but it really hates asking for help if it's a skill issue - it would prefer running in c…”

   proof ↗⚑ flag

Marketplace

1. developerEquip the agent with custom skills to perform specialized tasks

   weight 1 · round to Codex
   Codex✓fullclaimed7/10

   Codex CLI docs explicitly describe packaging repeatable instructions as "skills" and adding plugins to connect Codex to team tools/data from the CLI, directly matching the custom-skills story. Missing for 10: independent hands-on validation of skill creation/usage, and deeper documentation on skill authoring format/lifecycle beyond a single mention.

   • [claimed-docs] “Package repeatable instructions as skills, then add plugins to connect Codex to your team's tools and data without leaving the CLI.”

   proof ↗⚑ flag

   Devin~partialclaimed6/10

   Devin supports several skill-like extension mechanisms — 'Knowledge' (persistent instructions/advice for all sessions), AGENTS.md for structured agent instructions, and 'Playbooks' exposed via MCP — plus an open-source 'Devin Handoff' explicitly described as a 'plugin and skill'. This gives developers real levers to encode specialized, reusable task behavior, but the docs don't show a dedicated skill-authoring UI/marketplace or detailed examples of building a complex custom skill, and there is no independent/community evidence confirming this works well in practice. Missing for 10: concrete examples/tutorials of authoring a non-trivial custom skill, a discoverable skills registry, and independent hands-on corroboration.

   • [claimed-docs] “Knowledge is a collection of instructions and advice that Devin can reference in all sessions.”
   • [claimed-docs] “Devin supports AGENTS.md - a simple, open standard for providing context and instructions to AI agents.”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “Devin Handoff is an open-source plugin and skill that brings the same handoff workflow to any coding agent — Claude Code, Codex, Cursor, and…”

   proof ↗⚑ flag

2. engineering-leadIntegrate third-party partner-built agent apps into my workflows

   weight 1 · round to Codex
   Codex~partialclaimed6/10

   Codex documents integration points for third-party ecosystem tools — triggering work from GitHub, GitLab, Linear, and Slack (partner platforms), and connecting to third-party MCP servers, plugins, and skills that give access to tools like Figma or a browser — which supports embedding partner-built capabilities into engineering workflows. However, the evidence is framed around Codex consuming tools/data sources rather than a curated marketplace of partner-built 'agent apps,' and there's no independent case study of a partner agent integration working end-to-end. Missing for 10: evidence of a partner/agent-app marketplace or certified third-party agent integrations, and independent verification of such integrations working in practice.

   • [claimed-docs] “Run tasks in isolated cloud environments, work in parallel, and start work from the web, GitHub, GitLab, Linear, or Slack.”
   • [claimed-docs] “Start work in Codex cloud from GitHub pull requests, GitLab merge requests and issues, Linear issues, or Slack channels and threads.”
   • [claimed-docs] “Package repeatable instructions as skills, then add plugins to connect Codex to your team's tools and data without leaving the CLI.”
   • [claimed-docs] “Use it to give ChatGPT or Codex access to third-party documentation, or to let it interact with developer tools like your browser or Figma.”
   • [claimed-docs] “Connect external tools with MCP — codex mcp: Add local or remote MCP servers, authenticate when needed, and inspect the tools available to t…”
   • [claimed-docs] “Use skills and plugins: Package repeatable instructions as skills, then add plugins to connect Codex to your team's tools and data without l…”
   • [claimed-docs] “Model Context Protocol (MCP) connects models to tools and context. Use it to give ChatGPT or Codex access to third-party documentation, or t…”

   proof ↗⚑ flag

   Devin~partialprobed4/10

   Devin exposes an MCP server so external MCP-compatible agents/IDEs can access its session management and tools, and its open-source 'Devin Handoff' plugin interoperates with other coding agents (Claude Code, Codex, Cursor), showing some cross-agent workflow integration. However there is no evidence of a partner/marketplace ecosystem of third-party agent apps being integrated into Devin's own workflows. Missing for 10: a documented partner-app marketplace or catalog, evidence of installing/configuring third-party agent apps within Devin, and any case study of an engineering-lead orchestrating partner-built agents through Devin.

   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “Devin Handoff is an open-source plugin and skill that brings the same handoff workflow to any coding agent — Claude Code, Codex, Cursor, and…”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [probe] “official MCP server documented at https://docs.devin.ai/work-with-devin/devin-mcp”

   proof ↗⚑ flag

Team knowledge

1. engineering-leadCreate a shared workspace from my docs and repos as a common source of truth for the team

   weight 1 · round to Devin
   Codex—none0/10

   Codex documents repo-level cloud environments, RBAC, and MCP connections to team tools, but no evidence describes a shared 'workspace' feature that unifies docs and repos into a common source of truth for a team; this is a plausible ask for an engineering tool but Codex's evidence only covers per-task cloud environments and repo configuration, not a persistent shared knowledge/workspace layer.

   • [claimed-docs] “Configure the dependencies, tools, variables, and setup steps each repository needs.”
   • [claimed-docs] “Role-based access control (RBAC) lets you decide who can do what across your organization and projects—both through the API and in the Dashb…”
   • [claimed-docs] “Run tasks in isolated cloud environments, work in parallel, and start work from the web, GitHub, GitLab, Linear, or Slack.”

   proof ↗⚑ flag

   Devin~partialclaimed6/10

   Devin offers building blocks for a team-wide source of truth — repo indexing that powers 'Ask Devin' and DeepWiki architecture docs, org-wide 'Knowledge' referenced in all sessions, and AGENTS.md support for shared context — which collectively let a lead centralize docs/repo knowledge for the team. However, there's no explicit product feature framed as a 'shared workspace' UI for team-wide docs/repo browsing outside of Devin's own agent sessions. Missing for 10: a dedicated shared workspace/knowledge-base product surface for humans to browse, independent evidence of teams using it as a collaborative source of truth, and clarity on cross-repo doc aggregation beyond per-session knowledge.

   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”
   • [claimed-docs] “Knowledge is a collection of instructions and advice that Devin can reference in all sessions.”
   • [claimed-docs] “Devin supports AGENTS.md - a simple, open standard for providing context and instructions to AI agents.”
   • [claimed-docs] “Use DeepWiki to navigate architecture and code with auto-generated documentation.”
   • [claimed-docs] “Devin inspects your repository, figures out which tools, runtimes, and dependencies are needed, and generates the blueprint for you.”

   proof ↗⚑ flag

Tool integration

1. developerConnect the agent to workflow tools like Jira, Slack, and Google Drive to extend its context

   weight 3 · round drawn
   Codex~partialclaimed6/10

   Codex explicitly supports starting work from Slack (and GitHub/GitLab/Linear) and lets users add local or remote MCP servers to connect to third-party tools/docs (e.g. Figma, browser), giving a generic mechanism to extend context to workflow tools. However, there is no explicit documentation of native Jira or Google Drive connectors—only Slack is named among the story's specific tools, with Jira/Google Drive requiring the generic (and for one variant, deprecated/experimental) MCP server pathway. Missing for 10: named Jira integration, named Google Drive integration, and confirmation that the current (non-deprecated) MCP mechanism is broadly used for these specific SaaS tools.

   • [claimed-docs] “Run tasks in isolated cloud environments, work in parallel, and start work from the web, GitHub, GitLab, Linear, or Slack.”
   • [claimed-docs] “Start work in Codex cloud from GitHub pull requests, GitLab merge requests and issues, Linear issues, or Slack channels and threads.”
   • [claimed-docs] “Add local or remote MCP servers, authenticate when needed, and inspect the tools available to the current session before Codex uses them.”
   • [claimed-docs] “Package repeatable instructions as skills, then add plugins to connect Codex to your team's tools and data without leaving the CLI.”
   • [claimed-docs] “Use it to give ChatGPT or Codex access to third-party documentation, or to let it interact with developer tools like your browser or Figma.”
   • [claimed-docs] “Model Context Protocol (MCP) connects models to tools and context. Use it to give ChatGPT or Codex access to third-party documentation, or t…”
   • [claimed-docs] “Connect external tools with MCP — codex mcp: Add local or remote MCP servers, authenticate when needed, and inspect the tools available to t…”
   • [github] “Codex MCP Server Interface [experimental]: a JSON-RPC API that runs over the Model Context Protocol (MCP) transport to control a local Codex…”
   • [claimed-docs] “codex mcp-server is deprecated. Use the Codex app server instead. ... This page documents the deprecated command for existing integrations. …”

   proof ↗⚑ flag

   Devin~partialclaimed6/10

   Devin can be tagged in Slack/Teams threads and work Jira/Linear tickets, giving it direct workflow-tool integration, and its MCP server plus API enable further extension to other tools. However there is no explicit documented Google Drive integration, and no independent/hands-on corroboration of these integrations actually working in practice. missing for 10: Google Drive connector evidence, independent verification of Jira/Slack integration reliability.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Tagging Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”

   proof ↗⚑ flag

2. developerKick off agent tasks directly from GitHub, GitLab, Linear, or Slack

   weight 2 · round to Codex
   Codex✓fullclaimed8/10

   First-party docs explicitly state Codex cloud tasks can be started from GitHub pull requests, GitLab merge requests/issues, Linear issues, or Slack channels/threads, matching the story directly. Missing for 10: independent/hands-on verification of these specific integrations working in practice (community evidence covers CLI/app UX but not the GitHub/GitLab/Linear/Slack kickoff flows specifically).

   • [claimed-docs] “Run tasks in isolated cloud environments, work in parallel, and start work from the web, GitHub, GitLab, Linear, or Slack.”
   • [claimed-docs] “Start work in Codex cloud from GitHub pull requests, GitLab merge requests and issues, Linear issues, or Slack channels and threads.”
   • [claimed-docs] “Delegate a longer task and return when it is ready.”

   proof ↗⚑ flag

   Devin~partialclaimed6/10

   Docs confirm Devin can be invoked from Linear/Jira tickets and Slack/Teams threads, plus API integration for building custom workflow triggers, but there's no explicit mention of GitHub or GitLab issue/PR-based task kickoff in the evidence pack. Community evidence doesn't directly contradict the integration claims, only general effectiveness concerns. Missing for 10: explicit GitHub/GitLab-triggered task creation documentation, independent hands-on confirmation of these specific integrations working.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Tagging Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”

   proof ↗⚑ flag

Cross device continuity

1. developerStart a task on one device and continue it later from another device or browser

   weight 2 · round to Codex
   Codex✓fullclaimed8/10

   Codex supports starting tasks in the cloud from web/GitHub/GitLab/Linear/Slack, working in parallel cloud environments, and later resuming or continuing work from the CLI via 'codex cloud' (browse active/completed chats, submit/apply results) or 'codex resume' to reopen local chats, plus a shared MCP config across ChatGPT desktop, CLI, and IDE extension enabling cross-client continuity. missing for 10: independent hands-on confirmation of seamless state sync across devices/browsers, and no explicit mention of resuming a cloud-started task from a different physical device's browser session.

   • [claimed-docs] “Run tasks in isolated cloud environments, work in parallel, and start work from the web, GitHub, GitLab, Linear, or Slack.”
   • [claimed-docs] “Start work in Codex cloud from GitHub pull requests, GitLab merge requests and issues, Linear issues, or Slack channels and threads.”
   • [claimed-docs] “Delegate a longer task and return when it is ready.”
   • [claimed-docs] “Start and review work from the web or Codex CLI.”
   • [claimed-docs] “Move work to Codex cloud — codex cloud: Browse active and completed chats, submit work to a configured environment, and apply the result to …”
   • [claimed-docs] “`codex resume`: Reopen a recent chat from the current repository, or search across local chats when you need to return to older work.”
   • [claimed-docs] “The ChatGPT desktop app, Codex CLI, and IDE extension share this configuration. Once you configure your MCP servers, you can switch among th…”

   proof ↗⚑ flag

   Devin✓fullclaimed7/10

   Devin's cloud sessions run in a dedicated VM that persists independent of the local device (docs-13), can be started from Slack/Teams, IDE, CLI, or API and continued/taken over in the embedded IDE or web UI (docs-3, docs-4, docs-7, docs-24), and Devin Handoff explicitly lets you start work locally and continue in a cloud session accessible from a browser (docs-38). missing for 10: no explicit first-party walkthrough of resuming the same session from a different browser/device login, and no independent/community confirmation of cross-device continuity.

   • [claimed-docs] “Devin is designed to be a conversational user interface, and allows you to follow and take over Devin's development process in the embedded …”
   • [claimed-docs] “Devin CLI, a local command-line coding agent with deep Devin Cloud integration.”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “Hand a task off to a cloud Devin session and keep working locally.”

   proof ↗⚑ flag

Ide integration

1. developerChat with the coding assistant directly inside my IDE for contextual help

   weight 3 · round to Codex
   Codex✓fullcommunity8/10

   Codex explicitly offers an IDE extension for VS Code, Cursor, and Windsurf, plus a CLI usable within the terminal in your repo, both providing contextual chat/help with the codebase (edit files, run commands, review diffs). Community evidence confirms real-world usage of Codex CLI/app for editing and testing files in context, though some note UX friction compared to competitors. Missing for 10: deeper first-party documentation/screenshots of the IDE extension's chat UI specifically, and stronger independent hands-on corroboration of in-IDE chat quality.

   • [github] “If you want Codex in your code editor (VS Code, Cursor, Windsurf), install in your IDE.”
   • [claimed-docs] “Start Codex in a repository to explore unfamiliar code, plan a change, edit files, and run your local development tools.”
   • [claimed-docs] “Inspect code, make changes, run commands, and automate repeatable work without leaving your terminal.”
   • [community] “Codex is my favorite UX for anything as it edits the files and I can use the proper tooling to adjust and test stuff... However lately the l…”

   proof ↗⚑ flag

   Devin~partialclaimed5/10

   Devin exposes a conversational interface via its own embedded IDE within cloud sessions (devin-docs-3) and its Desktop app that imports VS Code/Cursor settings (devin-docs-9), plus an MCP server letting 'any MCP-compatible AI agent or IDE' access sessions (devin-docs-6/23). However there's no evidence of a native extension that lets a developer chat with Devin directly inside their own existing IDE (e.g., a VS Code/JetBrains plugin) — Devin's model is its own IDE/Desktop environment or MCP bridging rather than embedding in the user's IDE. Missing for 10: a first-party IDE extension for VS Code/JetBrains enabling in-IDE chat, and independent confirmation of this workflow working well in practice.

   • [claimed-docs] “Devin is designed to be a conversational user interface, and allows you to follow and take over Devin's development process in the embedded …”
   • [claimed-docs] “Import VS Code or Cursor settings, configure themes, and start coding with AI-powered assistance.”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”

   proof ↗⚑ flag

Session management

1. developerReview diffs visually and run multiple sessions side by side in a desktop app

   weight 2 · round to Codex
   Codex~partialcommunity5/10

   Codex ships a desktop app ("codex app"/Codex App page) and documents parallel task execution plus diff/summary inspection before merging, suggesting the underlying pieces exist, but the evidence never shows the desktop app UI actually presenting a visual diff viewer or multiple sessions arranged side by side. Community notes even flag basic desktop-app reliability issues (stuck on 'Loading projects...', Mac-only availability). Missing for 10: concrete documentation/screenshots of the desktop app's diff viewer, explicit multi-session/side-by-side UI description, and independent confirmation it works smoothly.

   • [github] “If you want the desktop app experience, run <code>codex app</code> or visit the Codex App page.”
   • [github] “If you want the desktop app experience, run <code>codex app</code> or visit <a href="https://chatgpt.com/codex?app-landing-page=true">the Co…”
   • [claimed-docs] “Inspect the summary and diff, request a follow-up, or open a pull request when the result is ready.”
   • [claimed-docs] “Run tasks in isolated cloud environments, work in parallel, and start work from the web, GitHub, GitLab, Linear, or Slack.”
   • [claimed-docs] “Run tasks in parallel without tying up your local machine.”
   • [claimed-docs] “Review changes before they ship: Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized f…”
   • [community] “Genuinely excited to try this out. I've started using Codex much more heavily in the past two months and honestly, it's been shockingly good…”
   • [community] “Mac only. Again. Apple is great but this is OpenAI devs showing their disconnect from the mainstream.”

   proof ↗⚑ flag

   Devin~partialclaimed4/10

   Devin has a documented Desktop app (docs-9) and supports running multiple independent sessions in parallel (docs-25), but there is no evidence of a visual diff review feature or explicit side-by-side session UI within the desktop app itself. Missing for 10: explicit documentation of an in-app diff viewer, UI showing multiple sessions simultaneously in one window, and any hands-on/community confirmation of this desktop workflow.

   • [claimed-docs] “Import VS Code or Cursor settings, configure themes, and start coding with AI-powered assistance.”
   • [claimed-docs] “Devin is designed to be a conversational user interface, and allows you to follow and take over Devin's development process in the embedded …”
   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”

   proof ↗⚑ flag

2. engineering-leadManage multiple agent-driven coding sessions from one unified workspace

   weight 2 · round to Codex
   Codex✓fullcommunity7/10

   Codex documents cloud parallel task execution across multiple repos/environments (codex-docs-1,3,6), a web/CLI dashboard to browse active and completed chats and apply results locally (codex-docs-15), and resuming/searching across sessions (codex-docs-11,24), which together support managing multiple concurrent agent sessions from a unified interface. However, evidence is vendor-documentation only with no independent hands-on confirmation of a true 'unified workspace' UX for an engineering-lead managing many sessions simultaneously, and some community comments note UX rough edges (codex-comm-9,18). Missing for 10: independent/hands-on verification of multi-session management at scale, and clearer detail on cross-session visibility/coordination for a lead overseeing a team's agents.

   • [claimed-docs] “Run tasks in isolated cloud environments, work in parallel, and start work from the web, GitHub, GitLab, Linear, or Slack.”
   • [claimed-docs] “Run tasks in parallel without tying up your local machine.”
   • [claimed-docs] “Delegate a longer task and return when it is ready.”
   • [claimed-docs] “Browse active and completed chats, submit work to a configured environment, and apply the result to your local repository from the terminal.”
   • [claimed-docs] “Reopen a recent chat from the current repository, or search across local chats when you need to return to older work.”
   • [claimed-docs] “`codex resume`: Reopen a recent chat from the current repository, or search across local chats when you need to return to older work.”
   • [community] “I wish Codex App was open source. I like it, but there are always a bunch of little paper cuts that, if you were using codex cli, you could …”

   proof ↗⚑ flag

   Devin~partialcommunity6/10

   Devin supports running multiple parallel sessions ('carve out independent tasks and run them simultaneously'), an API to create sessions on behalf of users, and an embedded IDE/CLI/desktop app to interact with sessions, which together enable a lead-like workspace for managing several agent sessions. However, there's no dedicated 'unified workspace' dashboard evidence for an engineering-lead specifically monitoring/managing a team's multiple concurrent sessions, and community feedback highlights session reliability issues that would complicate multi-session oversight. missing for 10: explicit multi-session dashboard/UI for a lead role, team-level session oversight features, independent corroboration of smooth multi-session management at scale.

   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “Devin is designed to be a conversational user interface, and allows you to follow and take over Devin's development process in the embedded …”
   • [claimed-docs] “Devin CLI, a local command-line coding agent with deep Devin Cloud integration.”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

Terminal workflow

1. developerRun a coding agent locally from my terminal

   weight 3 · round to Codex
   Codex✓fullcommunity9/10

   Codex CLI is explicitly documented as a coding agent that runs locally in the terminal, with npm/standalone install, working against the local repository, editing files, running commands, and offering interactive TUI plus non-interactive exec mode — well corroborated by first-party docs and GitHub README, with community usage discussion confirming real-world use. Missing for 10: independent hands-on verification specifically of pure local terminal usage (most community commentary discusses model quality/UX rather than the local-run mechanics) and some caveats about performance/limits reported by users.

   • [github] “Codex CLI is a coding agent from OpenAI that runs locally on your computer.”
   • [github] “npm install -g @openai/codex”
   • [claimed-docs] “Inspect code, make changes, run commands, and automate repeatable work without leaving your terminal.”
   • [claimed-docs] “Start Codex in a repository to explore unfamiliar code, plan a change, edit files, and run your local development tools.”
   • [claimed-docs] “Work against your local repository: Let Codex inspect files, make edits, and run the tools already installed on your machine.”
   • [claimed-docs] “Compose with scripts and CI: Use Codex interactively or call codex exec from repeatable workflows and pipelines.”
   • [claimed-docs] “Install the Codex CLI with the standalone installer for macOS and Linux.”
   • [community] “Codex is my favorite UX for anything as it edits the files and I can use the proper tooling to adjust and test stuff... However lately the l…”
   • [community] “Genuinely excited to try this out. I've started using Codex much more heavily in the past two months and honestly, it's been shockingly good…”

   proof ↗⚑ flag

   Devin✓fullprobed8/10

   Devin CLI is explicitly documented as a local command-line coding agent that can be invoked from terminal (e.g. `devin -- check out this code...`), with local sandboxing (--sandbox flag) and deep integration with Devin Cloud for handoff. Missing for 10: independent/hands-on community verification specifically of the CLI experience (community evidence only covers the cloud/browser Devin product, not the local CLI).

   • [claimed-docs] “Devin CLI, a local command-line coding agent with deep Devin Cloud integration.”
   • [claimed-docs] “devin -- check out this code and suggest a feasible, helpful feature”
   • [claimed-docs] “a local command-line coding agent with deep Devin Cloud integration”
   • [claimed-docs] “The --sandbox flag runs the CLI with OS-level isolation, enforcing writable paths and deny rules at the operating-system level and optionall…”
   • [claimed-docs] “The `--sandbox` flag runs the CLI with OS-level isolation, enforcing writable paths and `deny` rules at the operating-system level”
   • [probe] “official CLI documented at https://docs.devin.ai/cli/index”

   proof ↗⚑ flag

2. developerRun the agent non-interactively in scripts for workflow automation

   weight 2 · round to Codex
   Codex✓fullclaimed8/10

   Docs explicitly describe running 'a non-interactive command in a repeatable workflow' and automating repeatable work without leaving the terminal, plus support for submitting work to configured environments from scripts (codex exec-style usage implied). Missing for 10: independent hands-on confirmation of non-interactive/CI usage and detailed exit-code/output-format documentation for scripting.

   • [claimed-docs] “Run a non-interactive command in a repeatable workflow.”
   • [claimed-docs] “Inspect code, make changes, run commands, and automate repeatable work without leaving your terminal.”
   • [claimed-docs] “Browse active and completed chats, submit work to a configured environment, and apply the result to your local repository from the terminal.”
   • [github] “Codex CLI is a coding agent from OpenAI that runs locally on your computer.”

   proof ↗⚑ flag

   Devin✓fullclaimed7/10

   Devin exposes a documented API for creating sessions programmatically (including on behalf of users) and explicit CI/CD pipeline integration to auto-respond to static-analysis findings, plus a CLI (`devin -- <prompt>`) that can be invoked headlessly, all pointing to non-interactive, scriptable automation. Missing for 10: independent/hands-on confirmation of headless CLI scripting in real CI pipelines and more detail on CLI exit codes/output for scripting.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Devin CLI, a local command-line coding agent with deep Devin Cloud integration.”
   • [claimed-docs] “devin -- check out this code and suggest a feasible, helpful feature”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “a local command-line coding agent with deep Devin Cloud integration”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Devin
   Codex~partialcommunity3/10

   Codex ships rich CLI/UI-only capabilities (cloud tasks, resume/review, skills, plugins, MCP client integration) with no evidence these are exposed via a dedicated Codex API, and the general OpenAI API (RBAC, Responses API) is not shown to cover Codex-specific workflows; community evidence even confirms the latest gpt-5.3-codex model 'isn't available on the API yet,' a documented parity gap. Missing for 10: documented API endpoints for cloud task delegation, chat/session resume, MCP tool orchestration, and confirmation that current models/features are API-accessible at parity with CLI/UI.

   • [github] “You can also use Codex with an API key, but this requires additional setup.”
   • [community] “gpt-5.3-codex isn't available on the API yet — 'We are working to safely enable API access soon.'”
   • [claimed-docs] “Role-based access control (RBAC) lets you decide who can do what across your organization and projects—both through the API and in the Dashb…”
   • [claimed-docs] “Create a model response — request/response reference with runnable code samples selectable per language: HTTP, Python, TypeScript, Go, Ruby,…”
   • [claimed-docs] “codex mcp-server is deprecated. Use the Codex app server instead. ... This page documents the deprecated command for existing integrations. …”

   proof ↗⚑ flag

   Devin~partialprobed5/10

   Devin exposes a documented API (session creation, create_as_user_id) and an MCP server giving 'full access to session management, playbooks, knowledge, and scheduling', suggesting broad programmatic parity, but there's no explicit claim of full UI/API feature parity, and probes for an OpenAPI spec all 404'd, indicating the full API surface isn't transparently documented. Missing for 10: explicit parity statement covering UI-only features like Devin Review/Auto-Fix/Computer Use/desktop app settings, a discoverable OpenAPI schema, and independent confirmation that all UI actions are API-reachable.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round drawn
   Codex—none0/10

   No evidence of any data export feature or open-format export mechanism for chat history, project data, or configurations; Codex works with local files/git repos but there's no documented export/portability capability for user data (e.g., conversation history, settings) to leave the platform. Missing for 10: any documented data export tool, open-format export (JSON/Markdown dump), or data portability statement.

   ⚑ flag

   Devin—none0/10

   No evidence of a data export feature or open-format export/portability mechanism for Devin sessions, knowledge, or artifacts; the docs cover API, CLI, MCP, and infrastructure but nothing about exporting all user data in open formats to leave the platform.

   ⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round to Codex
   Codex~partialcommunity4/10

   The Codex CLI source lives in a public GitHub repo (openai/codex) and a community comment implies its openness lets users 'diagnose and file an issue' the way they can't with the closed-source Codex App, suggesting at least the CLI's code is publicly viewable. However, no evidence pack item states an explicit open-source license, and the App/cloud components are explicitly described as closed. missing for 10: explicit license file/name (MIT, Apache, etc.), confirmation the full product (not just CLI) is open, and independent corroboration beyond one forum remark.

   • [github] “Codex CLI is a coding agent from OpenAI that runs locally on your computer.”
   • [community] “I wish Codex App was open source. I like it, but there are always a bunch of little paper cuts that, if you were using codex cli, you could …”

   proof ↗⚑ flag

   Devin—none0/10

   Devin is a closed, proprietary commercial product; no evidence indicates its source code is available under an open license. The only mention of 'open-source' is for the separate Devin Handoff plugin/skill, not Devin itself.

   • [claimed-docs] “Devin Handoff is an open-source plugin and skill that brings the same handoff workflow to any coding agent — Claude Code, Codex, Cursor, and…”

   proof ↗⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round drawn
   Codex—none0/10

   Codex CLI runs locally but requires signing into a ChatGPT account or OpenAI API key, and the core inference/model and cloud environments are OpenAI-hosted only; there is no self-hosted backend option. A commenter explicitly wishes the Codex App were open source, implying it is not, which forecloses self-hosting the core product.

   • [github] “We recommend signing into your ChatGPT account to use Codex as part of your Plus, Pro, Business, Edu, or Enterprise plan.”
   • [github] “You can also use Codex with an API key, but this requires additional setup.”
   • [community] “I wish Codex App was open source. I like it, but there are always a bunch of little paper cuts that, if you were using codex cli, you could …”

   proof ↗⚑ flag

   Devin—none0/10

   Devin is a cloud-based SaaS agent; Outposts lets you run sessions on your own infrastructure but the core Devin model/orchestration itself remains Cognition-hosted, and there's no evidence of a self-hostable core product/model package. No mention of on-prem/self-hosted deployment of the core Devin engine.

   • [claimed-docs] “Outposts lets you run Devin sessions inside infrastructure you control — your own VMs, containers, Kubernetes clusters, or even a Mac Mini o…”

   proof ↗⚑ flag

Authentication

1. developerAuthenticate with an API key instead of an account login

   weight 2 · round to Devin
   Codex~partialcommunity5/10

   GitHub docs confirm Codex CLI supports API key authentication as an alternative to ChatGPT account login, but note it 'requires additional setup,' and the account-login flow (Sign in with ChatGPT) is the recommended default. Missing for 10: detailed API-key setup documentation, first-party quickstart parity with account login, and independent confirmation that API-key auth is fully feature-equivalent (e.g. codex-comm-5 shows some newer models aren't even available via API yet).

   • [github] “We recommend signing into your ChatGPT account to use Codex as part of your Plus, Pro, Business, Edu, or Enterprise plan.”
   • [github] “You can also use Codex with an API key, but this requires additional setup.”
   • [community] “gpt-5.3-codex isn't available on the API yet — 'We are working to safely enable API access soon.'”

   proof ↗⚑ flag

   Devin~partialprobed6/10

   Devin has a documented API (devin-docs-7) intended for programmatic integration, implying API key auth as an alternative to account login, and even supports creating sessions on behalf of other users (devin-docs-8), suggesting a service-level credential model. However, no explicit documentation of API key generation/management or authentication mechanics is present in the evidence pack. Missing for 10: explicit API key creation/management docs, confirmation that API key auth fully replaces login flows, and independent/community verification of this workflow.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

2. engineering-leadAuthenticate through an enterprise identity or cloud platform for compliance and scalability

   weight 2 · round to Codex
   Codex~partialclaimed5/10

   Codex supports signing in with a ChatGPT Business/Enterprise/Edu account (codex-gh-3, codex-gh-7) and OpenAI's platform offers RBAC to scope access at org/project level (codex-docs-28), suggesting enterprise-grade authentication and access control exist. However, there is no explicit documentation of SSO/SAML/OIDC federation with enterprise identity providers (e.g., Okta, Azure AD) specific to Codex, nor details on how ChatGPT Enterprise auth ties into RBAC for Codex usage. Missing for 10: explicit SSO/SAML/OIDC integration docs, enterprise IdP federation details, and independent confirmation of compliance-grade auth flows for Codex specifically.

   • [github] “We recommend signing into your ChatGPT account to use Codex as part of your Plus, Pro, Business, Edu, or Enterprise plan.”
   • [github] “Run `codex` and select **Sign in with ChatGPT**. We recommend signing into your ChatGPT account to use Codex as part of your Plus, Pro, Busi…”
   • [claimed-docs] “Role-based access control (RBAC) lets you decide who can do what across your organization and projects—both through the API and in the Dashb…”

   proof ↗⚑ flag

   Devin—none0/10

   No evidence of SSO/SAML, OIDC, or enterprise identity provider integration (e.g., Okta, Azure AD, Google Workspace) for authentication; the only related item is API session creation on behalf of a user, which is not enterprise identity/SSO authentication.

   ⚑ flag

3. developerSign in with my existing product subscription plan to use the coding agent

   weight 2 · round to Codex
   Codex✓fullclaimed9/10

   GitHub docs explicitly recommend signing in with ChatGPT to use Codex under existing Plus, Pro, Business, Edu, or Enterprise subscription plans, with API key as an alternative for those without such plans, directly confirming subscription-based sign-in. missing for 10: independent hands-on confirmation of the sign-in flow itself (evidence focuses on capability descriptions rather than a walkthrough).

   • [github] “We recommend signing into your ChatGPT account to use Codex as part of your Plus, Pro, Business, Edu, or Enterprise plan.”
   • [github] “Run `codex` and select **Sign in with ChatGPT**. We recommend signing into your ChatGPT account to use Codex as part of your Plus, Pro, Busi…”
   • [github] “Run codex and select Sign in with ChatGPT. We recommend signing into your ChatGPT account to use Codex as part of your Plus, Pro, Business, …”
   • [github] “You can also use Codex with an API key, but this requires additional setup.”

   proof ↗⚑ flag

   Devin—none0/10

   The evidence describes Devin's own subscription tiers (Pro, Max, Core/Team/Enterprise) but nothing about letting a developer sign in using an existing external subscription (e.g., an existing LLM provider or IDE subscription) to access the agent. No mention of SSO-linked billing or bring-your-own-subscription support.

   • [claimed-docs] “Max is for individual users who consistently exceed the Pro quota. It includes everything in Pro, plus a significantly larger weekly usage q…”
   • [claimed-docs] “Power users who need more quota”

   proof ↗⚑ flag

4. developerSign in with a personal account to get free-tier access without managing API keys

   weight 1 · round to Codex
   Codex✓fullclaimed8/10

   Codex CLI explicitly recommends signing in with a ChatGPT account (Plus/Pro/Business/Edu/Enterprise) to use Codex without an API key, with API key usage noted as an alternative requiring additional setup. This directly matches the story of personal-account sign-in without managing API keys, though the exact free-tier scope/limits aren't detailed. Missing for 10: explicit confirmation of a genuinely free tier (vs. paid ChatGPT plans) and independent corroboration of the login flow's simplicity.

   • [github] “We recommend signing into your ChatGPT account to use Codex as part of your Plus, Pro, Business, Edu, or Enterprise plan.”
   • [github] “You can also use Codex with an API key, but this requires additional setup.”
   • [github] “Run `codex` and select **Sign in with ChatGPT**. We recommend signing into your ChatGPT account to use Codex as part of your Plus, Pro, Busi…”

   proof ↗⚑ flag

   Devin—none0/10

   Evidence only describes paid tiers (Pro, Max, 'power users who need more quota') and API key based integration; there's no mention of a free tier accessible via personal account sign-in without API key management.

   • [claimed-docs] “Max is for individual users who consistently exceed the Pro quota. It includes everything in Pro, plus a significantly larger weekly usage q…”
   • [claimed-docs] “Power users who need more quota”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”

   proof ↗⚑ flag

Model choice

1. developerLet the tool automatically pick the best model for each task

   weight 1 · round drawn
   Codex—none0/10

   Evidence shows Codex lets users manually choose the model and reasoning effort ('Stay in control: Choose the model, reasoning effort, permissions...') rather than any automatic best-model-per-task selection; no docs or community evidence describe an automatic model-routing/selection feature tied to cost or task type.

   • [claimed-docs] “Stay in control: Choose the model, reasoning effort, permissions, and commands that fit the task.”
   • [community] “The main issue I have with Codex is that the best model is insanely slow, except at nights and weekends when Silicon Valley goes to bed... I…”
   • [community] “First thoughts using gpt-5.3-codex-spark in Codex CLI: Blazing fast but it definitely has a small model feel... It has to be prompted to do …”

   proof ↗⚑ flag

   Devin—none0/10

   No evidence that Devin offers automatic model selection or model-routing based on task complexity; docs discuss quotas, tiers (Pro/Max), and features but never mention choosing among multiple underlying models per task.

   ⚑ flag

2. developerChoose which underlying AI model powers my session from multiple providers

   weight 2 · round drawn
   Codex—none0/10

   Docs confirm Codex lets users 'Choose the model, reasoning effort, permissions' (codex-docs-31), but this refers to selecting among OpenAI's own Codex/GPT models, not switching between different AI providers (e.g., Anthropic, Google). No evidence shows Codex supports plugging in or selecting non-OpenAI models/providers within a session.

   • [claimed-docs] “Stay in control: Choose the model, reasoning effort, permissions, and commands that fit the task.”
   • [github] “We recommend signing into your ChatGPT account to use Codex as part of your Plus, Pro, Business, Edu, or Enterprise plan.”
   • [github] “You can also use Codex with an API key, but this requires additional setup.”

   proof ↗⚑ flag

   Devin—none0/10

   No evidence in the pack mentions selecting or switching between multiple underlying AI model providers (e.g., choosing between different LLMs) for a Devin session; documentation covers agent capabilities, environments, and billing tiers but not model selection.

   ⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round drawn
   Codex—none0/10

   No evidence in the pack mentions data residency, regional storage options, or geographic controls for where Codex data is stored; the pack covers RBAC, MCP, CLI features, and cloud task execution but nothing about choosing a storage region.

   ⚑ flag

   Devin—none0/10

   No evidence of data residency/region selection controls for where Devin stores data; the pack covers deployment options (Outposts, VPN, sandbox isolation) but nothing about choosing storage region/jurisdiction.

   ⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   Codex—none0/10

   The evidence pack contains no mention of data-training opt-out controls, enterprise data usage policies, or privacy settings for excluding user data from model training; it covers CLI features, MCP, RBAC, and community sentiment but nothing about training-data exclusion.

   ⚑ flag

   Devin—none0/10

   The evidence pack contains no mention of data-training opt-out, data usage policy, or privacy controls for AI model training; nothing in the docs or community sources addresses this axis.

   ⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round drawn
   Codex—none0/10

   No evidence pack items address data retention controls, deletion policies, or configurable retention windows for Codex; RBAC docs address access control, not retention/deletion. Missing for 10: any documentation of data retention settings, deletion APIs/workflows, or retention policy configuration.

   ⚑ flag

   Devin—none0/10

   No evidence pack items address data retention policies, data deletion controls, or privacy/data-handling settings for user data or session data; the closest items concern sandboxing/isolation of execution environments, not retention/deletion of stored data.

   ⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Codex—none0/10

   No evidence in the pack mentions telemetry, usage tracking, data collection settings, or an opt-out mechanism for Codex; the docs and community threads cover features like MCP, CLI usage, and performance but never privacy/telemetry controls.

   ⚑ flag

   Devin—none0/10

   No evidence pack item mentions telemetry opt-out, usage tracking settings, or privacy controls of that kind; the docs cover features, sandboxing, and infrastructure but not data-collection opt-out mechanisms.

   ⚑ flag

Data governance

1. engineering-leadOpt out of having my code and prompts used for AI model training

   weight 1 · round drawn
   Codex—none0/10

   No evidence in the pack addresses data usage or training opt-out policies for code/prompts; RBAC and MCP docs are unrelated to this axis. Missing for 10: any enterprise data-usage/training opt-out policy documentation, admin controls for opting out, or third-party confirmation of such a policy.

   ⚑ flag

   Devin—none0/10

   No evidence in the pack addresses data usage, training opt-out, or privacy/data-handling policies for Devin; nothing documents an opt-out mechanism for code/prompt use in model training.

   ⚑ flag

Pr review

1. developerHave the agent stage changes, write commit messages, create branches, and open pull requests

   weight 3 · round drawn
   Codex~partialclaimed6/10

   Codex docs explicitly describe inspecting diffs and opening a pull request when cloud work is ready (codex-docs-5), and CLI docs note reviewing changes 'before you commit or open a pull request' (codex-docs-45), implying git workflow integration. However, staging changes, writing commit messages, and creating branches are not explicitly documented as first-class agent actions — they are only implied via general local repo access and command execution (codex-docs-9, codex-docs-30, codex-docs-17). Missing for 10: explicit documentation of commit-message generation, branch creation, and staging as named agent capabilities, plus independent hands-on confirmation of full PR workflow automation.

   • [claimed-docs] “Inspect the summary and diff, request a follow-up, or open a pull request when the result is ready.”
   • [claimed-docs] “Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized findings without modifying your wo…”
   • [claimed-docs] “Review changes before they ship: Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized f…”
   • [claimed-docs] “Start Codex in a repository to explore unfamiliar code, plan a change, edit files, and run your local development tools.”
   • [claimed-docs] “Work against your local repository: Let Codex inspect files, make edits, and run the tools already installed on your machine.”

   proof ↗⚑ flag

   Devin~partialcommunity6/10

   Devin's docs describe it implementing features/fixing bugs and producing pull requests that get automated review and iteration (devin-docs-17, devin-docs-18), implying it handles the full git workflow (branch, commit, PR) autonomously, but no doc explicitly details staging, commit-message generation, or branch creation as discrete steps. Community feedback (devin-comm-1) also notes it can add extraneous changes it can't cleanly undo, a real caveat on commit hygiene. Missing for 10: explicit documentation of commit/staging/branch mechanics and independent confirmation that generated commits/PRs are clean and reviewable.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Devin Review provides automated first-pass reviews on pull requests, checking for correctness and conformance with organizational best pract…”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”

   proof ↗⚑ flag

2. developerGet automatic code review with contextual feedback on every pull request

   weight 3 · round to Devin
   Codex~partialclaimed5/10

   Codex CLI/cloud ships a dedicated 'review' capability that inspects uncommitted changes, a commit, or a base branch and reports prioritized findings without touching the working tree, and cloud tasks can be kicked off from GitHub PRs and later opened as PRs. However, there is no evidence of an automatic, PR-triggered review bot that comments on every pull request without manual invocation. missing for 10: evidence of automatic triggering on every PR (e.g., GitHub App/webhook auto-review), evidence of inline PR comments, independent confirmation of review quality on real PRs.

   • [claimed-docs] “Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized findings without modifying your wo…”
   • [claimed-docs] “Review changes before they ship: Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized f…”
   • [claimed-docs] “Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized findings without modifying your wo…”
   • [claimed-docs] “Run tasks in isolated cloud environments, work in parallel, and start work from the web, GitHub, GitLab, Linear, or Slack.”
   • [claimed-docs] “Start work in Codex cloud from GitHub pull requests, GitLab merge requests and issues, Linear issues, or Slack channels and threads.”
   • [claimed-docs] “Inspect the summary and diff, request a follow-up, or open a pull request when the result is ready.”

   proof ↗⚑ flag

   Devin✓fullclaimed8/10

   Devin Review is explicitly documented as an automated first-pass PR reviewer checking correctness and org best-practice conformance, with Auto-Fix closing the loop by responding to review comments and CI failures. This directly matches the story of automatic contextual code review on PRs, though evidence is vendor-documentation only with no independent hands-on validation of review quality/contextual accuracy. Missing for 10: independent/community corroboration of review quality, and detail on how 'contextual feedback' is surfaced per-PR beyond docs description.

   • [claimed-docs] “Devin Review provides automated first-pass reviews on pull requests, checking for correctness and conformance with organizational best pract…”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixe”

   proof ↗⚑ flag

3. developerInspect diffs and run checks to catch problems before merging

   weight 3 · round drawn
   Codex✓fullclaimed8/10

   Codex CLI has a dedicated review command that inspects diffs against uncommitted changes, a commit, or a base branch, reporting prioritized findings without modifying the working tree, plus cloud/web flows to inspect summaries and diffs before opening a PR. Missing for 10: independent/hands-on corroboration of the review command's accuracy and any CI-integrated check-running beyond exec/scripts.

   • [claimed-docs] “Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized findings without modifying your wo…”
   • [claimed-docs] “Review changes before they ship: Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized f…”
   • [claimed-docs] “Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized findings without modifying your wo…”
   • [claimed-docs] “Inspect the summary and diff, request a follow-up, or open a pull request when the result is ready.”
   • [claimed-docs] “Compose with scripts and CI: Use Codex interactively or call codex exec from repeatable workflows and pipelines.”

   proof ↗⚑ flag

   Devin✓fullclaimed8/10

   Devin's docs describe Devin Review, which performs automated first-pass PR reviews checking correctness and org conformance, plus CI/CD integration to respond to static-analysis findings (SonarQube, Fortify, Veracode) and Auto-Fix iterating on CI failures — directly enabling developers to inspect diffs and run checks before merge. This is corroborated by explicit SDLC integration workflow docs, not just a single mention. missing for 10: independent/hands-on evidence confirming Devin Review's diff-inspection quality in practice, and detail on how diffs are surfaced/inspected by the developer (UI specifics) beyond docs claims.

   • [claimed-docs] “Devin Review provides automated first-pass reviews on pull requests, checking for correctness and conformance with organizational best pract…”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixe”

   proof ↗⚑ flag

Safe execution

1. engineering-leadControl which external tools and integrations the agent is allowed to access

   weight 2 · round drawn
   Codex~partialclaimed6/10

   Codex documents fine-grained control over external tool access at the session/repo level: engineers can add/remove local or remote MCP servers, inspect available tools before they're used, and set permission boundaries for edits/commands via /permissions (codex-docs-16, codex-docs-38, codex-docs-39, codex-docs-46). This gives an engineer meaningful control over which integrations the agent can reach, and RBAC exists for org/project-level API access (codex-docs-28), but that RBAC is about API/dashboard permissions, not specifically about restricting agent tool/integration access org-wide for a lead managing a team's Codex usage. Missing for 10: evidence of centralized, lead-enforced policy that restricts which MCP servers/tools individual developers can enable (vs. per-session self-configuration), and independent confirmation this control actually prevents unauthorized tool access in practice.

   • [claimed-docs] “Add local or remote MCP servers, authenticate when needed, and inspect the tools available to the current session before Codex uses them.”
   • [claimed-docs] “Connect external tools with MCP — codex mcp: Add local or remote MCP servers, authenticate when needed, and inspect the tools available to t…”
   • [claimed-docs] “Set the boundaries for each run — /permissions: Choose when Codex can edit files or run commands without asking, and inspect the active sand…”
   • [claimed-docs] “In the `codex` TUI, use `/mcp` to see your active MCP servers.”
   • [claimed-docs] “Role-based access control (RBAC) lets you decide who can do what across your organization and projects—both through the API and in the Dashb…”

   proof ↗⚑ flag

   Devin~partialclaimed6/10

   Devin exposes some admin-level controls over its access—an org-wide 'Enable desktop mode' toggle for Computer Use, a CLI --sandbox flag enforcing OS-level writable-path/deny rules and network restriction, and Outposts for running sessions in infra you control—giving leads levers to constrain what Devin can reach or do. However there's no documented centralized policy/allowlist for specific external integrations (e.g., disabling Slack, Jira, GitHub, VPN access per-tool) or granular permission/audit management for engineering leads. missing for 10: a unified integration-permission/allowlist admin panel, per-tool enable/disable controls beyond desktop mode, and independent verification of these controls in practice.

   • [claimed-docs] “Computer Use is controlled by the Enable desktop mode toggle in your organization's customization options.”
   • [claimed-docs] “The --sandbox flag runs the CLI with OS-level isolation, enforcing writable paths and deny rules at the operating-system level and optionall…”
   • [claimed-docs] “The `--sandbox` flag runs the CLI with OS-level isolation, enforcing writable paths and `deny` rules at the operating-system level”
   • [claimed-docs] “Outposts lets you run Devin sessions inside infrastructure you control — your own VMs, containers, Kubernetes clusters, or even a Mac Mini o…”
   • [claimed-docs] “Devin can connect to a VPN from inside its workspace, so sessions can reach internal services such as package registries, databases, and int…”

   proof ↗⚑ flag

2. engineering-leadHave the agent operate inside a sandbox when interacting with code, tools, and network resources

   weight 2 · round to Devin
   Codex✓fullcommunity7/10

   First-party docs explicitly describe sandboxed execution: Codex lets you 'choose when Codex can edit files or run commands without asking, and inspect the active sandbox and writable roots' (codex-docs-17), and cloud tasks run in 'isolated cloud environments' with configurable dependencies/tools (codex-docs-1, codex-docs-4). This directly matches the engineering-lead's need for sandboxed code/tool interaction, though network-resource sandboxing specifics are not spelled out and there's no independent hands-on verification of sandbox robustness (a community comment raises but does not concretely confirm a sandbox-bypass issue). Missing for 10: explicit documentation of network-level sandbox controls, and independent/hands-on confirmation that the sandbox reliably contains tool/network access.

   • [claimed-docs] “Choose when Codex can edit files or run commands without asking, and inspect the active sandbox and writable roots before you continue.”
   • [claimed-docs] “Run tasks in isolated cloud environments, work in parallel, and start work from the web, GitHub, GitLab, Linear, or Slack.”
   • [claimed-docs] “Configure the dependencies, tools, variables, and setup steps each repository needs.”
   • [community] “Do people really want codex to have control over their computer and apps? I'm still paranoid about keeping things securely sandboxed.”

   proof ↗⚑ flag

   Devin✓fullclaimed8/10

   Devin runs cloud sessions in isolated VMs (devin-docs-13) and provides an explicit CLI --sandbox flag enforcing OS-level isolation, writable path restrictions, deny rules, and optional network restriction (devin-docs-12, devin-docs-37), directly matching the sandboxed code/tool/network isolation story. Missing for 10: independent/hands-on verification of sandbox robustness and more detail on network isolation guarantees beyond docs claims.

   • [claimed-docs] “The --sandbox flag runs the CLI with OS-level isolation, enforcing writable paths and deny rules at the operating-system level and optionall…”
   • [claimed-docs] “The `--sandbox` flag runs the CLI with OS-level isolation, enforcing writable paths and `deny` rules at the operating-system level”
   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “Devin can connect to a VPN from inside its workspace, so sessions can reach internal services such as package registries, databases, and int…”

   proof ↗⚑ flag

Security checks

1. engineering-leadSee license and public-code matching references for AI-suggested code

   weight 1 · round drawn
   Codex—none0/10

   No evidence anywhere in the pack mentions license detection, public-code matching, or provenance references for AI-suggested code; Codex's review features (codex-docs-10, -41, -45) only cover code quality/prioritized findings, not license/public-code attribution.

   ⚑ flag

   Devin—none0/10

   No evidence of license or public-code matching/attribution references for AI-suggested code anywhere in the docs; Devin Review covers correctness and best-practice conformance but nothing about license/provenance matching.

   ⚑ flag

2. developerGet contextual explanations and automatic fixes for security vulnerabilities

   weight 2 · round to Devin
   Codex~partialclaimed3/10

   Codex CLI has a dedicated review command that inspects uncommitted changes, commits, or branches and reports 'prioritized findings' (codex-docs-10, codex-docs-41, codex-docs-45), which could surface security issues, and as a general coding agent it can edit files/run commands. However, the review feature explicitly reports findings 'without modifying your working tree,' meaning it does not auto-fix, and no evidence specifically frames this as security-vulnerability detection/explanation with automatic remediation. missing for 10: explicit security-vulnerability scanning/explanation feature, evidence of automatic fix application (vs. just flagging), and any independent confirmation that Codex reliably identifies/fixes security issues.

   • [claimed-docs] “Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized findings without modifying your wo…”
   • [claimed-docs] “Review changes before they ship: Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized f…”
   • [claimed-docs] “Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized findings without modifying your wo…”
   • [claimed-docs] “Work against your local repository: Let Codex inspect files, make edits, and run the tools already installed on your machine.”

   proof ↗⚑ flag

   Devin✓fullclaimed7/10

   Devin's docs describe Devin Review giving automated PR reviews with explanations for correctness/best-practice issues, Auto-Fix automatically responding to review comments and fixing flagged bugs/CI failures, and CI/CD integration to respond to findings from security scanners like SonarQube, Fortify, and Veracode — directly matching contextual explanation plus automatic fixing of vulnerabilities. Missing for 10: independent/hands-on evidence specifically validating security-vulnerability fixes (community evidence only discusses general reliability/scope-creep issues, not security-fix accuracy).

   • [claimed-docs] “Devin Review provides automated first-pass reviews on pull requests, checking for correctness and conformance with organizational best pract…”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Ask Devin can answer questions about code structure and dependencies, and help you scope and plan tasks before implementation.”

   proof ↗⚑ flag