Devin vs Conductor

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round drawn
   Devin✓fullprobed9/10

   Devin's own docs site serves llms.txt (HTTP 200, confirmed by probe) and per-page .md variants, and also supports AGENTS.md as an agent-oriented instructions standard, directly matching the story. Missing for 10: no independent/community confirmation that external agents have actually consumed llms.txt successfully.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.devin.ai/llms.txt # Devin Docs - [Desktop (100 pages)](https://docs.devin.ai/_llms/en/desktop.md):…”
   • [probe] “PROBE docs-md: HTTP 200 at https://docs.devin.ai/get-started/devin-intro.md > ## Documentation Index > Fetch the complete documentation inde…”
   • [claimed-docs] “Devin supports AGENTS.md - a simple, open standard for providing context and instructions to AI agents.”

   proof ↗⚑ flag

   Conductor✓fullprobed9/10

   Direct probe confirms llms.txt is live and served at https://www.conductor.build/llms.txt with agent-oriented summary, plus a full docs.md markdown mirror for agent consumption. missing for 10: no independent/community confirmation that external agents actually consume these files successfully.

   • [probe] “PROBE llms.txt: HTTP 200 at https://www.conductor.build/llms.txt # Conductor > Conductor is a Mac app that lets you run many coding agents …”
   • [probe] “PROBE docs-md: HTTP 200 at https://www.conductor.build/docs.md --- title: "Introduction" url: "/docs" description: "Learn what Conductor is …”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to Devin
   Devin✓fullclaimed8/10

   Devin offers a full API for creating/managing sessions programmatically (including create_as_user_id for automation on behalf of users), a CLI with a --sandbox flag for OS-level isolated headless runs, and explicit CI/CD pipeline integration for responding to static analysis findings and PR checks, all supporting headless/automated usage without a human in the loop. Missing for 10: no independent/hands-on report specifically validating CI automation workflows end-to-end, and no explicit CI example (e.g., GitHub Actions snippet) beyond doc references.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “The --sandbox flag runs the CLI with OS-level isolation, enforcing writable paths and deny rules at the operating-system level and optionall…”
   • [claimed-docs] “a local command-line coding agent with deep Devin Cloud integration”

   proof ↗⚑ flag

   Conductor~partialprobed6/10

   Conductor supports scheduled/CI-like automation via 'routines' that run on a schedule or GitHub Action, plus a programmatic API and hosted MCP server for managing cloud workspaces headlessly, and cloud agents can run builds/tests without confirmation. However, it is fundamentally a Mac GUI app, and there's no evidence of a standalone CLI or true headless binary for arbitrary CI pipelines outside GitHub Actions. missing for 10: dedicated CLI/headless binary for generic CI systems, independent evidence of routines/GitHub Action working reliably in production, clarity on full non-interactive operation outside the Mac app.

   • [claimed-docs] “Introducing routines! You can now run your agents on a schedule or via GitHub action.”
   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically.”
   • [claimed-docs] “Conductor's hosted Model Context Protocol (MCP) server lets ChatGPT, Claude, Codex, and other MCP clients manage your cloud workspaces.”
   • [claimed-docs] “The agent can test your repositories, update install and setup scripts, and run builds — without asking you to confirm each step.”
   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically. Use it to do things like: Create workspaces, send prompts to the coding…”
   • [probe] “official MCP server documented at https://www.conductor.build/docs/api/mcp”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round drawn
   Devin—none0/10

   The evidence only documents Devin exposing its own MCP server so other agents/IDEs can call Devin's tools (session management, playbooks, knowledge, scheduling) — the reverse direction of this story. There is no evidence that a user can configure Devin itself to consume/plug in external MCP servers so Devin can use their tools.

   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [probe] “official MCP server documented at https://docs.devin.ai/work-with-devin/devin-mcp”

   proof ↗⚑ flag

   Conductor—none0/10

   Evidence only shows Conductor exposing its OWN hosted MCP server so external MCP clients (ChatGPT, Claude, Codex) can manage Conductor's cloud workspaces (conductor-docs-14, conductor-probe-4) — the reverse direction of what the story asks. There is no documentation or community mention of a user being able to add/configure external MCP servers inside Conductor so its hosted coding agents (Claude Code, Codex, Cursor, OpenCode) can consume their tools.

   • [claimed-docs] “Conductor's hosted Model Context Protocol (MCP) server lets ChatGPT, Claude, Codex, and other MCP clients manage your cloud workspaces.”
   • [probe] “official MCP server documented at https://www.conductor.build/docs/api/mcp”

   proof ↗⚑ flag

4. ai-native userConnect an agent via an official MCP server

   weight 3 · round drawn
   Devin✓fullprobed8/10

   Devin ships an official documented MCP server (devin-mcp) that gives any MCP-compatible agent or IDE full access to session management, playbooks, knowledge, and scheduling, confirmed both in docs and via probe. Missing for 10: independent/hands-on third-party corroboration of the MCP server working in practice, and detail on setup/auth specifics.

   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [probe] “official MCP server documented at https://docs.devin.ai/work-with-devin/devin-mcp”

   proof ↗⚑ flag

   Conductor✓fullprobed8/10

   Conductor documents a hosted MCP server that lets ChatGPT, Claude, Codex, and other MCP clients manage cloud workspaces, corroborated by a dedicated probe hit confirming the docs page exists. Missing for 10: independent/hands-on community confirmation of actually connecting an external agent via this MCP server (all community evidence discusses other features, not MCP usage).

   • [claimed-docs] “Conductor's hosted Model Context Protocol (MCP) server lets ChatGPT, Claude, Codex, and other MCP clients manage your cloud workspaces.”
   • [probe] “official MCP server documented at https://www.conductor.build/docs/api/mcp”

   proof ↗⚑ flag

5. ai-native userUse an official CLI

   weight 2 · round to Devin
   Devin✓fullprobed8/10

   Devin has an official CLI ('Devin CLI, a local command-line coding agent with deep Devin Cloud integration') with documented usage examples and a sandbox flag for OS-level isolation, confirmed by both docs and probe. Missing for 10: independent/hands-on community verification of the CLI specifically (community evidence covers the web/session product, not CLI usage).

   • [claimed-docs] “Devin CLI, a local command-line coding agent with deep Devin Cloud integration.”
   • [claimed-docs] “devin -- check out this code and suggest a feasible, helpful feature”
   • [claimed-docs] “The --sandbox flag runs the CLI with OS-level isolation, enforcing writable paths and deny rules at the operating-system level and optionall…”
   • [claimed-docs] “a local command-line coding agent with deep Devin Cloud integration”
   • [probe] “official CLI documented at https://docs.devin.ai/cli/index”

   proof ↗⚑ flag

   Conductor—none0/10

   Conductor is documented as a Mac GUI app with a programmatic API and hosted MCP server, but no evidence pack item describes an official Conductor CLI tool; the only CLI mention is a user leveraging their own 'local GitHub CLI auth', which is unrelated.

   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically.”
   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically. Use it to do things like: Create workspaces, send prompts to the coding…”
   • [community] “Fixed! You can now give Conductor fine-grained GitHub repository access. Or, skip the integration and use your local GitHub CLI auth.”

   proof ↗⚑ flag

6. ai-native userDrive the product through a documented public API

   weight 3 · round to Devin
   Devin✓fullprobed8/10

   Devin ships a documented public API (docs-7) with session creation, org-level features like create_as_user_id (docs-8), plus a CLI and MCP server for programmatic/agentic control (docs-4, docs-6, probe-4, probe-5), directly enabling AI-native users to drive it programmatically. Missing for 10: a discoverable OpenAPI/swagger spec (probe-3 found 404s on all candidate paths) and independent hands-on corroboration of API usage.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “Devin CLI, a local command-line coding agent with deep Devin Cloud integration.”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [probe] “official MCP server documented at https://docs.devin.ai/work-with-devin/devin-mcp”
   • [probe] “official CLI documented at https://docs.devin.ai/cli/index”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

   Conductor✓fullprobed7/10

   Conductor documents a public API for programmatically managing cloud workspaces (create workspaces, send prompts, read agent replies) plus a hosted MCP server for AI clients like ChatGPT/Claude/Codex to drive it. Missing for 10: a published OpenAPI/reference spec (probe found only 404s for schema files) and independent/hands-on developer corroboration of API usage.

   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically.”
   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically. Use it to do things like: Create workspaces, send prompts to the coding…”
   • [claimed-docs] “Conductor's hosted Model Context Protocol (MCP) server lets ChatGPT, Claude, Codex, and other MCP clients manage your cloud workspaces.”
   • [probe] “official MCP server documented at https://www.conductor.build/docs/api/mcp”
   • [probe] “PROBE openapi: all candidate paths 404 (https://www.conductor.build/openapi.json, https://www.conductor.build/swagger.json, https://www.cond…”

   proof ↗⚑ flag

7. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round to Conductor
   Devin—none0/10

   Devin exposes a general API (devin-docs-7) and can act on behalf of a specified user via create_as_user_id (devin-docs-8), but there is no evidence of scoped/least-privilege API key or token issuance, role-based permission scopes, or credential-level restriction mechanisms for agent access.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”

   proof ↗⚑ flag

   Conductor~partialcommunity4/10

   Community threads document that Conductor originally required full read/write GitHub access with no fine-grained scoping, which users flagged as risky; the developers later added a GitHub App integration for fine-grained repo access (or use of local GitHub CLI auth) as a fix, showing partial progress toward least-privilege credentials but not a documented, general mechanism for issuing scoped API credentials for agents beyond GitHub repo access. Missing for 10: no documentation of scoped/least-privilege credentials for the Conductor API/MCP server itself, no explicit policy on token scoping for non-GitHub integrations, and no independent verification that the new GitHub App permissions are truly minimal in practice.

   • [community] “Any way to have it not require full write access to your entire GitHub account?”
   • [community] “Full read-write access required to all your Github account's repos. Not just code. Settings, deploy keys. The works... Zero disclosure of da…”
   • [community] “I was really excited to try this but this does NOT work the way I expected. I wanted a simple git worktree manager for my existing, already-…”
   • [community] “Right now the app uses GitHub's OAuth sign in which unfortunately doesn't allow for fine-grained permissions... We're switching our sign-in …”
   • [community] “Fixed! You can now give Conductor fine-grained GitHub repository access. Or, skip the integration and use your local GitHub CLI auth.”
   • [claimed-docs] “Bring your own subscriptions and keys”

   proof ↗⚑ flag

8. ai-native userBuild against official SDKs

   weight 2 · round to Conductor
   Devin~partialprobed4/10

   Devin provides an API for integration (docs-7, docs-8) allowing developers to build applications and automate workflows, but the evidence never mentions dedicated official SDKs/client libraries (e.g., Python/JS packages), and probes for an OpenAPI spec that would back SDK generation all returned 404s (devin-probe-3). Missing for 10: named SDK packages in specific languages, SDK installation/usage docs, and a published OpenAPI/schema artifact confirming SDK-generation support.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

   Conductor~partialprobed5/10

   Conductor documents an official REST-style API for managing cloud workspaces and sending/reading agent prompts, plus a hosted MCP server for AI clients, which supports building AI-native integrations. However, no dedicated client SDK packages (e.g., npm/python libraries) are evidenced, and a probe for an OpenAPI spec returned 404s, suggesting the 'SDK' is really just a raw API/MCP interface rather than a polished, language-specific SDK. missing for 10: official language SDK packages, OpenAPI/schema-based codegen support, independent hands-on confirmation of SDK usage.

   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically.”
   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically. Use it to do things like: Create workspaces, send prompts to the coding…”
   • [claimed-docs] “Conductor's hosted Model Context Protocol (MCP) server lets ChatGPT, Claude, Codex, and other MCP clients manage your cloud workspaces.”
   • [probe] “official MCP server documented at https://www.conductor.build/docs/api/mcp”
   • [probe] “PROBE openapi: all candidate paths 404 (https://www.conductor.build/openapi.json, https://www.conductor.build/swagger.json, https://www.cond…”

   proof ↗⚑ flag

9. ai-native userSubscribe to events via webhooks

   weight 2 · round drawn
   Devin—none0/10

   No evidence in the pack mentions webhooks or event subscription mechanisms; Devin's API/MCP docs describe session creation and management but nothing about outbound webhook notifications for events.

   ⚑ flag

   Conductor—none0/10

   The evidence pack documents a programmatic API and an MCP server for managing cloud workspaces, but nowhere mentions webhooks or any event-subscription mechanism for AI-native users to receive push notifications on workspace/task events.

   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically.”
   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically. Use it to do things like: Create workspaces, send prompts to the coding…”
   • [claimed-docs] “Conductor's hosted Model Context Protocol (MCP) server lets ChatGPT, Claude, Codex, and other MCP clients manage your cloud workspaces.”

   proof ↗⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round to Devin
   Devin✓fullclaimed7/10

   Devin generates insights/suggestions from a user's own codebase data via 'Ask Devin' (code structure/dependency Q&A), auto-generated DeepWiki documentation, and Devin Review's automated PR feedback, all built on repository indexing. Missing for 10: independent/hands-on validation of the quality of these AI-generated insights (community evidence covers general task execution issues, not this specific feature) and no benchmark of insight accuracy.

   • [claimed-docs] “Ask Devin can answer questions about code structure and dependencies, and help you scope and plan tasks before implementation.”
   • [claimed-docs] “Devin Review provides automated first-pass reviews on pull requests, checking for correctness and conformance with organizational best pract…”
   • [claimed-docs] “Use DeepWiki to navigate architecture and code with auto-generated documentation.”
   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”

   proof ↗⚑ flag

   Conductor~partialclaimed4/10

   Conductor orchestrates third-party coding agents (Claude Code, Codex, Cursor) that analyze the codebase and produce diffs, suggested changes, and PR reviews, which can be seen as data-driven suggestions, but Conductor itself does not document any native analytics/insights engine — the 'insight' generation is delegated entirely to the underlying agents. Missing for 10: no first-party insight/analytics feature, no evidence of Conductor synthesizing patterns or trends from user data beyond agent chat/diff output, no independent corroboration of this specific capability.

   • [claimed-docs] “When the work is ready, Conductor helps you review the diff, open a pull request, merge, and archive the workspace.”
   • [claimed-docs] “The agent can test your repositories, update install and setup scripts, and run builds — without asking you to confirm each step.”
   • [claimed-docs] “You can also have an agent configure the computer for you: it can test your repositories, edit setup scripts, and run builds.”
   • [claimed-docs] “Checkpoints | Session/workspace | Revert code and chat state to an earlier turn”

   proof ↗⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Devin
   Devin✓fullclaimed8/10

   Devin explicitly supports background/autonomous execution: cloud sessions run in their own VM and 'keep going after you close your laptop' (devin-docs-13), MCP access includes 'scheduling' (devin-docs-6/23), the API lets you 'automate workflows' and create sessions programmatically (devin-docs-7/8), CI/CD integration triggers Devin on findings (devin-docs-19), and Auto-Fix creates a closed loop that iterates PRs 'without you in the loop' (devin-docs-18). This spans scheduled triggers, API-driven automation, and hands-off background operation. Missing for 10: independent/community verification that scheduled automations run reliably unattended, and more detail on a dedicated 'automation/schedule' UI beyond scattered doc mentions.

   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”

   proof ↗⚑ flag

   Conductor✓fullclaimed7/10

   Conductor's "routines" feature explicitly lets users run agents on a schedule or via GitHub Action, and cloud workspaces continue running autonomously ("agents keep working after you close your laptop") without requiring step-by-step confirmation. This directly matches background, autonomous automation for an AI-native user. Missing for 10: independent/hands-on confirmation that routines work reliably in practice, and more detail on scheduling configuration options beyond the changelog mention.

   • [claimed-docs] “Introducing routines! You can now run your agents on a schedule or via GitHub action.”
   • [claimed-docs] “Sandboxes spin up in seconds, and agents keep working after you close your laptop.”
   • [claimed-docs] “The agent can test your repositories, update install and setup scripts, and run builds — without asking you to confirm each step.”
   • [claimed-docs] “You can also have an agent configure the computer for you: it can test your repositories, edit setup scripts, and run builds.”

   proof ↗⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round to Conductor
   Devin~partialcommunity6/10

   Devin's core product is designed for task delegation — via Ask Devin, ticket assignment, Slack/Teams tagging, and a conversational IDE (devin-docs-1, devin-docs-3, devin-docs-24) — so the axis clearly applies and is well documented. However, hands-on community reports show real caveats: Devin can add extraneous unrequested changes it can't undo, gets stuck for long periods without asking for help, and requires active babysitting/session termination to get value (devin-comm-1, devin-comm-2, devin-comm-5), undercutting a fully seamless delegation experience. Missing for 10: independent verification that delegated tasks reliably complete without extraneous side-effects or getting stuck, and stronger corroboration beyond one HN thread.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Devin is designed to be a conversational user interface, and allows you to follow and take over Devin's development process in the embedded …”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

   Conductor✓fullcommunity8/10

   Conductor lets users delegate coding tasks to agents (Claude Code, Codex, Cursor, OpenCode) that run inside its own workspaces, autonomously testing repos, running builds, and continuing work unattended, with checkpoints and review flow built into the product (conductor-docs-1, -17, -20, -29, -32). Community reports confirm the agent runs live inside the app during real use (conductor-comm-7, conductor-comm-15). Missing for 10: independent benchmarking of assistant quality/reliability beyond docs and mixed anecdotal UX feedback (conductor-comm-9).

   • [claimed-docs] “Conductor lets you run Claude Code, Codex, Cursor, and OpenCode in parallel.”
   • [claimed-docs] “The agent can test your repositories, update install and setup scripts, and run builds — without asking you to confirm each step.”
   • [claimed-docs] “Sandboxes spin up in seconds, and agents keep working after you close your laptop.”
   • [claimed-docs] “Conductor creates a separate working tree for each workspace. That lets agents work in parallel without editing the same files on disk.”
   • [claimed-docs] “You can also have an agent configure the computer for you: it can test your repositories, edit setup scripts, and run builds.”
   • [community] “I have been imagining something like this would be perfect for working with Claude Code... This feels like just a nice clean simple extensio…”
   • [community] “Your messages are just between you and Claude Code — it all runs on your local Claude Code installation via the SDK.”

   proof ↗⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round drawn
   Devin✓fullcommunity8/10

   Devin is explicitly designed as a conversational agent: users assign tasks via natural language (Slack/Teams tagging, chat interface, CLI prompts like 'devin -- check out this code...'), and it interprets these into autonomous coding/dev actions across IDE, CLI, and cloud sessions. Community evidence corroborates it operates on natural-language task descriptions in practice, though with noted friction around scope creep and knowing when to stop. Missing for 10: independent benchmarking of NL command accuracy/robustness and richer detail on how ambiguous instructions are resolved.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Tagging Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “Devin is designed to be a conversational user interface, and allows you to follow and take over Devin's development process in the embedded …”
   • [claimed-docs] “devin -- check out this code and suggest a feasible, helpful feature”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

   Conductor✓fullprobed8/10

   Conductor's entire interaction model is natural-language chat with coding agents (Claude Code, Codex, Cursor, OpenCode) that can autonomously test, build, and edit without step confirmation, and it exposes a hosted MCP server so ChatGPT/Claude/Codex or other AI clients can manage workspaces via natural language, plus an API to send prompts and read agent replies. missing for 10: independent/hands-on validation of natural-language command reliability beyond vendor docs.

   • [claimed-docs] “Conductor's hosted Model Context Protocol (MCP) server lets ChatGPT, Claude, Codex, and other MCP clients manage your cloud workspaces.”
   • [claimed-docs] “The agent can test your repositories, update install and setup scripts, and run builds — without asking you to confirm each step.”
   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically. Use it to do things like: Create workspaces, send prompts to the coding…”
   • [claimed-docs] “You can also have an agent configure the computer for you: it can test your repositories, edit setup scripts, and run builds.”
   • [probe] “official MCP server documented at https://www.conductor.build/docs/api/mcp”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   Devin—none0/10

   Devin has an API reference overview but the openapi.json probe returned 404 on all candidate paths, and there's no mention of an interactive reference with runnable examples (e.g., 'try it' console) in the docs pack.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

   Conductor—none0/10

   Conductor has documented API endpoints and an MCP server, so an interactive API reference with runnable examples is a plausible feature, but the evidence pack shows no such reference exists — the docs page is static markdown and probes for OpenAPI/Swagger specs all returned 404.

   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically.”
   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically. Use it to do things like: Create workspaces, send prompts to the coding…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://www.conductor.build/openapi.json, https://www.conductor.build/swagger.json, https://www.cond…”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   Devin—none0/10

   Devin has a documented API (devin-docs-7) but a direct probe for a machine-readable OpenAPI/swagger spec returned 404 on all candidate paths, and no documentation item references a downloadable spec file.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”

   proof ↗⚑ flag

   Conductor—none0/10

   Conductor documents a REST-like API and an MCP server, but a direct probe for machine-readable OpenAPI/Swagger specs at standard locations returned 404 on all candidate paths, and no evidence pack item links to a downloadable spec file.

   • [probe] “PROBE openapi: all candidate paths 404 (https://www.conductor.build/openapi.json, https://www.conductor.build/swagger.json, https://www.cond…”
   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically.”
   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically. Use it to do things like: Create workspaces, send prompts to the coding…”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round drawn
   Devin~partialclaimed6/10

   Devin sessions run in isolated cloud VMs with their own shell/browser/full repo access, configurable environment blueprints for a 'known-good state,' and a CLI --sandbox flag enforcing OS-level write/network isolation, all of which support testing in isolated environments away from live infrastructure. However, there is no explicit documentation addressing production-data isolation or masking, or confirmation that these sandboxes are guaranteed free of production data. missing for 10: explicit statement on production-data separation/masking, independent hands-on confirmation that sandbox testing never touches production data.

   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “Configure it once, and every session boots into that known-good state.”
   • [claimed-docs] “The --sandbox flag runs the CLI with OS-level isolation, enforcing writable paths and deny rules at the operating-system level and optionall…”
   • [claimed-docs] “The `--sandbox` flag runs the CLI with OS-level isolation, enforcing writable paths and `deny` rules at the operating-system level”
   • [claimed-docs] “Outposts lets you run Devin sessions inside infrastructure you control — your own VMs, containers, Kubernetes clusters, or even a Mac Mini o…”
   • [claimed-docs] “Devin inspects your repository, figures out which tools, runtimes, and dependencies are needed, and generates the blueprint for you.”

   proof ↗⚑ flag

   Conductor~partialcommunity6/10

   Conductor's core architecture creates isolated workspaces (separate git worktrees, branches, cloud sandboxes) so each agent task runs independently without touching the main/production branch (conductor-docs-2, conductor-docs-20, conductor-docs-27, conductor-docs-29), and community users confirm the git-worktree-based isolation (conductor-comm-1, conductor-comm-17). However, this isolation is code/branch-level, not explicitly a data-layer sandbox (e.g., staging DB, mock services), and one community report notes full GitHub write-access requirements that undercut a clean 'no touching production' guarantee (conductor-comm-5, conductor-comm-6). Missing for 10: explicit handling/isolation of production data stores or environment variables, and confirmation that sandbox workspaces cannot inadvertently write to production systems.

   • [claimed-docs] “Each task gets its own workspace, branch, files, terminal, diff, and review path.”
   • [claimed-docs] “Sandboxes spin up in seconds, and agents keep working after you close your laptop.”
   • [claimed-docs] “Create a new workspace with Command + N when work should have its own branch, files, running environment, and review path.”
   • [claimed-docs] “Conductor creates a separate working tree for each workspace. That lets agents work in parallel without editing the same files on disk.”
   • [community] “Oh cool, I was already doing this with git worktrees but a ui for it would be handy.”
   • [community] “We create an isolated git worktree locally on your machine — whereas Codex (I believe) is running a container on the cloud.”
   • [community] “Full read-write access required to all your Github account's repos. Not just code. Settings, deploy keys. The works... Zero disclosure of da…”
   • [community] “I was really excited to try this but this does NOT work the way I expected. I wanted a simple git worktree manager for my existing, already-…”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Devin—none0/10

   There's an API reference (devin-docs-7) and even an OpenAPI probe, but that probe found no OpenAPI spec (devin-probe-3), and no evidence anywhere mentions API versioning scheme or a documented deprecation policy for the API.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

   Conductor—none0/10

   There's an API and MCP server documented, but no evidence of API versioning scheme or a deprecation policy; probes show no OpenAPI spec found and no changelog/policy on version deprecation.

   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically.”
   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically. Use it to do things like: Create workspaces, send prompts to the coding…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://www.conductor.build/openapi.json, https://www.conductor.build/swagger.json, https://www.cond…”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Conductor
   Devin~partialcommunity5/10

   Devin's API supports programmatic session creation (including on behalf of other users) and docs explicitly encourage carving out independent tasks to run simultaneously, which together enable bulk-style automation across many tickets/items, but there is no dedicated 'bulk operations' or batch-processing feature documented, and community feedback raises concerns about reliability/oversight needed per session that would complicate true bulk workflows. Missing for 10: an explicit batch/bulk API endpoint or UI for processing many items in one request, and independent evidence of successful large-scale bulk runs.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”
   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”

   proof ↗⚑ flag

   Conductor~partialclaimed7/10

   Conductor supports running many coding agents in parallel across isolated workspaces, and exposes a programmatic API plus scheduled/CI-triggered 'routines' that can create workspaces and send prompts at scale — a reasonable basis for bulk, automation-driven operations across many items. However, there's no documented UI for batch-selecting and acting on many existing workspaces at once (e.g., bulk archive/merge), and no independent evidence of large-scale parallel runs in practice. Missing for 10: explicit multi-item batch actions in the UI, evidence of scale/limits on parallel agents, and third-party corroboration of bulk automation workflows via the API or routines.

   • [claimed-docs] “Conductor lets you run Claude Code, Codex, Cursor, and OpenCode in parallel.”
   • [claimed-docs] “Create a new workspace with Command + N when work should have its own branch, files, running environment, and review path.”
   • [claimed-docs] “Run multiple agents in one workspace when the work belongs on the same branch and should share the same files and context.”
   • [claimed-docs] “Conductor creates a separate working tree for each workspace. That lets agents work in parallel without editing the same files on disk.”
   • [claimed-docs] “Introducing routines! You can now run your agents on a schedule or via GitHub action.”
   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically.”
   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically. Use it to do things like: Create workspaces, send prompts to the coding…”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to Devin
   Devin~partialclaimed6/10

   Devin supports several built-in event-triggered automations — Auto-Fix responds automatically to PR review comments and CI failures, CI/CD integration triggers Devin off static-analysis findings (SonarQube/Fortify/Veracode), and MCP exposes 'scheduling' as a capability — but there's no evidence of a general-purpose, user-defined rules/webhook engine for arbitrary custom triggers. Missing for 10: documentation of a configurable custom-rule/webhook trigger system, details on the scheduling feature's flexibility, and independent confirmation that these automations work reliably as event triggers.

   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixe”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”

   proof ↗⚑ flag

   Conductor~partialclaimed5/10

   Conductor's 'routines' feature lets agents run on a schedule or via GitHub Action trigger, which is a limited form of event-driven automation, but there's no evidence of a general rules engine supporting arbitrary event types (e.g., webhooks, file changes, custom conditions) or complex trigger-action definitions. Missing for 10: broader event-type support, custom rule/condition definitions, and hands-on evidence that routines fire reliably on GitHub events.

   • [claimed-docs] “Introducing routines! You can now run your agents on a schedule or via GitHub action.”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round to Conductor
   Devin~partialclaimed3/10

   Devin's MCP docs mention 'scheduling' as one of the capabilities exposed to MCP-compatible agents, implying some scheduling functionality exists, but there is no dedicated documentation, UI, or examples describing recurring jobs, cron-like triggers, or workflow automation configuration. missing for 10: dedicated scheduling feature docs, examples of recurring/cron jobs, independent confirmation of scheduled workflows in practice.

   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”

   proof ↗⚑ flag

   Conductor✓fullclaimed6/10

   Conductor's changelog explicitly introduces 'routines' that let agents run on a schedule or via GitHub Action, directly matching the recurring-jobs/workflows story. However, this is a single brief changelog mention with no dedicated documentation page, configuration details, or community corroboration of the feature in practice. Missing for 10: dedicated docs on routine/schedule configuration, independent/hands-on confirmation, details on failure handling or monitoring of scheduled runs.

   • [claimed-docs] “Introducing routines! You can now run your agents on a schedule or via GitHub action.”

   proof ↗⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round to Conductor
   Devin—none0/10

   The evidence shows Devin has knowledge, playbooks, and scheduling features but nothing about versioning, reviewing, or rolling back those automation configurations themselves; Devin Review/Auto-Fix pertains to PR code review, not to the automation definitions. Missing for 10: version history for playbooks/knowledge, a review workflow for automation changes, and a rollback mechanism for automations.

   ⚑ flag

   Conductor~partialclaimed6/10

   Conductor provides git-based versioning (separate branches/worktrees per workspace), diff review before merge/PR, and 'Checkpoints' to revert code and chat state to an earlier turn—covering version, review, and rollback at the workspace/agent-session level. However, the newer 'Routines' (scheduled/GitHub-Action automations) feature has no documented versioning, review, or rollback mechanism specific to the automation definitions themselves. Missing for 10: explicit version history/rollback for Routines/scheduled automations, independent hands-on confirmation of checkpoint reliability.

   • [claimed-docs] “Checkpoints | Session/workspace | Revert code and chat state to an earlier turn”
   • [claimed-docs] “When the work is ready, Conductor helps you review the diff, open a pull request, merge, and archive the workspace.”
   • [claimed-docs] “Conductor creates a separate working tree for each workspace. That lets agents work in parallel without editing the same files on disk.”
   • [claimed-docs] “Introducing routines! You can now run your agents on a schedule or via GitHub action.”
   • [claimed-docs] “Create a new workspace with Command + N when work should have its own branch, files, running environment, and review path.”

   proof ↗⚑ flag

Background execution

1. ai-native userHave a cloud agent build, test, and demo a feature end-to-end for my review

   weight 2 · round to Conductor
   Devin~partialcommunity7/10

   Devin's cloud sessions run in dedicated VMs with shell/browser/full repo access, can implement features, run tests, and continue autonomously after handoff, then present PRs for review (docs-1,13,17). However, community hands-on reports describe unreliable autonomy — extraneous breaking changes, inability to self-correct, and needing frequent human monitoring/termination — undercutting the 'build, test, demo end-to-end' promise. missing for 10: reliable independent verification of unattended end-to-end demo quality, and clearer evidence of a built-in demo/walkthrough artifact for reviewers beyond PR creation.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “Devin Review provides automated first-pass reviews on pull requests, checking for correctness and conformance with organizational best pract…”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

   Conductor✓fullclaimed7/10

   Conductor's cloud agents can autonomously test repos, update setup scripts, and run builds without step-by-step confirmation (conductor-docs-17, conductor-docs-32), continue working after the laptop closes (conductor-docs-20), and then help the user review the diff, open a PR, and merge (conductor-docs-21) — covering build, test, and review end-to-end for a feature. Missing for 10: no explicit 'demo' artifact (e.g., preview links/screenshots) beyond diff/PR review, and no independent/hands-on account confirming a full autonomous build-test-review cycle worked as described.

   • [claimed-docs] “The agent can test your repositories, update install and setup scripts, and run builds — without asking you to confirm each step.”
   • [claimed-docs] “Sandboxes spin up in seconds, and agents keep working after you close your laptop.”
   • [claimed-docs] “When the work is ready, Conductor helps you review the diff, open a pull request, merge, and archive the workspace.”
   • [claimed-docs] “You can also have an agent configure the computer for you: it can test your repositories, edit setup scripts, and run builds.”

   proof ↗⚑ flag

2. developerDelegate longer-running coding tasks to run in the background in an isolated cloud environment

   weight 3 · round to Devin
   Devin✓fullcommunity9/10

   Devin runs tasks in isolated cloud VMs with full shell/browser/repo access that persist after the user disconnects, explicitly supporting long-running background work and parallel independent tasks, corroborated by community reports of multi-day autonomous runs. Missing for 10: independent third-party benchmarking of long-running task reliability/quality beyond anecdotal HN reports.

   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”
   • [claimed-docs] “Configure it once, and every session boots into that known-good state.”
   • [claimed-docs] “Hand a task off to a cloud Devin session and keep working locally.”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “You can set a 'max work time' before Devin pauses so it won't go for days endlessly spending your credits. By default it's set to 10 credits…”

   proof ↗⚑ flag

   Conductor~partialcommunity7/10

   Docs describe a dedicated 'cloud workspace' feature where agents run in isolated sandboxes that 'spin up in seconds' and 'keep working after you close your laptop,' can test repos/run builds unattended, and continue processing PR checks while 'asleep' (conductor-docs-20, conductor-docs-17, conductor-docs-11, conductor-docs-13). However, community reports describe the core product as creating an isolated git worktree locally rather than a cloud container, contrasting it with Codex's cloud sandbox (conductor-comm-17, conductor-comm-6), suggesting the cloud-isolation capability may be a newer/optional layer rather than the default experience. Missing for 10: independent hands-on verification that background cloud tasks are fully isolated/persistent, and clarity on whether cloud workspaces are the default vs. opt-in given local-worktree-first community accounts.

   • [claimed-docs] “Sandboxes spin up in seconds, and agents keep working after you close your laptop.”
   • [claimed-docs] “The agent can test your repositories, update install and setup scripts, and run builds — without asking you to confirm each step.”
   • [claimed-docs] “PR comments and failing-check logs now load while a cloud workspace is asleep.”
   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically.”
   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically. Use it to do things like: Create workspaces, send prompts to the coding…”
   • [community] “We create an isolated git worktree locally on your machine — whereas Codex (I believe) is running a container on the cloud.”
   • [community] “I was really excited to try this but this does NOT work the way I expected. I wanted a simple git worktree manager for my existing, already-…”

   proof ↗⚑ flag

3. developerConfigure a reproducible cloud environment with the dependencies and setup steps my repository needs

   weight 2 · round to Devin
   Devin✓fullclaimed9/10

   Devin explicitly supports configuring environment 'blueprints' that specify tools, runtimes, and dependencies so 'every session boots into that known-good state,' with auto-detection of requirements from the repo (docs-31, docs-32), plus indexing (docs-27), knowledge/AGENTS.md context files (docs-29, docs-30), and VPN access for internal dependencies (docs-28), all running in isolated cloud VMs (docs-13). Missing for 10: independent/hands-on confirmation that blueprint-based environments reliably reproduce across sessions in practice.

   • [claimed-docs] “Configure it once, and every session boots into that known-good state.”
   • [claimed-docs] “Devin inspects your repository, figures out which tools, runtimes, and dependencies are needed, and generates the blueprint for you.”
   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”
   • [claimed-docs] “Knowledge is a collection of instructions and advice that Devin can reference in all sessions.”
   • [claimed-docs] “Devin supports AGENTS.md - a simple, open standard for providing context and instructions to AI agents.”
   • [claimed-docs] “Devin can connect to a VPN from inside its workspace, so sessions can reach internal services such as package registries, databases, and int…”
   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”

   proof ↗⚑ flag

   Conductor~partialclaimed6/10

   Docs show Conductor's cloud workspaces spin up sandboxes, check for needed tools/credentials, and let agents edit install/setup scripts and run builds automatically, which supports configuring an environment with the right dependencies (conductor-docs-17, conductor-docs-20, conductor-docs-32, conductor-docs-33). However there's no explicit first-party description of a declarative, versioned environment-config file (e.g., a devcontainer-style spec) guaranteeing reproducibility across runs/teammates, and no independent confirmation that these setup scripts persist reliably across sessions. missing for 10: explicit reproducible-config artifact/spec, independent verification that environment setup is consistent across workspace recreations.

   • [claimed-docs] “The agent can test your repositories, update install and setup scripts, and run builds — without asking you to confirm each step.”
   • [claimed-docs] “Sandboxes spin up in seconds, and agents keep working after you close your laptop.”
   • [claimed-docs] “You can also have an agent configure the computer for you: it can test your repositories, edit setup scripts, and run builds.”
   • [claimed-docs] “When you open Conductor, it checks for the tools and credentials it needs. If anything is missing, Conductor walks you through setup.”

   proof ↗⚑ flag

Parallel agents

1. ai-native userLaunch fleets of autonomous agents that work in parallel on different tasks for hours or days

   weight 2 · round to Conductor
   Devin~partialcommunity6/10

   Docs confirm parallel task execution ('Carve out independent tasks and run them simultaneously'), cloud sessions that persist after closing the laptop, and an API to spin up multiple sessions programmatically (including on behalf of other users), which together support a 'fleet of parallel long-running agents' story. However, community hands-on reports show real friction with the 'hours/days autonomous' claim: sessions can get stuck without an escape hatch, users must babysit and manually terminate sessions every 10-15 minutes, and there's a default max-work-time cap limiting unsupervised runtime. Missing for 10: independent verification of successful multi-day/multi-task fleets running unattended, and evidence addressing the reported lack of a reliable 'ask for help' escalation during long runs.

   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”
   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “Devin does ask for help when it can't do something, but it really hates asking for help if it's a skill issue - it would prefer running in c…”
   • [community] “You can set a 'max work time' before Devin pauses so it won't go for days endlessly spending your credits. By default it's set to 10 credits…”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

   Conductor✓fullclaimed7/10

   Docs show Conductor explicitly designed for running multiple agents (Claude Code, Codex, Cursor, OpenCode) in parallel across isolated workspaces/worktrees, with cloud workspaces that 'keep working after you close your laptop' and 'routines' to run agents on a schedule or via GitHub Action, supporting long-running autonomous fleets. Community feedback focuses on GitHub permission/privacy concerns rather than disputing the parallel-autonomy capability itself. Missing for 10: independent/hands-on confirmation of agents actually running unattended for multi-day spans and evidence of fleet scale (e.g., dozens of simultaneous agents).

   • [claimed-docs] “Conductor lets you run Claude Code, Codex, Cursor, and OpenCode in parallel.”
   • [claimed-docs] “Each task gets its own workspace, branch, files, terminal, diff, and review path.”
   • [claimed-docs] “Introducing routines! You can now run your agents on a schedule or via GitHub action.”
   • [claimed-docs] “Sandboxes spin up in seconds, and agents keep working after you close your laptop.”
   • [claimed-docs] “Create a new workspace with Command + N when work should have its own branch, files, running environment, and review path.”
   • [claimed-docs] “Run multiple agents in one workspace when the work belongs on the same branch and should share the same files and context.”
   • [claimed-docs] “Conductor creates a separate working tree for each workspace. That lets agents work in parallel without editing the same files on disk.”
   • [claimed-docs] “You can also have an agent configure the computer for you: it can test your repositories, edit setup scripts, and run builds.”

   proof ↗⚑ flag

2. developerRun several task attempts in parallel and compare results before choosing one

   weight 1 · round to Conductor
   Devin—none0/10

   Devin's docs mention running multiple independent tasks simultaneously (devin-docs-25) but this describes parallelizing different tasks, not running several parallel attempts of the SAME task to compare and choose the best result. No evidence describes a compare/choose-best-attempt workflow.

   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”

   proof ↗⚑ flag

   Conductor✓fullprobed8/10

   Conductor's core design is running multiple coding agents in parallel, each in its own isolated workspace/git worktree with its own branch, files, and diff/review path, letting a developer inspect and choose before merging (docs-2, docs-27, docs-29, docs-21, probe-1). Community hands-on comments corroborate the git-worktree-based parallel workspace model (conductor-comm-1, conductor-comm-17). missing for 10: explicit first-party description of a side-by-side comparison UI across multiple simultaneous attempts (evidence shows parallel isolated workspaces and per-workspace diff/review, but not an explicit 'compare attempts' feature or independent review confirming the comparison workflow).

   • [claimed-docs] “Each task gets its own workspace, branch, files, terminal, diff, and review path.”
   • [claimed-docs] “When the work is ready, Conductor helps you review the diff, open a pull request, merge, and archive the workspace.”
   • [claimed-docs] “Create a new workspace with Command + N when work should have its own branch, files, running environment, and review path.”
   • [claimed-docs] “Conductor creates a separate working tree for each workspace. That lets agents work in parallel without editing the same files on disk.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://www.conductor.build/llms.txt # Conductor > Conductor is a Mac app that lets you run many coding agents …”
   • [community] “Oh cool, I was already doing this with git worktrees but a ui for it would be handy.”
   • [community] “We create an isolated git worktree locally on your machine — whereas Codex (I believe) is running a container on the cloud.”

   proof ↗⚑ flag

Scheduled automation

1. ai-native userSet up always-on agents that run on schedules or triggers to maintain and fix my software autonomously

   weight 2 · round drawn
   Devin~partialcommunity6/10

   Devin supports trigger-based autonomous work (Slack/Teams tags, PR review comments, CI/CD/static-analysis findings) and Auto-Fix creates a closed loop that iterates on CI failures without a human in the loop, and MCP exposes 'scheduling' as a session capability, all suggesting some always-on/triggered agent operation. However there's no dedicated docs for cron-like recurring schedules, and community reports describe sessions needing frequent human monitoring/termination rather than fully unattended long-running maintenance. Missing for 10: explicit scheduling/cron configuration docs, independent evidence of reliable unattended multi-day maintenance loops, and confirmation that Auto-Fix/CI triggers work without human oversight in practice.

   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “You can set a 'max work time' before Devin pauses so it won't go for days endlessly spending your credits. By default it's set to 10 credits…”

   proof ↗⚑ flag

   Conductor~partialclaimed6/10

   Conductor documents 'routines' that run agents on a schedule or via GitHub Action, plus cloud agents that keep working after you close your laptop and can autonomously test, fix, and rebuild repos without step-by-step confirmation — directly supporting always-on autonomous maintenance. However, the routines feature is only briefly mentioned in a changelog entry with no deep documentation of trigger types, monitoring, or failure-handling, and no independent/hands-on evidence confirms long-running unattended reliability. Missing for 10: detailed docs on trigger configuration (webhooks, cron specifics), evidence of long-term unattended reliability, and community confirmation of the scheduling/autonomy feature working in practice.

   • [claimed-docs] “Introducing routines! You can now run your agents on a schedule or via GitHub action.”
   • [claimed-docs] “The agent can test your repositories, update install and setup scripts, and run builds — without asking you to confirm each step.”
   • [claimed-docs] “Sandboxes spin up in seconds, and agents keep working after you close your laptop.”
   • [claimed-docs] “You can also have an agent configure the computer for you: it can test your repositories, edit setup scripts, and run builds.”

   proof ↗⚑ flag

Debugging

1. developerDebug issues and troubleshoot using natural-language queries

   weight 2 · round to Devin
   Devin~partialcommunity7/10

   Docs clearly support NL-driven debugging: 'repro and fix bugs', 'Ask Devin can answer questions about code structure...help you scope and plan tasks', and tagging Devin in Slack/Teams about a bug thread. However, hands-on community reports describe practical caveats—Devin adding extraneous changes it can't undo, getting stuck without escalating, requiring manual babysitting—that temper reliability for troubleshooting workflows. Missing for 10: independent benchmark/case study specifically on debugging accuracy, and resolution of the 'getting stuck on bugs' community complaint.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Tagging Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “Ask Devin can answer questions about code structure and dependencies, and help you scope and plan tasks before implementation.”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “Devin does ask for help when it can't do something, but it really hates asking for help if it's a skill issue - it would prefer running in c…”

   proof ↗⚑ flag

   Conductor~partialclaimed5/10

   Conductor orchestrates coding agents (Claude Code, Codex, Cursor) that support natural-language chat, and each workspace has its own terminal, diff, and chat interface, implying a developer could ask an agent to debug/troubleshoot via NL queries. However, there's no Conductor-specific documentation describing a dedicated debugging/troubleshooting NL workflow, error-log analysis, or diagnostic features beyond generic agent chat and build/test execution. Missing for 10: explicit docs on NL-driven debugging workflows, log/error analysis features, or examples of troubleshooting via chat distinct from general coding tasks.

   • [claimed-docs] “Each task gets its own workspace, branch, files, terminal, diff, and review path.”
   • [claimed-docs] “The agent can test your repositories, update install and setup scripts, and run builds — without asking you to confirm each step.”
   • [claimed-docs] “You can also have an agent configure the computer for you: it can test your repositories, edit setup scripts, and run builds.”
   • [claimed-docs] “Checkpoints | Session/workspace | Revert code and chat state to an earlier turn”

   proof ↗⚑ flag

Feature implementation

1. developerTurn a tracked issue into a complete pull request end-to-end

   weight 3 · round to Conductor
   Devin~partialcommunity6/10

   Devin's docs explicitly describe taking Linear/Jira tickets and implementing full features, with Devin Review/Auto-Fix looping PRs toward merge-ready status without human involvement, covering the issue-to-PR pipeline end-to-end. However, hands-on community reports describe practical friction — extraneous unrelated changes that can break things, inability to easily undo them, and agents getting stuck for days rather than asking for help — casting doubt on how cleanly the 'complete' PR is delivered without oversight. Missing for 10: independent verification of a clean ticket→merged-PR flow without manual intervention, and resolution of the reported reliability/quality issues.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Devin Review provides automated first-pass reviews on pull requests, checking for correctness and conformance with organizational best pract…”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Code migrations, refactors, and modernization”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “Devin does ask for help when it can't do something, but it really hates asking for help if it's a skill issue - it would prefer running in c…”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

   Conductor✓fullclaimed7/10

   Docs show workspaces can be created directly from a GitHub issue (conductor-docs-12), agents run autonomously to implement, test, and build (conductor-docs-17, conductor-docs-20), and Conductor then helps review the diff, open a PR, merge, and archive the workspace (conductor-docs-21) — covering the full issue-to-PR loop. Missing for 10: independent/hands-on confirmation of the complete issue→PR flow (community evidence covers worktree/permissions concerns but not this specific workflow), and no example of a merged PR originating from an issue.

   • [claimed-docs] “Use Command + Shift + N or the `...` button next to `New workspace` to create a workspace from a branch, pull request, GitHub issue, or Line…”
   • [claimed-docs] “The agent can test your repositories, update install and setup scripts, and run builds — without asking you to confirm each step.”
   • [claimed-docs] “Sandboxes spin up in seconds, and agents keep working after you close your laptop.”
   • [claimed-docs] “When the work is ready, Conductor helps you review the diff, open a pull request, merge, and archive the workspace.”

   proof ↗⚑ flag

2. developerDescribe a feature or bug in plain language and have the agent implement or fix it across multiple files

   weight 3 · round to Conductor
   Devin!disputedcontradicted5/10

   Devin's docs strongly claim end-to-end feature/bug implementation across a full repo (Linear/Jira tickets, multi-file fixes, code migrations) with full workspace/VM access [devin-docs-1, devin-docs-13, devin-docs-21, devin-docs-27], but hands-on community reports concretely contradict smooth delivery: it 'always adds extraneous changes beyond the task that can break other things, and can't undo those changes if asked' and required constant supervision/termination to get real work done [devin-comm-1, devin-comm-5], with another user noting it can run for days without an escape hatch when stuck [devin-comm-2]. Missing for 10: independent benchmark data on multi-file correctness, and resolution of the extraneous-change/undo failure mode reported by users.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “Code migrations, refactors, and modernization”
   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

   Conductor~partialcommunity6/10

   Conductor orchestrates underlying coding agents (Claude Code, Codex, Cursor, OpenCode) that implement plain-language feature requests across files, with workspaces, diffs, and PR flows supporting this, and community feedback confirms it works as a Claude Code-like workflow wrapper. However, the actual code-generation quality depends entirely on the underlying agent, not Conductor itself, and no hands-on example of a multi-file feature/bug fix is shown in the evidence. missing for 10: a concrete hands-on example of Conductor implementing a described feature/bug across multiple files, and clarity on Conductor's own contribution versus the wrapped agent's capability.

   • [claimed-docs] “Conductor lets you run Claude Code, Codex, Cursor, and OpenCode in parallel.”
   • [claimed-docs] “Each task gets its own workspace, branch, files, terminal, diff, and review path.”
   • [claimed-docs] “The agent can test your repositories, update install and setup scripts, and run builds — without asking you to confirm each step.”
   • [claimed-docs] “When the work is ready, Conductor helps you review the diff, open a pull request, merge, and archive the workspace.”
   • [claimed-docs] “Conductor creates a separate working tree for each workspace. That lets agents work in parallel without editing the same files on disk.”
   • [community] “I have been imagining something like this would be perfect for working with Claude Code... This feels like just a nice clean simple extensio…”
   • [community] “We create an isolated git worktree locally on your machine — whereas Codex (I believe) is running a container on the cloud.”

   proof ↗⚑ flag

Maintenance automation

1. developerHave the agent write tests, fix lint errors, resolve merge conflicts, and update dependencies for me

   weight 3 · round to Devin
   Devin~partialcommunity6/10

   Docs show Devin is a general-purpose coding agent that can fix bugs, implement features, iterate on CI failures, respond to review comments (Auto-Fix), and handle code migrations/refactors/modernization, which plausibly covers lint fixes and CI-related work, but none of the docs explicitly mention writing tests, resolving merge conflicts, or updating dependencies as named capabilities. Community reports (devin-comm-1) also note Devin can introduce extraneous changes and struggles to cleanly undo them, tempering confidence in reliably delivering these specific maintenance tasks. Missing for 10: explicit documentation/evidence of test-writing, lint-fixing, merge-conflict resolution, and dependency-update workflows, plus independent hands-on confirmation of these specific tasks succeeding.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Code migrations, refactors, and modernization”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”

   proof ↗⚑ flag

   Conductor~partialclaimed5/10

   Docs confirm the underlying agents can test repositories, edit setup/install scripts, and run builds autonomously (conductor-docs-17, conductor-docs-32), which covers test-writing/fixing to some degree, but there is no explicit documentation or community evidence of lint-error fixing, merge-conflict resolution, or dependency updates as distinct capabilities. Missing for 10: explicit evidence of lint fixing, merge conflict resolution, and dependency-update automation.

   • [claimed-docs] “The agent can test your repositories, update install and setup scripts, and run builds — without asking you to confirm each step.”
   • [claimed-docs] “You can also have an agent configure the computer for you: it can test your repositories, edit setup scripts, and run builds.”

   proof ↗⚑ flag

Codebase mapping

1. developerUnderstand how a codebase fits together to find where to start making changes

   weight 3 · round to Devin
   Devin✓fullclaimed7/10

   Devin explicitly indexes repositories to power 'Ask Devin' and 'DeepWiki', which answer questions about code structure and dependencies and help developers scope/plan where to start making changes, directly matching the story. missing for 10: independent/hands-on evidence validating DeepWiki/Ask Devin's accuracy on real codebases (community evidence only covers task execution, not codebase-understanding features).

   • [claimed-docs] “Use DeepWiki to navigate architecture and code with auto-generated documentation.”
   • [claimed-docs] “Ask Devin can answer questions about code structure and dependencies, and help you scope and plan tasks before implementation.”
   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”

   proof ↗⚑ flag

   Conductor—none0/10

   Conductor's evidence focuses on orchestrating parallel coding agents, worktrees, and workspace management, not on codebase comprehension features; the only related item is a basic file-content search (⌘⇧F), which does not constitute understanding how a codebase fits together or where to start making changes.

   • [claimed-docs] “Search file contents in your current local project or cloud workspace with ⌘⇧F.”

   proof ↗⚑ flag

2. developerHave the agent map and explain an entire unfamiliar codebase without manually selecting context files

   weight 3 · round to Devin
   Devin✓fullclaimed7/10

   Docs describe repo indexing that lets Devin understand the codebase and power 'Ask Devin' and DeepWiki auto-generated architecture docs, directly enabling exploration/explanation of an unfamiliar codebase without manual file selection (devin-docs-27, devin-docs-15, devin-docs-16). No community evidence contradicts this specific capability. Missing for 10: independent/hands-on verification of codebase-mapping accuracy and no concrete example of DeepWiki output quality.

   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”
   • [claimed-docs] “Use DeepWiki to navigate architecture and code with auto-generated documentation.”
   • [claimed-docs] “Ask Devin can answer questions about code structure and dependencies, and help you scope and plan tasks before implementation.”

   proof ↗⚑ flag

   Conductor—none0/10

   Conductor's evidence focuses on orchestrating parallel agent workspaces, worktrees, git branches, and collaboration—not on any built-in whole-codebase mapping or explanation capability. The closest feature is manual file-content search (⌘⇧F), which requires the developer to search rather than having the agent autonomously map/explain the codebase.

   • [claimed-docs] “Search file contents in your current local project or cloud workspace with ⌘⇧F.”
   • [claimed-docs] “The agent can test your repositories, update install and setup scripts, and run builds — without asking you to confirm each step.”
   • [claimed-docs] “Conductor creates a separate working tree for each workspace. That lets agents work in parallel without editing the same files on disk.”

   proof ↗⚑ flag

Context management

1. developerHave the agent build and recall memory automatically across sessions

   weight 2 · round to Devin
   Devin~partialclaimed5/10

   Devin supports persistent cross-session context via "Knowledge" (instructions referenced in all sessions), AGENTS.md, and environment blueprints that let every session boot into a known-good state, which enables some recall across sessions. However, these mechanisms are largely user-configured/onboarded rather than autonomously built by the agent from its own experience, and there's no evidence of automatic memory creation or recall behavior demonstrated in practice. Missing for 10: evidence the agent automatically extracts/updates memory from its own task experience without manual setup, and independent confirmation that recalled memory improves subsequent session performance.

   • [claimed-docs] “Knowledge is a collection of instructions and advice that Devin can reference in all sessions.”
   • [claimed-docs] “Devin supports AGENTS.md - a simple, open standard for providing context and instructions to AI agents.”
   • [claimed-docs] “Configure it once, and every session boots into that known-good state.”
   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”

   proof ↗⚑ flag

   Conductor—none0/10

   Evidence covers checkpoints (revert to earlier turn), static 'general preferences' for repo-wide instructions, and parallel workspace/session management, but nothing describes the agent automatically building or recalling memory across sessions (e.g., persistent knowledge base, learned context reuse). This is a fair axis for a coding-agent orchestration tool, so absence of evidence yields none.

   • [claimed-docs] “Checkpoints | Session/workspace | Revert code and chat state to an earlier turn”
   • [claimed-docs] “`General preferences` apply broad instructions to agents in a repository.”

   proof ↗⚑ flag

2. developerInclude multiple project directories in a single session for broader context

   weight 2 · round drawn
   Devin—none0/10

   No evidence describes attaching or indexing multiple project directories/repositories within a single Devin session; docs discuss single-repo indexing, knowledge, and environment setup but not multi-directory context in one session.

   ⚑ flag

   Conductor—none0/10

   Conductor's workspace model is built on git worktrees scoped to a single repository/branch per workspace (conductor-docs-27, conductor-docs-29), and there's no documentation of combining multiple project directories into one session. A community member explicitly requested multi-repo task support, implying it isn't currently available (conductor-comm-12).

   • [claimed-docs] “Create a new workspace with Command + N when work should have its own branch, files, running environment, and review path.”
   • [claimed-docs] “Conductor creates a separate working tree for each workspace. That lets agents work in parallel without editing the same files on disk.”
   • [community] “I've been looking for a tool like this, that lets Claude operate on multiple repos... but all the tools for background/multiplexing are alwa…”

   proof ↗⚑ flag

3. developerAdd a project instructions file to set coding standards and conventions the agent follows

   weight 3 · round to Devin
   Devin✓fullclaimed9/10

   Devin explicitly supports AGENTS.md, an open standard for providing context and instructions to AI agents, plus a separate 'Knowledge' feature for instructions/advice referenced across all sessions, directly covering project-level coding standards/conventions. Missing for 10: independent/hands-on confirmation that these instructions are reliably followed in practice.

   • [claimed-docs] “Devin supports AGENTS.md - a simple, open standard for providing context and instructions to AI agents.”
   • [claimed-docs] “Knowledge is a collection of instructions and advice that Devin can reference in all sessions.”

   proof ↗⚑ flag

   Conductor~partialclaimed4/10

   Docs mention 'General preferences' that 'apply broad instructions to agents in a repository,' which is the closest match to a project instructions/conventions file, but there is no detail on file format, location, or how it maps to underlying agents' native instruction files (e.g., CLAUDE.md). Missing for 10: documentation of the actual file/config mechanism, examples of setting coding standards, and independent confirmation it works across all supported agents (Claude Code, Codex, Cursor, OpenCode).

   • [claimed-docs] “`General preferences` apply broad instructions to agents in a repository.”

   proof ↗⚑ flag

Issue diagnosis

1. developerReproduce issues, narrow down root causes, and verify fixes

   weight 3 · round drawn
   Devin~partialcommunity6/10

   Docs explicitly claim Devin can 'repro and fix bugs' (devin-docs-1), use Ask Devin/DeepWiki to narrow root causes via codebase understanding (devin-docs-15, devin-docs-16, devin-docs-27), and verify fixes through CI iteration/Auto-Fix loops (devin-docs-18). However, hands-on community testimony reports Devin often adds extraneous changes beyond the task scope and cannot reliably undo them when asked, undermining clean verification of fixes (devin-comm-1), and lacks an escape hatch when stuck on root-cause diagnosis (devin-comm-2, devin-comm-3). Missing for 10: independent verification of successful bug reproduction/root-cause narrowing at scale, and resolution of the reported inability to cleanly revert unwanted changes during fix verification.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Use DeepWiki to navigate architecture and code with auto-generated documentation.”
   • [claimed-docs] “Ask Devin can answer questions about code structure and dependencies, and help you scope and plan tasks before implementation.”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “Devin does ask for help when it can't do something, but it really hates asking for help if it's a skill issue - it would prefer running in c…”

   proof ↗⚑ flag

   Conductor~partialclaimed6/10

   Conductor provides isolated worktrees/workspaces where agents can run builds, tests, and setup scripts (conductor-docs-17, conductor-docs-32, conductor-docs-29), diff/PR review paths to verify fixes (conductor-docs-2, conductor-docs-21), and checkpoints to revert code/chat state when narrowing down a bad change (conductor-docs-18). These features support the reproduce→diagnose→verify loop, but the evidence is all first-party docs describing environment/orchestration features rather than direct debugging tooling (log inspection, stack traces, targeted bisection) or independent hands-on accounts of successfully reproducing/root-causing a bug. missing for 10: dedicated debugging/log-inspection features, independent user reports of using Conductor to isolate root causes or verify fixes end-to-end.

   • [claimed-docs] “The agent can test your repositories, update install and setup scripts, and run builds — without asking you to confirm each step.”
   • [claimed-docs] “You can also have an agent configure the computer for you: it can test your repositories, edit setup scripts, and run builds.”
   • [claimed-docs] “Conductor creates a separate working tree for each workspace. That lets agents work in parallel without editing the same files on disk.”
   • [claimed-docs] “Each task gets its own workspace, branch, files, terminal, diff, and review path.”
   • [claimed-docs] “When the work is ready, Conductor helps you review the diff, open a pull request, merge, and archive the workspace.”
   • [claimed-docs] “Checkpoints | Session/workspace | Revert code and chat state to an earlier turn”
   • [claimed-docs] “Search file contents in your current local project or cloud workspace with ⌘⇧F.”

   proof ↗⚑ flag

Marketplace

1. developerEquip the agent with custom skills to perform specialized tasks

   weight 1 · round to Devin
   Devin~partialclaimed6/10

   Devin supports several skill-like extension mechanisms — 'Knowledge' (persistent instructions/advice for all sessions), AGENTS.md for structured agent instructions, and 'Playbooks' exposed via MCP — plus an open-source 'Devin Handoff' explicitly described as a 'plugin and skill'. This gives developers real levers to encode specialized, reusable task behavior, but the docs don't show a dedicated skill-authoring UI/marketplace or detailed examples of building a complex custom skill, and there is no independent/community evidence confirming this works well in practice. Missing for 10: concrete examples/tutorials of authoring a non-trivial custom skill, a discoverable skills registry, and independent hands-on corroboration.

   • [claimed-docs] “Knowledge is a collection of instructions and advice that Devin can reference in all sessions.”
   • [claimed-docs] “Devin supports AGENTS.md - a simple, open standard for providing context and instructions to AI agents.”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “Devin Handoff is an open-source plugin and skill that brings the same handoff workflow to any coding agent — Claude Code, Codex, Cursor, and…”

   proof ↗⚑ flag

   Conductor—none0/10

   Conductor orchestrates existing coding agents (Claude Code, Codex, Cursor, OpenCode) and offers 'general preferences' for broad instructions, but there's no evidence of a custom skills/plugin/tool system for equipping agents with specialized capabilities; a community request even notes the lack of 'custom tools' in its menu (conductor-comm-2).

   • [claimed-docs] “`General preferences` apply broad instructions to agents in a repository.”
   • [community] “It'd be great to change the default branch used for creating new workspaces. I'd like the ability to add custom tools to the 'Open in...' me…”

   proof ↗⚑ flag

2. engineering-leadIntegrate third-party partner-built agent apps into my workflows

   weight 1 · round to Conductor
   Devin~partialprobed4/10

   Devin exposes an MCP server so external MCP-compatible agents/IDEs can access its session management and tools, and its open-source 'Devin Handoff' plugin interoperates with other coding agents (Claude Code, Codex, Cursor), showing some cross-agent workflow integration. However there is no evidence of a partner/marketplace ecosystem of third-party agent apps being integrated into Devin's own workflows. Missing for 10: a documented partner-app marketplace or catalog, evidence of installing/configuring third-party agent apps within Devin, and any case study of an engineering-lead orchestrating partner-built agents through Devin.

   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “Devin Handoff is an open-source plugin and skill that brings the same handoff workflow to any coding agent — Claude Code, Codex, Cursor, and…”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [probe] “official MCP server documented at https://docs.devin.ai/work-with-devin/devin-mcp”

   proof ↗⚑ flag

   Conductor~partialcommunity7/10

   Conductor natively integrates several third-party agent apps (Claude Code, Codex, Cursor, OpenCode) into its parallel-workspace workflow, with per-org connection configuration and subscription/API-key support, and even exposes its own MCP server so other agent clients can manage workspaces. However, community feedback shows requests for additional partners (Gemini CLI, Amazon Q) that aren't yet supported, indicating a fixed rather than open/extensible partner ecosystem. Missing for 10: an open plugin/marketplace model for arbitrary partner agents, and independent confirmation of seamless integration beyond the listed agents.

   • [claimed-docs] “Conductor lets you run Claude Code, Codex, Cursor, and OpenCode in parallel.”
   • [claimed-docs] “Configure cloud agent connections separately for each organization, and choose whether Claude Code, Codex, and Cursor use an API key or subs…”
   • [claimed-docs] “Sign in to your Cursor subscription for cloud workspaces.”
   • [claimed-docs] “Conductor's hosted Model Context Protocol (MCP) server lets ChatGPT, Claude, Codex, and other MCP clients manage your cloud workspaces.”
   • [community] “Love the design. does it build on electron? and will it support other code agents, like gemini cli, codex, opencode ext.”
   • [community] “Would be cool if I can use this with opencode, Amazon Q or whatever. I reckon the logic would be quite similar. Seen a few of these tools bu…”

   proof ↗⚑ flag

Team knowledge

1. engineering-leadCreate a shared workspace from my docs and repos as a common source of truth for the team

   weight 1 · round to Devin
   Devin~partialclaimed6/10

   Devin offers building blocks for a team-wide source of truth — repo indexing that powers 'Ask Devin' and DeepWiki architecture docs, org-wide 'Knowledge' referenced in all sessions, and AGENTS.md support for shared context — which collectively let a lead centralize docs/repo knowledge for the team. However, there's no explicit product feature framed as a 'shared workspace' UI for team-wide docs/repo browsing outside of Devin's own agent sessions. Missing for 10: a dedicated shared workspace/knowledge-base product surface for humans to browse, independent evidence of teams using it as a collaborative source of truth, and clarity on cross-repo doc aggregation beyond per-session knowledge.

   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”
   • [claimed-docs] “Knowledge is a collection of instructions and advice that Devin can reference in all sessions.”
   • [claimed-docs] “Devin supports AGENTS.md - a simple, open standard for providing context and instructions to AI agents.”
   • [claimed-docs] “Use DeepWiki to navigate architecture and code with auto-generated documentation.”
   • [claimed-docs] “Devin inspects your repository, figures out which tools, runtimes, and dependencies are needed, and generates the blueprint for you.”

   proof ↗⚑ flag

   Conductor~partialclaimed4/10

   Conductor's cloud workspaces are shared with the whole organization and teammates can follow, reassign, or pick up the same workspace/chat, giving some sense of a shared team space tied to a repo (conductor-docs-24, conductor-docs-25, conductor-docs-16). However, there's no evidence of a workspace built from 'docs' (knowledge base, wiki, or design docs) alongside repos, or of any feature explicitly positioned as a team 'source of truth' beyond per-repo agent preferences. missing for 10: docs ingestion/aggregation into a workspace, explicit source-of-truth knowledge base feature, independent corroboration of team-wide shared-workspace usage.

   • [claimed-docs] “Workspaces and chats are shared with your organization, so a teammate can open the same workspace, follow the same conversation, and pick up…”
   • [claimed-docs] “browse everyone's work in **Home**, contribute to a teammate's chat, follow workspaces you care about, and hand work off with **Reassign to*…”
   • [claimed-docs] “Right-click the workspace and choose **Reassign to** to make a teammate responsible for it.”
   • [claimed-docs] “`General preferences` apply broad instructions to agents in a repository.”

   proof ↗⚑ flag

Tool integration

1. developerConnect the agent to workflow tools like Jira, Slack, and Google Drive to extend its context

   weight 3 · round to Devin
   Devin~partialclaimed6/10

   Devin can be tagged in Slack/Teams threads and work Jira/Linear tickets, giving it direct workflow-tool integration, and its MCP server plus API enable further extension to other tools. However there is no explicit documented Google Drive integration, and no independent/hands-on corroboration of these integrations actually working in practice. missing for 10: Google Drive connector evidence, independent verification of Jira/Slack integration reliability.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Tagging Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”

   proof ↗⚑ flag

   Conductor—none0/10

   Evidence shows Conductor integrates with GitHub and Linear (issue/branch creation) and exposes an MCP server for managing cloud workspaces, but there is no mention of Jira, Slack, or Google Drive integrations anywhere in the docs or community evidence.

   ⚑ flag

2. developerKick off agent tasks directly from GitHub, GitLab, Linear, or Slack

   weight 2 · round to Devin
   Devin~partialclaimed6/10

   Docs confirm Devin can be invoked from Linear/Jira tickets and Slack/Teams threads, plus API integration for building custom workflow triggers, but there's no explicit mention of GitHub or GitLab issue/PR-based task kickoff in the evidence pack. Community evidence doesn't directly contradict the integration claims, only general effectiveness concerns. Missing for 10: explicit GitHub/GitLab-triggered task creation documentation, independent hands-on confirmation of these specific integrations working.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Tagging Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”

   proof ↗⚑ flag

   Conductor~partialclaimed5/10

   Conductor lets you create a workspace (kick off an agent task) from a GitHub branch, pull request, GitHub issue, or Linear issue, and can trigger agent runs via GitHub Actions/scheduled routines, but there is no evidence of GitLab or Slack integration for starting tasks. missing for 10: GitLab task-kickoff support, Slack task-kickoff support, and independent confirmation of these triggers working in practice.

   • [claimed-docs] “Use Command + Shift + N or the `...` button next to `New workspace` to create a workspace from a branch, pull request, GitHub issue, or Line…”
   • [claimed-docs] “Introducing routines! You can now run your agents on a schedule or via GitHub action.”

   proof ↗⚑ flag

Cross device continuity

1. developerStart a task on one device and continue it later from another device or browser

   weight 2 · round to Devin
   Devin✓fullclaimed7/10

   Devin's cloud sessions run in a dedicated VM that persists independent of the local device (docs-13), can be started from Slack/Teams, IDE, CLI, or API and continued/taken over in the embedded IDE or web UI (docs-3, docs-4, docs-7, docs-24), and Devin Handoff explicitly lets you start work locally and continue in a cloud session accessible from a browser (docs-38). missing for 10: no explicit first-party walkthrough of resuming the same session from a different browser/device login, and no independent/community confirmation of cross-device continuity.

   • [claimed-docs] “Devin is designed to be a conversational user interface, and allows you to follow and take over Devin's development process in the embedded …”
   • [claimed-docs] “Devin CLI, a local command-line coding agent with deep Devin Cloud integration.”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “Hand a task off to a cloud Devin session and keep working locally.”

   proof ↗⚑ flag

   Conductor~partialclaimed6/10

   Cloud workspaces are shared with the organization and support handoff via 'Reassign to' and shared links, so a teammate (or the same developer on another device) can open a workspace and pick up where they left off, and cloud agents keep working after the laptop closes. However, evidence is framed around team collaboration/handoff rather than explicit single-user cross-device continuity, and local (non-cloud) workspaces are tied to the machine's worktree. missing for 10: explicit documentation of the same developer resuming a *local* task from a different device, confirmation of seamless single-user cross-browser/device session continuity, and independent hands-on confirmation of this specific workflow.

   • [claimed-docs] “Workspaces and chats are shared with your organization, so a teammate can open the same workspace, follow the same conversation, and pick up…”
   • [claimed-docs] “browse everyone's work in **Home**, contribute to a teammate's chat, follow workspaces you care about, and hand work off with **Reassign to*…”
   • [claimed-docs] “The link opens the workspace in Conductor for any member of the organization.”
   • [claimed-docs] “Sandboxes spin up in seconds, and agents keep working after you close your laptop.”
   • [claimed-docs] “Following is useful when someone else is assigned to the workspace but you want to keep it in your workflow.”

   proof ↗⚑ flag

Ide integration

1. developerChat with the coding assistant directly inside my IDE for contextual help

   weight 3 · round to Conductor
   Devin~partialclaimed5/10

   Devin exposes a conversational interface via its own embedded IDE within cloud sessions (devin-docs-3) and its Desktop app that imports VS Code/Cursor settings (devin-docs-9), plus an MCP server letting 'any MCP-compatible AI agent or IDE' access sessions (devin-docs-6/23). However there's no evidence of a native extension that lets a developer chat with Devin directly inside their own existing IDE (e.g., a VS Code/JetBrains plugin) — Devin's model is its own IDE/Desktop environment or MCP bridging rather than embedding in the user's IDE. Missing for 10: a first-party IDE extension for VS Code/JetBrains enabling in-IDE chat, and independent confirmation of this workflow working well in practice.

   • [claimed-docs] “Devin is designed to be a conversational user interface, and allows you to follow and take over Devin's development process in the embedded …”
   • [claimed-docs] “Import VS Code or Cursor settings, configure themes, and start coding with AI-powered assistance.”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”

   proof ↗⚑ flag

   Conductor✓fullcommunity7/10

   Conductor provides each task/workspace its own chat, terminal, diff and review path directly alongside the running coding agent (Claude Code, Codex, Cursor, OpenCode), letting a developer converse with the assistant in context of their code (conductor-docs-2, conductor-docs-27). Community reports confirm the chat works locally against Claude Code with no meaningful complaint about chat context/quality beyond stylistic preference (conductor-comm-9, conductor-comm-15). Missing for 10: no evidence of a native plugin embedding this chat inside third-party IDEs like VS Code/JetBrains (it's a separate Mac app), and no independent hands-on review of contextual-help quality beyond one HN thread.

   • [claimed-docs] “Each task gets its own workspace, branch, files, terminal, diff, and review path.”
   • [claimed-docs] “Create a new workspace with Command + N when work should have its own branch, files, running environment, and review path.”
   • [community] “There's a 'feel' to the way Claude Code outputs the text. And for input as well. Sadly, this is lost with conductor. I just don't feel as jo…”
   • [community] “Your messages are just between you and Claude Code — it all runs on your local Claude Code installation via the SDK.”

   proof ↗⚑ flag

Session management

1. developerReview diffs visually and run multiple sessions side by side in a desktop app

   weight 2 · round to Conductor
   Devin~partialclaimed4/10

   Devin has a documented Desktop app (docs-9) and supports running multiple independent sessions in parallel (docs-25), but there is no evidence of a visual diff review feature or explicit side-by-side session UI within the desktop app itself. Missing for 10: explicit documentation of an in-app diff viewer, UI showing multiple sessions simultaneously in one window, and any hands-on/community confirmation of this desktop workflow.

   • [claimed-docs] “Import VS Code or Cursor settings, configure themes, and start coding with AI-powered assistance.”
   • [claimed-docs] “Devin is designed to be a conversational user interface, and allows you to follow and take over Devin's development process in the embedded …”
   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”

   proof ↗⚑ flag

   Conductor✓fullprobed8/10

   Conductor is a native desktop (Mac) app that runs multiple coding agents (Claude Code, Codex, Cursor, OpenCode) in parallel, each in its own workspace/branch/worktree with a dedicated diff and review path before opening a PR, and community users independently confirm the git-worktree-based parallel session model. Missing for 10: independent hands-on evaluation specifically praising the visual diff-review UI's quality/UX (only vendor docs describe the diff view) and no screenshots/video corroboration.

   • [claimed-docs] “Each task gets its own workspace, branch, files, terminal, diff, and review path.”
   • [claimed-docs] “When the work is ready, Conductor helps you review the diff, open a pull request, merge, and archive the workspace.”
   • [claimed-docs] “Create a new workspace with Command + N when work should have its own branch, files, running environment, and review path.”
   • [claimed-docs] “Conductor creates a separate working tree for each workspace. That lets agents work in parallel without editing the same files on disk.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://www.conductor.build/llms.txt # Conductor > Conductor is a Mac app that lets you run many coding agents …”
   • [community] “Oh cool, I was already doing this with git worktrees but a ui for it would be handy.”
   • [community] “We create an isolated git worktree locally on your machine — whereas Codex (I believe) is running a container on the cloud.”

   proof ↗⚑ flag

2. engineering-leadManage multiple agent-driven coding sessions from one unified workspace

   weight 2 · round to Conductor
   Devin~partialcommunity6/10

   Devin supports running multiple parallel sessions ('carve out independent tasks and run them simultaneously'), an API to create sessions on behalf of users, and an embedded IDE/CLI/desktop app to interact with sessions, which together enable a lead-like workspace for managing several agent sessions. However, there's no dedicated 'unified workspace' dashboard evidence for an engineering-lead specifically monitoring/managing a team's multiple concurrent sessions, and community feedback highlights session reliability issues that would complicate multi-session oversight. missing for 10: explicit multi-session dashboard/UI for a lead role, team-level session oversight features, independent corroboration of smooth multi-session management at scale.

   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “Devin is designed to be a conversational user interface, and allows you to follow and take over Devin's development process in the embedded …”
   • [claimed-docs] “Devin CLI, a local command-line coding agent with deep Devin Cloud integration.”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

   Conductor✓fullcommunity8/10

   Conductor is explicitly built as a unified workspace for running multiple coding agents (Claude Code, Codex, Cursor, OpenCode) in parallel, each with its own workspace/branch/terminal/diff, plus team collaboration features (reassign, follow, shared workspaces) that support engineering-lead oversight. Community hands-on posts corroborate the parallel-agent workflow, though some raised concerns about permissions/data practices unrelated to the core multi-session management claim. missing for 10: independent lead-level testimony specifically on cross-team oversight at scale, and clearer evidence of a dashboard view aggregating all sessions' status for a lead.

   • [claimed-docs] “Conductor lets you run Claude Code, Codex, Cursor, and OpenCode in parallel.”
   • [claimed-docs] “Each task gets its own workspace, branch, files, terminal, diff, and review path.”
   • [claimed-docs] “Workspaces and chats are shared with your organization, so a teammate can open the same workspace, follow the same conversation, and pick up…”
   • [claimed-docs] “browse everyone's work in **Home**, contribute to a teammate's chat, follow workspaces you care about, and hand work off with **Reassign to*…”
   • [claimed-docs] “Create a new workspace with Command + N when work should have its own branch, files, running environment, and review path.”
   • [claimed-docs] “Conductor creates a separate working tree for each workspace. That lets agents work in parallel without editing the same files on disk.”
   • [community] “I have been imagining something like this would be perfect for working with Claude Code... This feels like just a nice clean simple extensio…”
   • [community] “Oh cool, I was already doing this with git worktrees but a ui for it would be handy.”

   proof ↗⚑ flag

Terminal workflow

1. developerRun a coding agent locally from my terminal

   weight 3 · round to Devin
   Devin✓fullprobed8/10

   Devin CLI is explicitly documented as a local command-line coding agent that can be invoked from terminal (e.g. `devin -- check out this code...`), with local sandboxing (--sandbox flag) and deep integration with Devin Cloud for handoff. Missing for 10: independent/hands-on community verification specifically of the CLI experience (community evidence only covers the cloud/browser Devin product, not the local CLI).

   • [claimed-docs] “Devin CLI, a local command-line coding agent with deep Devin Cloud integration.”
   • [claimed-docs] “devin -- check out this code and suggest a feasible, helpful feature”
   • [claimed-docs] “a local command-line coding agent with deep Devin Cloud integration”
   • [claimed-docs] “The --sandbox flag runs the CLI with OS-level isolation, enforcing writable paths and deny rules at the operating-system level and optionall…”
   • [claimed-docs] “The `--sandbox` flag runs the CLI with OS-level isolation, enforcing writable paths and `deny` rules at the operating-system level”
   • [probe] “official CLI documented at https://docs.devin.ai/cli/index”

   proof ↗⚑ flag

   Conductor~partialprobed7/10

   Conductor documents running local coding agents (Claude Code, Codex, Cursor, OpenCode) with per-task local git worktrees and a dedicated terminal per workspace, and community confirms it runs the agent locally via the local CLI/SDK install (conductor-comm-15, conductor-comm-17). However, hands-on reports show it isn't a pure lightweight local terminal wrapper—it requires GitHub OAuth/cloning rather than just running an existing local repo, and some users complain the local CLI 'feel' (e.g., Claude Code's native terminal UX) is lost inside Conductor's GUI (conductor-comm-6, conductor-comm-9). Missing for 10: independent confirmation that pure terminal-only (non-GUI) workflows are fully supported, and clearer first-party disclosure addressing the community concerns about local vs. cloud/GitHub dependency.

   • [claimed-docs] “Conductor lets you run Claude Code, Codex, Cursor, and OpenCode in parallel.”
   • [claimed-docs] “Each task gets its own workspace, branch, files, terminal, diff, and review path.”
   • [claimed-docs] “Conductor creates a separate working tree for each workspace. That lets agents work in parallel without editing the same files on disk.”
   • [community] “Your messages are just between you and Claude Code — it all runs on your local Claude Code installation via the SDK.”
   • [community] “We create an isolated git worktree locally on your machine — whereas Codex (I believe) is running a container on the cloud.”
   • [community] “I was really excited to try this but this does NOT work the way I expected. I wanted a simple git worktree manager for my existing, already-…”
   • [community] “There's a 'feel' to the way Claude Code outputs the text. And for input as well. Sadly, this is lost with conductor. I just don't feel as jo…”
   • [probe] “PROBE llms.txt: HTTP 200 at https://www.conductor.build/llms.txt # Conductor > Conductor is a Mac app that lets you run many coding agents …”

   proof ↗⚑ flag

2. developerRun the agent non-interactively in scripts for workflow automation

   weight 2 · round to Devin
   Devin✓fullclaimed7/10

   Devin exposes a documented API for creating sessions programmatically (including on behalf of users) and explicit CI/CD pipeline integration to auto-respond to static-analysis findings, plus a CLI (`devin -- <prompt>`) that can be invoked headlessly, all pointing to non-interactive, scriptable automation. Missing for 10: independent/hands-on confirmation of headless CLI scripting in real CI pipelines and more detail on CLI exit codes/output for scripting.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Devin CLI, a local command-line coding agent with deep Devin Cloud integration.”
   • [claimed-docs] “devin -- check out this code and suggest a feasible, helpful feature”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “a local command-line coding agent with deep Devin Cloud integration”

   proof ↗⚑ flag

   Conductor~partialclaimed6/10

   Conductor exposes a programmatic API to create workspaces, send prompts and read agent replies, and supports 'routines' to run agents on a schedule or via GitHub Action, which enables non-interactive, scripted automation of the agent outside the GUI. However, this is all first-party documentation with no independent/hands-on confirmation, and Conductor is fundamentally a GUI-first Mac app rather than a CLI tool built for scripting. Missing for 10: independent verification that the API/routines work reliably in real automation pipelines, and clearer CLI-style invocation/flags for non-interactive use.

   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically.”
   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically. Use it to do things like: Create workspaces, send prompts to the coding…”
   • [claimed-docs] “Introducing routines! You can now run your agents on a schedule or via GitHub action.”
   • [claimed-docs] “Conductor's hosted Model Context Protocol (MCP) server lets ChatGPT, Claude, Codex, and other MCP clients manage your cloud workspaces.”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round drawn
   Devin~partialprobed5/10

   Devin exposes a documented API (session creation, create_as_user_id) and an MCP server giving 'full access to session management, playbooks, knowledge, and scheduling', suggesting broad programmatic parity, but there's no explicit claim of full UI/API feature parity, and probes for an OpenAPI spec all 404'd, indicating the full API surface isn't transparently documented. Missing for 10: explicit parity statement covering UI-only features like Devin Review/Auto-Fix/Computer Use/desktop app settings, a discoverable OpenAPI schema, and independent confirmation that all UI actions are API-reachable.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

   Conductor~partialprobed5/10

   Conductor documents a programmatic API and a hosted MCP server that let you create cloud workspaces, send prompts, and read agent replies, giving genuine API access to core agent workflows (conductor-docs-13, conductor-docs-14, conductor-docs-30, conductor-probe-4). However, the API is explicitly scoped to 'cloud workspaces' only, with no evidence it exposes local workspace/worktree management, collaboration features (reassign, follow, sharing), settings like port forwarding, or UI-specific conveniences (loadouts, sections, checkpoints) — and no OpenAPI spec is discoverable (conductor-probe-3), suggesting the API surface is narrower than the full UI. missing for 10: full parity coverage of local workspace/git-worktree operations via API, coverage of collaboration/organization features via API, and a public OpenAPI spec or independent confirmation of API completeness.

   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically.”
   • [claimed-docs] “Conductor's hosted Model Context Protocol (MCP) server lets ChatGPT, Claude, Codex, and other MCP clients manage your cloud workspaces.”
   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically. Use it to do things like: Create workspaces, send prompts to the coding…”
   • [probe] “official MCP server documented at https://www.conductor.build/docs/api/mcp”
   • [probe] “PROBE openapi: all candidate paths 404 (https://www.conductor.build/openapi.json, https://www.conductor.build/swagger.json, https://www.cond…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round drawn
   Devin—none0/10

   No evidence of a data export feature or open-format export/portability mechanism for Devin sessions, knowledge, or artifacts; the docs cover API, CLI, MCP, and infrastructure but nothing about exporting all user data in open formats to leave the platform.

   ⚑ flag

   Conductor—none0/10

   Conductor stores workspace state, chat history, and cloud workspace data, but no evidence in the pack shows an explicit data-export feature or open-format export guarantee; while code lives in git worktrees (inherently portable), there's no documentation of exporting chats, settings, or cloud workspace metadata. Community threads even raise unresolved concerns about data practices and lack of transparency (conductor-comm-3, conductor-comm-5), reinforcing the absence of an export/leave story.

   • [claimed-docs] “Conductor creates a separate working tree for each workspace. That lets agents work in parallel without editing the same files on disk.”
   • [community] “Love it! Even just simply freeing my main branch would be a big win so I can keep working as well. But no way to find out if there's any dat…”
   • [community] “Full read-write access required to all your Github account's repos. Not just code. Settings, deploy keys. The works... Zero disclosure of da…”

   proof ↗⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round drawn
   Devin—none0/10

   Devin is a closed, proprietary commercial product; no evidence indicates its source code is available under an open license. The only mention of 'open-source' is for the separate Devin Handoff plugin/skill, not Devin itself.

   • [claimed-docs] “Devin Handoff is an open-source plugin and skill that brings the same handoff workflow to any coding agent — Claude Code, Codex, Cursor, and…”

   proof ↗⚑ flag

   Conductor—none0/10

   There is no evidence Conductor's source is available under any open license; it is distributed as a compiled Mac app with docs/API only, and a community comment explicitly contrasts it with an open-source alternative ('Crystal... unlike Conductor is open source'), indicating Conductor's source is not open.

   • [community] “Crystal can do all of this and more, and unlike Conductor is open source.”

   proof ↗⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round drawn
   Devin—none0/10

   Devin is a cloud-based SaaS agent; Outposts lets you run sessions on your own infrastructure but the core Devin model/orchestration itself remains Cognition-hosted, and there's no evidence of a self-hostable core product/model package. No mention of on-prem/self-hosted deployment of the core Devin engine.

   • [claimed-docs] “Outposts lets you run Devin sessions inside infrastructure you control — your own VMs, containers, Kubernetes clusters, or even a Mac Mini o…”

   proof ↗⚑ flag

   Conductor—none0/10

   Conductor is a proprietary Mac app with a hosted cloud service and API/MCP server; there is no evidence of a self-hostable core product—no open-source repo, on-prem deployment option, or self-hosting docs are mentioned. Community even contrasts it unfavorably with 'Crystal,' which is explicitly noted as open source unlike Conductor, reinforcing that self-hosting isn't offered.

   • [community] “Crystal can do all of this and more, and unlike Conductor is open source.”
   • [claimed-docs] “Sandboxes spin up in seconds, and agents keep working after you close your laptop.”
   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically.”

   proof ↗⚑ flag

Authentication

1. developerAuthenticate with an API key instead of an account login

   weight 2 · round to Conductor
   Devin~partialprobed6/10

   Devin has a documented API (devin-docs-7) intended for programmatic integration, implying API key auth as an alternative to account login, and even supports creating sessions on behalf of other users (devin-docs-8), suggesting a service-level credential model. However, no explicit documentation of API key generation/management or authentication mechanics is present in the evidence pack. Missing for 10: explicit API key creation/management docs, confirmation that API key auth fully replaces login flows, and independent/community verification of this workflow.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

   Conductor✓fullclaimed7/10

   Conductor explicitly supports 'bring your own subscriptions and keys' and lets you configure Claude Code, Codex, and Cursor connections to use an API key instead of a subscription/account login per organization. This directly satisfies the developer's need to authenticate via API key rather than an account login flow. missing for 10: independent/hands-on confirmation that API-key auth works end-to-end without any account sign-in step, and detail on whether Conductor's own app access also supports API-key-only login (vs. GitHub OAuth).

   • [claimed-docs] “Bring your own subscriptions and keys”
   • [claimed-docs] “Configure cloud agent connections separately for each organization, and choose whether Claude Code, Codex, and Cursor use an API key or subs…”

   proof ↗⚑ flag

2. engineering-leadAuthenticate through an enterprise identity or cloud platform for compliance and scalability

   weight 2 · round drawn
   Devin—none0/10

   No evidence of SSO/SAML, OIDC, or enterprise identity provider integration (e.g., Okta, Azure AD, Google Workspace) for authentication; the only related item is API session creation on behalf of a user, which is not enterprise identity/SSO authentication.

   ⚑ flag

   Conductor—none0/10

   Evidence shows only GitHub OAuth/GitHub App authentication and per-organization agent connection settings (conductor-comm-13, conductor-comm-14, conductor-docs-23); there is no mention of SSO, SAML, OIDC, or enterprise identity provider integration for compliance/scale. missing for 10: SSO/SAML/OIDC support, enterprise IdP integration, admin/compliance controls tied to identity.

   • [community] “Right now the app uses GitHub's OAuth sign in which unfortunately doesn't allow for fine-grained permissions... We're switching our sign-in …”
   • [community] “Fixed! You can now give Conductor fine-grained GitHub repository access. Or, skip the integration and use your local GitHub CLI auth.”
   • [claimed-docs] “Configure cloud agent connections separately for each organization, and choose whether Claude Code, Codex, and Cursor use an API key or subs…”

   proof ↗⚑ flag

3. developerSign in with my existing product subscription plan to use the coding agent

   weight 2 · round to Conductor
   Devin—none0/10

   The evidence describes Devin's own subscription tiers (Pro, Max, Core/Team/Enterprise) but nothing about letting a developer sign in using an existing external subscription (e.g., an existing LLM provider or IDE subscription) to access the agent. No mention of SSO-linked billing or bring-your-own-subscription support.

   • [claimed-docs] “Max is for individual users who consistently exceed the Pro quota. It includes everything in Pro, plus a significantly larger weekly usage q…”
   • [claimed-docs] “Power users who need more quota”

   proof ↗⚑ flag

   Conductor✓fullclaimed8/10

   Docs explicitly state you can 'bring your own subscriptions and keys' and sign in with existing Cursor, Claude Code, or Codex subscriptions rather than requiring a separate Conductor-specific plan, with per-organization control over subscription vs API key. Missing for 10: independent hands-on confirmation that subscription sign-in works smoothly across all supported agents (only vendor changelog/docs evidence).

   • [claimed-docs] “Bring your own subscriptions and keys”
   • [claimed-docs] “Sign in to your Cursor subscription for cloud workspaces.”
   • [claimed-docs] “Configure cloud agent connections separately for each organization, and choose whether Claude Code, Codex, and Cursor use an API key or subs…”

   proof ↗⚑ flag

4. developerSign in with a personal account to get free-tier access without managing API keys

   weight 1 · round drawn
   Devin—none0/10

   Evidence only describes paid tiers (Pro, Max, 'power users who need more quota') and API key based integration; there's no mention of a free tier accessible via personal account sign-in without API key management.

   • [claimed-docs] “Max is for individual users who consistently exceed the Pro quota. It includes everything in Pro, plus a significantly larger weekly usage q…”
   • [claimed-docs] “Power users who need more quota”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”

   proof ↗⚑ flag

   Conductor—none0/10

   Conductor's docs describe a 'bring your own subscriptions and keys' model where users must sign in to their own Claude Code, Codex, or Cursor subscription or supply an API key (conductor-docs-19, conductor-docs-23, conductor-docs-7); there is no mention of a free tier accessible purely via personal account sign-in without managing credentials. Community discussion also focuses on GitHub OAuth/permissions issues, not a free-tier access model.

   • [claimed-docs] “Bring your own subscriptions and keys”
   • [claimed-docs] “Configure cloud agent connections separately for each organization, and choose whether Claude Code, Codex, and Cursor use an API key or subs…”
   • [claimed-docs] “Sign in to your Cursor subscription for cloud workspaces.”

   proof ↗⚑ flag

Model choice

1. developerLet the tool automatically pick the best model for each task

   weight 1 · round drawn
   Devin—none0/10

   No evidence that Devin offers automatic model selection or model-routing based on task complexity; docs discuss quotas, tiers (Pro/Max), and features but never mention choosing among multiple underlying models per task.

   ⚑ flag

   Conductor—none0/10

   Conductor documents manual model selection via 'loadouts' and keyboard shortcuts to switch between chosen models, but there is no evidence of an automatic mechanism that picks the best model per task based on cost/performance tradeoffs.

   • [claimed-docs] “Pick a loadout of your favorite models to quickly switch between. It’s keyboard accessible too: change models (⌃⌘ 1-5), effort (⌘⇧/), speed …”
   • [claimed-docs] “Configure cloud agent connections separately for each organization, and choose whether Claude Code, Codex, and Cursor use an API key or subs…”

   proof ↗⚑ flag

2. developerChoose which underlying AI model powers my session from multiple providers

   weight 2 · round to Conductor
   Devin—none0/10

   No evidence in the pack mentions selecting or switching between multiple underlying AI model providers (e.g., choosing between different LLMs) for a Devin session; documentation covers agent capabilities, environments, and billing tiers but not model selection.

   ⚑ flag

   Conductor✓fullcommunity8/10

   Conductor explicitly supports running Claude Code, Codex, Cursor, and OpenCode as interchangeable providers, with a 'loadout' UI and keyboard shortcuts to switch models per session, plus per-organization configuration of API key vs subscription for each provider. Community comments confirm interest in and some support for multi-agent/provider use, though no independent hands-on review specifically validates seamless mid-session switching. Missing for 10: independent/hands-on verification of the model-switching UX and confirmation across all listed providers beyond vendor docs.

   • [claimed-docs] “Conductor lets you run Claude Code, Codex, Cursor, and OpenCode in parallel.”
   • [claimed-docs] “Pick a loadout of your favorite models to quickly switch between. It’s keyboard accessible too: change models (⌃⌘ 1-5), effort (⌘⇧/), speed …”
   • [claimed-docs] “Configure cloud agent connections separately for each organization, and choose whether Claude Code, Codex, and Cursor use an API key or subs…”
   • [community] “Love the design. does it build on electron? and will it support other code agents, like gemini cli, codex, opencode ext.”
   • [community] “Would be cool if I can use this with opencode, Amazon Q or whatever. I reckon the logic would be quite similar. Seen a few of these tools bu…”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round drawn
   Devin—none0/10

   No evidence of data residency/region selection controls for where Devin stores data; the pack covers deployment options (Outposts, VPN, sandbox isolation) but nothing about choosing storage region/jurisdiction.

   ⚑ flag

   Conductor—none0/10

   No evidence anywhere in the pack addresses data residency, region selection, or storage location controls for cloud workspaces; community comments even highlight lack of disclosure about where data is sent.

   • [community] “Love it! Even just simply freeing my main branch would be a big win so I can keep working as well. But no way to find out if there's any dat…”
   • [community] “Full read-write access required to all your Github account's repos. Not just code. Settings, deploy keys. The works... Zero disclosure of da…”

   proof ↗⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   Devin—none0/10

   The evidence pack contains no mention of data-training opt-out, data usage policy, or privacy controls for AI model training; nothing in the docs or community sources addresses this axis.

   ⚑ flag

   Conductor—none0/10

   No documentation or policy statement anywhere in the evidence pack addresses training-data opt-out or data-usage controls; in fact community reports explicitly note 'no way to find out if there's any data sent to your servers' and 'zero disclosure of data practices,' underscoring the absence of any such privacy control.

   • [community] “Love it! Even just simply freeing my main branch would be a big win so I can keep working as well. But no way to find out if there's any dat…”
   • [community] “Full read-write access required to all your Github account's repos. Not just code. Settings, deploy keys. The works... Zero disclosure of da…”

   proof ↗⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round drawn
   Devin—none0/10

   No evidence pack items address data retention policies, data deletion controls, or privacy/data-handling settings for user data or session data; the closest items concern sandboxing/isolation of execution environments, not retention/deletion of stored data.

   ⚑ flag

   Conductor—none0/10

   No documentation describes retention periods, data-deletion controls, or export/purge mechanisms for cloud workspace data; community feedback explicitly flags 'zero disclosure of data practices' and no way to verify what is sent to Conductor's servers.

   • [community] “Love it! Even just simply freeing my main branch would be a big win so I can keep working as well. But no way to find out if there's any dat…”
   • [community] “Full read-write access required to all your Github account's repos. Not just code. Settings, deploy keys. The works... Zero disclosure of da…”
   • [claimed-docs] “The Conductor API lets you manage cloud workspaces programmatically.”

   proof ↗⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Devin—none0/10

   No evidence pack item mentions telemetry opt-out, usage tracking settings, or privacy controls of that kind; the docs cover features, sandboxing, and infrastructure but not data-collection opt-out mechanisms.

   ⚑ flag

   Conductor—none0/10

   No documentation or changelog entry describes any telemetry/usage-tracking settings or an opt-out mechanism; community commenters explicitly note there is 'no way to find out if there's any data sent to your servers' and 'zero disclosure of data practices,' confirming the absence of any documented privacy control for telemetry.

   • [community] “Love it! Even just simply freeing my main branch would be a big win so I can keep working as well. But no way to find out if there's any dat…”
   • [community] “Full read-write access required to all your Github account's repos. Not just code. Settings, deploy keys. The works... Zero disclosure of da…”

   proof ↗⚑ flag

Data governance

1. engineering-leadOpt out of having my code and prompts used for AI model training

   weight 1 · round drawn
   Devin—none0/10

   No evidence in the pack addresses data usage, training opt-out, or privacy/data-handling policies for Devin; nothing documents an opt-out mechanism for code/prompt use in model training.

   ⚑ flag

   Conductor—none0/10

   No evidence anywhere in the pack of a data-usage/training opt-out policy or setting; in fact community reports explicitly complain about 'zero disclosure of data practices' and no way to find out what is sent to Conductor's servers, reinforcing the absence of any documented opt-out mechanism.

   • [community] “Love it! Even just simply freeing my main branch would be a big win so I can keep working as well. But no way to find out if there's any dat…”
   • [community] “Full read-write access required to all your Github account's repos. Not just code. Settings, deploy keys. The works... Zero disclosure of da…”

   proof ↗⚑ flag

Pr review

1. developerHave the agent stage changes, write commit messages, create branches, and open pull requests

   weight 3 · round to Conductor
   Devin~partialcommunity6/10

   Devin's docs describe it implementing features/fixing bugs and producing pull requests that get automated review and iteration (devin-docs-17, devin-docs-18), implying it handles the full git workflow (branch, commit, PR) autonomously, but no doc explicitly details staging, commit-message generation, or branch creation as discrete steps. Community feedback (devin-comm-1) also notes it can add extraneous changes it can't cleanly undo, a real caveat on commit hygiene. Missing for 10: explicit documentation of commit/staging/branch mechanics and independent confirmation that generated commits/PRs are clean and reviewable.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Devin Review provides automated first-pass reviews on pull requests, checking for correctness and conformance with organizational best pract…”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”

   proof ↗⚑ flag

   Conductor✓fullcommunity8/10

   Docs explicitly state each task gets its own branch/worktree, agents can be given autonomy to test/build without confirmation, and Conductor 'helps you review the diff, open a pull request, merge, and archive the workspace' — covering branch creation, staging/commits (implied by agent workflow), diffs, and PR creation. Community evidence corroborates git worktree branch isolation and GitHub integration for PR workflows. Missing for 10: explicit first-party mention of 'commit message writing' as a distinct feature and independent hands-on confirmation of the full stage→commit→branch→PR pipeline working end-to-end.

   • [claimed-docs] “When the work is ready, Conductor helps you review the diff, open a pull request, merge, and archive the workspace.”
   • [claimed-docs] “Create a new workspace with Command + N when work should have its own branch, files, running environment, and review path.”
   • [claimed-docs] “Conductor creates a separate working tree for each workspace. That lets agents work in parallel without editing the same files on disk.”
   • [claimed-docs] “The agent can test your repositories, update install and setup scripts, and run builds — without asking you to confirm each step.”
   • [community] “We create an isolated git worktree locally on your machine — whereas Codex (I believe) is running a container on the cloud.”
   • [community] “Fixed! You can now give Conductor fine-grained GitHub repository access. Or, skip the integration and use your local GitHub CLI auth.”

   proof ↗⚑ flag

2. developerGet automatic code review with contextual feedback on every pull request

   weight 3 · round to Devin
   Devin✓fullclaimed8/10

   Devin Review is explicitly documented as an automated first-pass PR reviewer checking correctness and org best-practice conformance, with Auto-Fix closing the loop by responding to review comments and CI failures. This directly matches the story of automatic contextual code review on PRs, though evidence is vendor-documentation only with no independent hands-on validation of review quality/contextual accuracy. Missing for 10: independent/community corroboration of review quality, and detail on how 'contextual feedback' is surfaced per-PR beyond docs description.

   • [claimed-docs] “Devin Review provides automated first-pass reviews on pull requests, checking for correctness and conformance with organizational best pract…”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixe”

   proof ↗⚑ flag

   Conductor—none0/10

   Conductor's evidence describes parallel agent orchestration, diffs, and human-facing review workflows (e.g., 'Conductor helps you review the diff, open a pull request' and PR comments loading from GitHub) but no automated code-review bot that posts contextual feedback on pull requests. No evidence of an AI reviewer analyzing PR diffs and commenting automatically.

   • [claimed-docs] “When the work is ready, Conductor helps you review the diff, open a pull request, merge, and archive the workspace.”
   • [claimed-docs] “PR comments and failing-check logs now load while a cloud workspace is asleep.”

   proof ↗⚑ flag

3. developerInspect diffs and run checks to catch problems before merging

   weight 3 · round to Devin
   Devin✓fullclaimed8/10

   Devin's docs describe Devin Review, which performs automated first-pass PR reviews checking correctness and org conformance, plus CI/CD integration to respond to static-analysis findings (SonarQube, Fortify, Veracode) and Auto-Fix iterating on CI failures — directly enabling developers to inspect diffs and run checks before merge. This is corroborated by explicit SDLC integration workflow docs, not just a single mention. missing for 10: independent/hands-on evidence confirming Devin Review's diff-inspection quality in practice, and detail on how diffs are surfaced/inspected by the developer (UI specifics) beyond docs claims.

   • [claimed-docs] “Devin Review provides automated first-pass reviews on pull requests, checking for correctness and conformance with organizational best pract…”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixe”

   proof ↗⚑ flag

   Conductor~partialclaimed6/10

   Docs show each workspace has its own diff and review path, and Conductor explicitly helps you 'review the diff, open a pull request, merge' before finishing work, plus it surfaces PR comments and failing-check logs even while a cloud workspace sleeps, and agents can run builds/tests as part of setup. However, there's no detailed description of built-in linting/test-runner integration beyond agent-run builds, and no independent/hands-on confirmation that this catches real problems pre-merge. Missing for 10: dedicated CI/check-running feature docs, independent verification of diff/check accuracy, and coverage of how failing checks block or warn before merge.

   • [claimed-docs] “Each task gets its own workspace, branch, files, terminal, diff, and review path.”
   • [claimed-docs] “When the work is ready, Conductor helps you review the diff, open a pull request, merge, and archive the workspace.”
   • [claimed-docs] “PR comments and failing-check logs now load while a cloud workspace is asleep.”
   • [claimed-docs] “The agent can test your repositories, update install and setup scripts, and run builds — without asking you to confirm each step.”
   • [claimed-docs] “You can also have an agent configure the computer for you: it can test your repositories, edit setup scripts, and run builds.”

   proof ↗⚑ flag

Safe execution

1. engineering-leadControl which external tools and integrations the agent is allowed to access

   weight 2 · round to Devin
   Devin~partialclaimed6/10

   Devin exposes some admin-level controls over its access—an org-wide 'Enable desktop mode' toggle for Computer Use, a CLI --sandbox flag enforcing OS-level writable-path/deny rules and network restriction, and Outposts for running sessions in infra you control—giving leads levers to constrain what Devin can reach or do. However there's no documented centralized policy/allowlist for specific external integrations (e.g., disabling Slack, Jira, GitHub, VPN access per-tool) or granular permission/audit management for engineering leads. missing for 10: a unified integration-permission/allowlist admin panel, per-tool enable/disable controls beyond desktop mode, and independent verification of these controls in practice.

   • [claimed-docs] “Computer Use is controlled by the Enable desktop mode toggle in your organization's customization options.”
   • [claimed-docs] “The --sandbox flag runs the CLI with OS-level isolation, enforcing writable paths and deny rules at the operating-system level and optionall…”
   • [claimed-docs] “The `--sandbox` flag runs the CLI with OS-level isolation, enforcing writable paths and `deny` rules at the operating-system level”
   • [claimed-docs] “Outposts lets you run Devin sessions inside infrastructure you control — your own VMs, containers, Kubernetes clusters, or even a Mac Mini o…”
   • [claimed-docs] “Devin can connect to a VPN from inside its workspace, so sessions can reach internal services such as package registries, databases, and int…”

   proof ↗⚑ flag

   Conductor~partialcommunity5/10

   Conductor lets an org configure agent connections per organization (choosing API-key vs subscription per agent) and, after community pushback over full GitHub OAuth access, added fine-grained GitHub repository permissions or local GitHub CLI auth as an alternative [conductor-docs-23, conductor-comm-13, conductor-comm-14]. However there's no documented allow-list/deny-list for arbitrary external tools, MCP servers, or third-party integrations beyond GitHub scopes and model provider choice, and the initial full-write-access design (comm-4, comm-5, comm-6) shows the control was originally coarse and only partially remedied. missing for 10: granular per-tool/integration allow-listing beyond GitHub and model provider, admin-level policy enforcement across the org, and independent verification that fine-grained access covers all agent-invoked external services (e.g., MCP servers).

   • [claimed-docs] “Configure cloud agent connections separately for each organization, and choose whether Claude Code, Codex, and Cursor use an API key or subs…”
   • [community] “Any way to have it not require full write access to your entire GitHub account?”
   • [community] “Full read-write access required to all your Github account's repos. Not just code. Settings, deploy keys. The works... Zero disclosure of da…”
   • [community] “Right now the app uses GitHub's OAuth sign in which unfortunately doesn't allow for fine-grained permissions... We're switching our sign-in …”
   • [community] “Fixed! You can now give Conductor fine-grained GitHub repository access. Or, skip the integration and use your local GitHub CLI auth.”

   proof ↗⚑ flag

2. engineering-leadHave the agent operate inside a sandbox when interacting with code, tools, and network resources

   weight 2 · round to Devin
   Devin✓fullclaimed8/10

   Devin runs cloud sessions in isolated VMs (devin-docs-13) and provides an explicit CLI --sandbox flag enforcing OS-level isolation, writable path restrictions, deny rules, and optional network restriction (devin-docs-12, devin-docs-37), directly matching the sandboxed code/tool/network isolation story. Missing for 10: independent/hands-on verification of sandbox robustness and more detail on network isolation guarantees beyond docs claims.

   • [claimed-docs] “The --sandbox flag runs the CLI with OS-level isolation, enforcing writable paths and deny rules at the operating-system level and optionall…”
   • [claimed-docs] “The `--sandbox` flag runs the CLI with OS-level isolation, enforcing writable paths and `deny` rules at the operating-system level”
   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “Devin can connect to a VPN from inside its workspace, so sessions can reach internal services such as package registries, databases, and int…”

   proof ↗⚑ flag

   Conductor~partialcommunity5/10

   Conductor's cloud workspaces are explicitly described as spinning up in "sandboxes" and the agent can run builds/tests without step-by-step confirmation, suggesting isolated execution for cloud mode. However, the local mode (the primary use case per community feedback) uses plain git worktrees on the user's own machine with no described network/tool sandboxing, and early versions required full read-write GitHub account access with no disclosed data practices, which is the opposite of a hardened sandbox model (though later mitigated with fine-grained GitHub App permissions). Missing for 10: explicit sandbox isolation details (container/VM boundaries, network egress controls) for local workspaces, and independent confirmation that cloud sandboxes restrict network/tool access beyond marketing language.

   • [claimed-docs] “Sandboxes spin up in seconds, and agents keep working after you close your laptop.”
   • [claimed-docs] “The agent can test your repositories, update install and setup scripts, and run builds — without asking you to confirm each step.”
   • [community] “We create an isolated git worktree locally on your machine — whereas Codex (I believe) is running a container on the cloud.”
   • [community] “Any way to have it not require full write access to your entire GitHub account?”
   • [community] “Full read-write access required to all your Github account's repos. Not just code. Settings, deploy keys. The works... Zero disclosure of da…”
   • [community] “Right now the app uses GitHub's OAuth sign in which unfortunately doesn't allow for fine-grained permissions... We're switching our sign-in …”
   • [community] “Fixed! You can now give Conductor fine-grained GitHub repository access. Or, skip the integration and use your local GitHub CLI auth.”

   proof ↗⚑ flag

Security checks

1. engineering-leadSee license and public-code matching references for AI-suggested code

   weight 1 · round drawn
   Devin—none0/10

   No evidence of license or public-code matching/attribution references for AI-suggested code anywhere in the docs; Devin Review covers correctness and best-practice conformance but nothing about license/provenance matching.

   ⚑ flag

   Conductor—none0/10

   No evidence anywhere in the pack of license compliance checks, public-code/plagiarism matching, or provenance references for AI-suggested code; Conductor's documentation focuses on orchestration, workspaces, and diffs/PRs but never mentions license or code-provenance scanning.

   ⚑ flag