Devin vs Slate

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to Devin
   Devin✓fullprobed9/10

   Devin's own docs site serves llms.txt (HTTP 200, confirmed by probe) and per-page .md variants, and also supports AGENTS.md as an agent-oriented instructions standard, directly matching the story. Missing for 10: no independent/community confirmation that external agents have actually consumed llms.txt successfully.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.devin.ai/llms.txt # Devin Docs - [Desktop (100 pages)](https://docs.devin.ai/_llms/en/desktop.md):…”
   • [probe] “PROBE docs-md: HTTP 200 at https://docs.devin.ai/get-started/devin-intro.md > ## Documentation Index > Fetch the complete documentation inde…”
   • [claimed-docs] “Devin supports AGENTS.md - a simple, open standard for providing context and instructions to AI agents.”

   proof ↗⚑ flag

   Slate✓fullprobed8/10

   A probe confirms Slate's docs site serves a valid llms.txt at the root with links to actual docs pages, directly satisfying the ability to point an agent at agent-oriented docs. Missing for 10: no independent/community confirmation that agents successfully consume this llms.txt in practice, and no broader agent-oriented doc format beyond the single file.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.randomlabs.ai/llms.txt # Slate ## Docs - [Introduction](https://docs.randomlabs.ai/en/getting-sta…”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to Devin
   Devin✓fullclaimed8/10

   Devin offers a full API for creating/managing sessions programmatically (including create_as_user_id for automation on behalf of users), a CLI with a --sandbox flag for OS-level isolated headless runs, and explicit CI/CD pipeline integration for responding to static analysis findings and PR checks, all supporting headless/automated usage without a human in the loop. Missing for 10: no independent/hands-on report specifically validating CI automation workflows end-to-end, and no explicit CI example (e.g., GitHub Actions snippet) beyond doc references.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “The --sandbox flag runs the CLI with OS-level isolation, enforcing writable paths and deny rules at the operating-system level and optionall…”
   • [claimed-docs] “a local command-line coding agent with deep Devin Cloud integration”

   proof ↗⚑ flag

   Slate—none0/10

   The docs describe Slate as an interactive terminal agent (onboarding, slash commands, hotkeys, subagent cards) with a permission-bypass flag (--dangerously-skip-permissions/--yolo), but there is no mention of a headless mode, non-interactive CLI flags, exit-code/scriptable output, or CI integration examples. Missing for 10: documented headless/non-interactive invocation, CI pipeline examples, scriptable output format, and any evidence of automation use outside the interactive TUI.

   • [claimed-docs] “We support `--dangerously-skip-permissions` (alias: `--yolo`) to bypass permission prompts.”
   • [claimed-docs] “npm i -g @randomlabs/slate”
   • [claimed-docs] “Use `/sessions` to switch between existing sessions”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round drawn
   Devin—none0/10

   The evidence only documents Devin exposing its own MCP server so other agents/IDEs can call Devin's tools (session management, playbooks, knowledge, scheduling) — the reverse direction of this story. There is no evidence that a user can configure Devin itself to consume/plug in external MCP servers so Devin can use their tools.

   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [probe] “official MCP server documented at https://docs.devin.ai/work-with-devin/devin-mcp”

   proof ↗⚑ flag

   Slate—none0/10

   No evidence in the pack mentions MCP servers or integrating external tool providers into Slate; the docs cover skills, permissions, orchestration, and CLI usage but never MCP support. Missing for 10: any documentation or claim about connecting/plugging in MCP servers, configuring MCP tool sources, or using MCP-provided tools.

   ⚑ flag

4. ai-native userUse an official CLI

   weight 2 · round drawn
   Devin✓fullprobed8/10

   Devin has an official CLI ('Devin CLI, a local command-line coding agent with deep Devin Cloud integration') with documented usage examples and a sandbox flag for OS-level isolation, confirmed by both docs and probe. Missing for 10: independent/hands-on community verification of the CLI specifically (community evidence covers the web/session product, not CLI usage).

   • [claimed-docs] “Devin CLI, a local command-line coding agent with deep Devin Cloud integration.”
   • [claimed-docs] “devin -- check out this code and suggest a feasible, helpful feature”
   • [claimed-docs] “The --sandbox flag runs the CLI with OS-level isolation, enforcing writable paths and deny rules at the operating-system level and optionall…”
   • [claimed-docs] “a local command-line coding agent with deep Devin Cloud integration”
   • [probe] “official CLI documented at https://docs.devin.ai/cli/index”

   proof ↗⚑ flag

   Slate✓fullprobed8/10

   Slate is delivered as an official CLI (npm-installed, terminal-based) with rich first-party docs covering install, sessions, hotkeys, shell execution, and configuration — squarely matching the 'official CLI' story for an AI-native user. Missing for 10: independent/hands-on corroboration of the CLI experience itself (community evidence found only relates to unrelated porting-quality claims, not CLI usage).

   • [claimed-docs] “npm i -g @randomlabs/slate”
   • [claimed-docs] “Use `/sessions` to switch between existing sessions”
   • [claimed-docs] “Press Tab to queue the current message so it runs after the current turn finishes.”
   • [claimed-docs] “Execute shell commands directly with `!`”
   • [claimed-docs] “Ctrl+X then N New session”
   • [probe] “official CLI documented at https://docs.randomlabs.ai/en/getting-started/quickstart”

   proof ↗⚑ flag

5. ai-native userDrive the product through a documented public API

   weight 3 · round to Devin
   Devin✓fullprobed8/10

   Devin ships a documented public API (docs-7) with session creation, org-level features like create_as_user_id (docs-8), plus a CLI and MCP server for programmatic/agentic control (docs-4, docs-6, probe-4, probe-5), directly enabling AI-native users to drive it programmatically. Missing for 10: a discoverable OpenAPI/swagger spec (probe-3 found 404s on all candidate paths) and independent hands-on corroboration of API usage.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “Devin CLI, a local command-line coding agent with deep Devin Cloud integration.”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [probe] “official MCP server documented at https://docs.devin.ai/work-with-devin/devin-mcp”
   • [probe] “official CLI documented at https://docs.devin.ai/cli/index”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

   Slate—none0/10

   Slate is documented as a CLI/terminal agent with configuration, skills, and hotkeys, but there is no evidence of a documented public API for programmatic/agentic access—the OpenAPI probe returned 404 across all candidate paths and no SDK or REST/API docs are mentioned anywhere in the pack.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.randomlabs.ai/openapi.json, https://docs.randomlabs.ai/swagger.json, https://docs.rando…”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.randomlabs.ai/llms.txt # Slate ## Docs - [Introduction](https://docs.randomlabs.ai/en/getting-sta…”

   proof ↗⚑ flag

6. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round drawn
   Devin—none0/10

   Devin exposes a general API (devin-docs-7) and can act on behalf of a specified user via create_as_user_id (devin-docs-8), but there is no evidence of scoped/least-privilege API key or token issuance, role-based permission scopes, or credential-level restriction mechanisms for agent access.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”

   proof ↗⚑ flag

   Slate—none0/10

   Slate is a coding-agent CLI; its evidence only covers permission settings (allow/ask/deny) for tool actions, not issuance of scoped/least-privilege API credentials or tokens for agents. No mention of credential/token scoping, API key generation, or IAM-style access control.

   • [claimed-docs] “Each permission key maps to an action ("allow", "ask", or "deny"), or a pattern object for fine-grained control.”
   • [claimed-docs] “We support `--dangerously-skip-permissions` (alias: `--yolo`) to bypass permission prompts.”

   proof ↗⚑ flag

7. ai-native userBuild against official SDKs

   weight 2 · round to Devin
   Devin~partialprobed4/10

   Devin provides an API for integration (docs-7, docs-8) allowing developers to build applications and automate workflows, but the evidence never mentions dedicated official SDKs/client libraries (e.g., Python/JS packages), and probes for an OpenAPI spec that would back SDK generation all returned 404s (devin-probe-3). Missing for 10: named SDK packages in specific languages, SDK installation/usage docs, and a published OpenAPI/schema artifact confirming SDK-generation support.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

   Slate—none0/10

   The evidence pack covers Slate's CLI, skills, configuration, and orchestration features but contains no mention of an official SDK (Python/TypeScript/etc.) for building applications on top of Slate, and the OpenAPI probe returned 404s across all candidate paths. Missing for 10: any documented SDK package, API reference, or programmatic interface for building against Slate.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.randomlabs.ai/openapi.json, https://docs.randomlabs.ai/swagger.json, https://docs.rando…”
   • [claimed-docs] “npm i -g @randomlabs/slate”

   proof ↗⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round to Devin
   Devin✓fullclaimed7/10

   Devin generates insights/suggestions from a user's own codebase data via 'Ask Devin' (code structure/dependency Q&A), auto-generated DeepWiki documentation, and Devin Review's automated PR feedback, all built on repository indexing. Missing for 10: independent/hands-on validation of the quality of these AI-generated insights (community evidence covers general task execution issues, not this specific feature) and no benchmark of insight accuracy.

   • [claimed-docs] “Ask Devin can answer questions about code structure and dependencies, and help you scope and plan tasks before implementation.”
   • [claimed-docs] “Devin Review provides automated first-pass reviews on pull requests, checking for correctness and conformance with organizational best pract…”
   • [claimed-docs] “Use DeepWiki to navigate architecture and code with auto-generated documentation.”
   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”

   proof ↗⚑ flag

   Slate~partialcommunity4/10

   Slate is a coding-agent CLI whose evidence shows it can analyze a codebase and produce suggestions (e.g., generating an ARCH.md with improvement ideas), which maps loosely to 'AI-generated insights from data' but only in the narrow sense of source code, not general data analysis. Community evidence also raises skepticism about the real quality of generated output (e.g., criticism of a ported-code example as low quality/unverified). Missing for 10: evidence of insights/suggestions over non-code datasets, dashboards or analytics-style outputs, and independent validation of suggestion quality.

   • [claimed-docs] “Please review the architecture of my entire codebase creating an ARCH.md and then give me ways I can improve it.”
   • [community] “Blog post claimed porting a library with one sentence, but critic noted it was JS->TS (trivial rename) not Python->TS, excluded tests/exampl…”
   • [community] “"Why trumpet code that is so ready for the garbage that you wouldn't even bother to publish it" - skepticism about the quality/usefulness of…”

   proof ↗⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Devin
   Devin✓fullclaimed8/10

   Devin explicitly supports background/autonomous execution: cloud sessions run in their own VM and 'keep going after you close your laptop' (devin-docs-13), MCP access includes 'scheduling' (devin-docs-6/23), the API lets you 'automate workflows' and create sessions programmatically (devin-docs-7/8), CI/CD integration triggers Devin on findings (devin-docs-19), and Auto-Fix creates a closed loop that iterates PRs 'without you in the loop' (devin-docs-18). This spans scheduled triggers, API-driven automation, and hands-off background operation. Missing for 10: independent/community verification that scheduled automations run reliably unattended, and more detail on a dedicated 'automation/schedule' UI beyond scattered doc mentions.

   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”

   proof ↗⚑ flag

   Slate~partialclaimed5/10

   Slate supports background subagents, parallel task orchestration, and built-in workflows like goal/deep-research that run while the user keeps interacting, which shows some autonomous background execution. However, this is task-level parallelism within an active session, not scheduled or trigger-based automations that run independently of user presence. missing for 10: evidence of scheduled/cron-like automations, persistent background jobs surviving session end, or trigger-based (event-driven) autonomous runs without an active user session.

   • [claimed-docs] “Those agents show up as a grid of inline subagent cards, one per agent.”
   • [claimed-docs] “While one or more agents run in the background, you can keep talking with Slate: plan next steps, queue up additional tasks, or spin up more…”
   • [claimed-docs] “`goal` and `deep-research` are built-in programs. They are user-visible workflows, not something you need to author before using Slate.”

   proof ↗⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round to Slate
   Devin~partialcommunity6/10

   Devin's core product is designed for task delegation — via Ask Devin, ticket assignment, Slack/Teams tagging, and a conversational IDE (devin-docs-1, devin-docs-3, devin-docs-24) — so the axis clearly applies and is well documented. However, hands-on community reports show real caveats: Devin can add extraneous unrequested changes it can't undo, gets stuck for long periods without asking for help, and requires active babysitting/session termination to get value (devin-comm-1, devin-comm-2, devin-comm-5), undercutting a fully seamless delegation experience. Missing for 10: independent verification that delegated tasks reliably complete without extraneous side-effects or getting stuck, and stronger corroboration beyond one HN thread.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Devin is designed to be a conversational user interface, and allows you to follow and take over Devin's development process in the embedded …”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

   Slate✓fullcommunity7/10

   Slate is a CLI-based AI assistant where users delegate whole tasks (e.g., 'review architecture and write ARCH.md') and it spins up parallel subagents, orchestration programs like goal/deep-research, and long multi-hour sessions per first-party docs. Community evidence (comm-1/2/3) raises skepticism about output quality/novelty but does not contradict the core delegation mechanism itself. Missing for 10: independent hands-on validation that delegated multi-agent tasks reliably complete as advertised.

   • [claimed-docs] “Parallelize working and orchestration of many tasks at once.”
   • [claimed-docs] “Please review the architecture of my entire codebase creating an ARCH.md and then give me ways I can improve it.”
   • [claimed-docs] “Those agents show up as a grid of inline subagent cards, one per agent.”
   • [claimed-docs] “While one or more agents run in the background, you can keep talking with Slate: plan next steps, queue up additional tasks, or spin up more…”
   • [claimed-docs] “`goal` and `deep-research` are built-in programs. They are user-visible workflows, not something you need to author before using Slate.”
   • [community] “"Why trumpet code that is so ready for the garbage that you wouldn't even bother to publish it" - skepticism about the quality/usefulness of…”

   proof ↗⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round to Devin
   Devin✓fullcommunity8/10

   Devin is explicitly designed as a conversational agent: users assign tasks via natural language (Slack/Teams tagging, chat interface, CLI prompts like 'devin -- check out this code...'), and it interprets these into autonomous coding/dev actions across IDE, CLI, and cloud sessions. Community evidence corroborates it operates on natural-language task descriptions in practice, though with noted friction around scope creep and knowing when to stop. Missing for 10: independent benchmarking of NL command accuracy/robustness and richer detail on how ambiguous instructions are resolved.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Tagging Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “Devin is designed to be a conversational user interface, and allows you to follow and take over Devin's development process in the embedded …”
   • [claimed-docs] “devin -- check out this code and suggest a feasible, helpful feature”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

   Slate✓fullclaimed7/10

   Docs show Slate is driven primarily via natural-language prompts (e.g. the quickstart example 'Please review the architecture of my entire codebase...') alongside slash-commands, shell escapes, and file references, indicating natural-language is the core interaction mode for an AI-native agent CLI. Missing for 10: independent/hands-on confirmation that complex natural-language commands are reliably parsed and executed as intended (community evidence only discusses code-porting quality, not NL command usage itself).

   • [claimed-docs] “Please review the architecture of my entire codebase creating an ARCH.md and then give me ways I can improve it.”
   • [claimed-docs] “Execute shell commands directly with `!`”
   • [claimed-docs] “Use `@filename` references”
   • [claimed-docs] “Use `/sessions` to switch between existing sessions”
   • [claimed-docs] “`goal` and `deep-research` are built-in programs. They are user-visible workflows, not something you need to author before using Slate.”

   proof ↗⚑ flag

Api quality

1. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   Devin—none0/10

   Devin has a documented API (devin-docs-7) but a direct probe for a machine-readable OpenAPI/swagger spec returned 404 on all candidate paths, and no documentation item references a downloadable spec file.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”

   proof ↗⚑ flag

   Slate—none0/10

   Slate's docs site was directly probed for an OpenAPI/swagger spec at standard locations and all returned 404, and no documentation anywhere mentions a machine-readable API spec for AI-native consumption.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.randomlabs.ai/openapi.json, https://docs.randomlabs.ai/swagger.json, https://docs.rando…”

   proof ↗⚑ flag

2. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Devin—none0/10

   There's an API reference (devin-docs-7) and even an OpenAPI probe, but that probe found no OpenAPI spec (devin-probe-3), and no evidence anywhere mentions API versioning scheme or a documented deprecation policy for the API.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

   Slate—none0/10

   Slate is a CLI coding agent product; no evidence of any versioned public API, API reference, or deprecation policy documentation exists—openapi probes returned 404 and no docs mention API versioning or deprecation. Absence of evidence for this applicable axis (a product could plausibly document API stability) yields 'none'.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.randomlabs.ai/openapi.json, https://docs.randomlabs.ai/swagger.json, https://docs.rando…”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Slate
   Devin~partialcommunity5/10

   Devin's API supports programmatic session creation (including on behalf of other users) and docs explicitly encourage carving out independent tasks to run simultaneously, which together enable bulk-style automation across many tickets/items, but there is no dedicated 'bulk operations' or batch-processing feature documented, and community feedback raises concerns about reliability/oversight needed per session that would complicate true bulk workflows. Missing for 10: an explicit batch/bulk API endpoint or UI for processing many items in one request, and independent evidence of successful large-scale bulk runs.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”
   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”

   proof ↗⚑ flag

   Slate~partialclaimed6/10

   Docs describe running many agents in parallel and orchestrating multiple tasks simultaneously (grid of subagent cards, spinning up more agents to parallelize work), which supports bulk-style operations across many items. However, there's no independent corroboration of this working at scale and no detail on failure handling, limits, or item-level bulk operations (e.g., bulk file edits/refactors) beyond task orchestration. Missing for 10: independent/hands-on verification of large-scale parallel task execution, concrete examples of bulk item processing (files/records), and failure/error handling details at scale.

   • [claimed-docs] “Parallelize working and orchestration of many tasks at once.”
   • [claimed-docs] “Those agents show up as a grid of inline subagent cards, one per agent.”
   • [claimed-docs] “While one or more agents run in the background, you can keep talking with Slate: plan next steps, queue up additional tasks, or spin up more…”
   • [claimed-docs] “`goal` and `deep-research` are built-in programs. They are user-visible workflows, not something you need to author before using Slate.”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to Devin
   Devin~partialclaimed6/10

   Devin supports several built-in event-triggered automations — Auto-Fix responds automatically to PR review comments and CI failures, CI/CD integration triggers Devin off static-analysis findings (SonarQube/Fortify/Veracode), and MCP exposes 'scheduling' as a capability — but there's no evidence of a general-purpose, user-defined rules/webhook engine for arbitrary custom triggers. Missing for 10: documentation of a configurable custom-rule/webhook trigger system, details on the scheduling feature's flexibility, and independent confirmation that these automations work reliably as event triggers.

   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixe”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”

   proof ↗⚑ flag

   Slate—none0/10

   Slate's docs describe agent rules for permissions/behavior ordering (docs-15, docs-20) and orchestration of parallel agents (docs-12, docs-13), but there is no evidence of user-defined rules that trigger actions automatically on external events (e.g., file changes, webhooks, schedule, git events). This is a plausible axis for a coding agent (many support hooks/triggers), so absence of evidence yields none rather than na.

   • [claimed-docs] “Slate by default respects agent rules in the following order”
   • [claimed-docs] “Each permission key maps to an action ("allow", "ask", or "deny"), or a pattern object for fine-grained control.”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round to Devin
   Devin~partialclaimed3/10

   Devin's MCP docs mention 'scheduling' as one of the capabilities exposed to MCP-compatible agents, implying some scheduling functionality exists, but there is no dedicated documentation, UI, or examples describing recurring jobs, cron-like triggers, or workflow automation configuration. missing for 10: dedicated scheduling feature docs, examples of recurring/cron jobs, independent confirmation of scheduled workflows in practice.

   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”

   proof ↗⚑ flag

   Slate—none0/10

   Slate is a coding-agent CLI with orchestration/parallel-agent features and sessions, but nothing in the evidence describes scheduling recurring jobs or workflows (e.g., cron-like triggers, timed recurring runs). Orchestration docs cover on-demand parallelization, not recurrence.

   • [claimed-docs] “Those agents show up as a grid of inline subagent cards, one per agent.”
   • [claimed-docs] “While one or more agents run in the background, you can keep talking with Slate: plan next steps, queue up additional tasks, or spin up more…”
   • [claimed-docs] “`goal` and `deep-research` are built-in programs. They are user-visible workflows, not something you need to author before using Slate.”

   proof ↗⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round drawn
   Devin—none0/10

   The evidence shows Devin has knowledge, playbooks, and scheduling features but nothing about versioning, reviewing, or rolling back those automation configurations themselves; Devin Review/Auto-Fix pertains to PR code review, not to the automation definitions. Missing for 10: version history for playbooks/knowledge, a review workflow for automation changes, and a rollback mechanism for automations.

   ⚑ flag

   Slate—none0/10

   Evidence shows session management (/sessions, /workspace) and built-in 'programs' like goal/deep-research, but nothing about versioning automations, reviewing history of changes, or rolling back to prior states of an automation/workflow. Missing for 10: any documentation of version history, diffing, or rollback mechanisms for automations/workflows.

   • [claimed-docs] “Use `/sessions` to switch between existing sessions”
   • [claimed-docs] “Use `/workspace` to open the workspace manager, where you can review and remove workspace directories.”
   • [claimed-docs] “`goal` and `deep-research` are built-in programs. They are user-visible workflows, not something you need to author before using Slate.”

   proof ↗⚑ flag

Background execution

1. ai-native userHave a cloud agent build, test, and demo a feature end-to-end for my review

   weight 2 · round to Devin
   Devin~partialcommunity7/10

   Devin's cloud sessions run in dedicated VMs with shell/browser/full repo access, can implement features, run tests, and continue autonomously after handoff, then present PRs for review (docs-1,13,17). However, community hands-on reports describe unreliable autonomy — extraneous breaking changes, inability to self-correct, and needing frequent human monitoring/termination — undercutting the 'build, test, demo end-to-end' promise. missing for 10: reliable independent verification of unattended end-to-end demo quality, and clearer evidence of a built-in demo/walkthrough artifact for reviewers beyond PR creation.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “Devin Review provides automated first-pass reviews on pull requests, checking for correctness and conformance with organizational best pract…”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

   Slate!disputedcontradicted4/10

   Slate's docs claim orchestration of parallel background subagents and being 'one of the few agents capable of performing integration tests manually,' suggesting it could build and test a feature autonomously, but no docs mention a 'demo' output or cloud-hosted execution environment. Community hands-on critique of an actual Slate-produced port directly contradicts the build/test claim: reviewers found the work excluded tests/examples and provided no verifiable repo, undermining confidence that Slate reliably builds+tests end-to-end for review. Missing for 10: evidence of cloud/remote execution infra, an explicit demo-generation feature, and independent confirmation that test suites are actually run and pass.

   • [claimed-docs] “Those agents show up as a grid of inline subagent cards, one per agent.”
   • [claimed-docs] “While one or more agents run in the background, you can keep talking with Slate: plan next steps, queue up additional tasks, or spin up more…”
   • [claimed-docs] “Slate is one of the few agents capable of performing integration tests manually.”
   • [community] “Blog post claimed porting a library with one sentence, but critic noted it was JS->TS (trivial rename) not Python->TS, excluded tests/exampl…”
   • [community] “"Why trumpet code that is so ready for the garbage that you wouldn't even bother to publish it" - skepticism about the quality/usefulness of…”

   proof ↗⚑ flag

2. developerDelegate longer-running coding tasks to run in the background in an isolated cloud environment

   weight 3 · round to Devin
   Devin✓fullcommunity9/10

   Devin runs tasks in isolated cloud VMs with full shell/browser/repo access that persist after the user disconnects, explicitly supporting long-running background work and parallel independent tasks, corroborated by community reports of multi-day autonomous runs. Missing for 10: independent third-party benchmarking of long-running task reliability/quality beyond anecdotal HN reports.

   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”
   • [claimed-docs] “Configure it once, and every session boots into that known-good state.”
   • [claimed-docs] “Hand a task off to a cloud Devin session and keep working locally.”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “You can set a 'max work time' before Devin pauses so it won't go for days endlessly spending your credits. By default it's set to 10 credits…”

   proof ↗⚑ flag

   Slate—none0/10

   Slate's docs describe subagents running 'in the background' locally while you keep chatting and orchestration/parallelization of tasks, but there is no mention of an isolated cloud environment, remote execution sandbox, or delegation to a hosted service — everything described appears to run within the local CLI session. This axis is applicable to coding agent tools generally, but no evidence supports a cloud-isolated background execution capability for Slate.

   • [claimed-docs] “Those agents show up as a grid of inline subagent cards, one per agent.”
   • [claimed-docs] “While one or more agents run in the background, you can keep talking with Slate: plan next steps, queue up additional tasks, or spin up more…”
   • [claimed-docs] “Slate works with you across long, multi-hour sessions.”
   • [claimed-docs] “Parallelize working and orchestration of many tasks at once.”

   proof ↗⚑ flag

3. developerConfigure a reproducible cloud environment with the dependencies and setup steps my repository needs

   weight 2 · round to Devin
   Devin✓fullclaimed9/10

   Devin explicitly supports configuring environment 'blueprints' that specify tools, runtimes, and dependencies so 'every session boots into that known-good state,' with auto-detection of requirements from the repo (docs-31, docs-32), plus indexing (docs-27), knowledge/AGENTS.md context files (docs-29, docs-30), and VPN access for internal dependencies (docs-28), all running in isolated cloud VMs (docs-13). Missing for 10: independent/hands-on confirmation that blueprint-based environments reliably reproduce across sessions in practice.

   • [claimed-docs] “Configure it once, and every session boots into that known-good state.”
   • [claimed-docs] “Devin inspects your repository, figures out which tools, runtimes, and dependencies are needed, and generates the blueprint for you.”
   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”
   • [claimed-docs] “Knowledge is a collection of instructions and advice that Devin can reference in all sessions.”
   • [claimed-docs] “Devin supports AGENTS.md - a simple, open standard for providing context and instructions to AI agents.”
   • [claimed-docs] “Devin can connect to a VPN from inside its workspace, so sessions can reach internal services such as package registries, databases, and int…”
   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”

   proof ↗⚑ flag

   Slate—none0/10

   Slate's docs describe a local CLI agent (npm install, terminal sessions, permissions, skills, orchestration) but contain no mention of provisioning or configuring a reproducible cloud environment, dependency setup, or devcontainer-style configuration for a repository. This is a fair capability to ask of an autonomous coding agent, but no evidence shows Slate supports it.

   • [claimed-docs] “npm i -g @randomlabs/slate”
   • [claimed-docs] “Slate by default respects agent rules in the following order”
   • [claimed-docs] “Each permission key maps to an action ("allow", "ask", or "deny"), or a pattern object for fine-grained control.”

   proof ↗⚑ flag

Parallel agents

1. ai-native userLaunch fleets of autonomous agents that work in parallel on different tasks for hours or days

   weight 2 · round to Slate
   Devin~partialcommunity6/10

   Docs confirm parallel task execution ('Carve out independent tasks and run them simultaneously'), cloud sessions that persist after closing the laptop, and an API to spin up multiple sessions programmatically (including on behalf of other users), which together support a 'fleet of parallel long-running agents' story. However, community hands-on reports show real friction with the 'hours/days autonomous' claim: sessions can get stuck without an escape hatch, users must babysit and manually terminate sessions every 10-15 minutes, and there's a default max-work-time cap limiting unsupervised runtime. Missing for 10: independent verification of successful multi-day/multi-task fleets running unattended, and evidence addressing the reported lack of a reliable 'ask for help' escalation during long runs.

   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”
   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “Devin does ask for help when it can't do something, but it really hates asking for help if it's a skill issue - it would prefer running in c…”
   • [community] “You can set a 'max work time' before Devin pauses so it won't go for days endlessly spending your credits. By default it's set to 10 credits…”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

   Slate~partialcommunity7/10

   Docs describe genuine parallel agent orchestration (grid of subagent cards, spinning up more agents while others run in background) and explicitly support multi-hour sessions, matching much of the story. However, evidence only confirms 'multi-hour' not multi-day autonomy, and community commentary raises skepticism about output quality/novelty without directly refuting the parallel-agent mechanics themselves. Missing for 10: confirmation of multi-day unattended runs, independent hands-on validation of fleet-scale parallel task completion.

   • [claimed-docs] “Slate works with you across long, multi-hour sessions.”
   • [claimed-docs] “Parallelize working and orchestration of many tasks at once.”
   • [claimed-docs] “Those agents show up as a grid of inline subagent cards, one per agent.”
   • [claimed-docs] “While one or more agents run in the background, you can keep talking with Slate: plan next steps, queue up additional tasks, or spin up more…”
   • [community] “Commenter compared the approach to 'Ralph as a service' referencing an existing agentic coding technique (ghuntley.com/ralph), suggesting Sl…”

   proof ↗⚑ flag

2. developerRun several task attempts in parallel and compare results before choosing one

   weight 1 · round to Slate
   Devin—none0/10

   Devin's docs mention running multiple independent tasks simultaneously (devin-docs-25) but this describes parallelizing different tasks, not running several parallel attempts of the SAME task to compare and choose the best result. No evidence describes a compare/choose-best-attempt workflow.

   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”

   proof ↗⚑ flag

   Slate~partialclaimed6/10

   Slate's orchestration docs show multiple subagents running in parallel as a grid of cards while the user keeps working, directly supporting parallel task execution (docs-12, docs-13). However, there's no explicit documentation of a compare/diff view or a 'choose winning attempt' workflow for reconciling multiple parallel results into one choice. Missing for 10: explicit comparison/selection UI or workflow for multiple attempts of the same task, and independent/hands-on confirmation of this specific use case.

   • [claimed-docs] “Those agents show up as a grid of inline subagent cards, one per agent.”
   • [claimed-docs] “While one or more agents run in the background, you can keep talking with Slate: plan next steps, queue up additional tasks, or spin up more…”
   • [claimed-docs] “`goal` and `deep-research` are built-in programs. They are user-visible workflows, not something you need to author before using Slate.”

   proof ↗⚑ flag

Scheduled automation

1. ai-native userSet up always-on agents that run on schedules or triggers to maintain and fix my software autonomously

   weight 2 · round to Devin
   Devin~partialcommunity6/10

   Devin supports trigger-based autonomous work (Slack/Teams tags, PR review comments, CI/CD/static-analysis findings) and Auto-Fix creates a closed loop that iterates on CI failures without a human in the loop, and MCP exposes 'scheduling' as a session capability, all suggesting some always-on/triggered agent operation. However there's no dedicated docs for cron-like recurring schedules, and community reports describe sessions needing frequent human monitoring/termination rather than fully unattended long-running maintenance. Missing for 10: explicit scheduling/cron configuration docs, independent evidence of reliable unattended multi-day maintenance loops, and confirmation that Auto-Fix/CI triggers work without human oversight in practice.

   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “You can set a 'max work time' before Devin pauses so it won't go for days endlessly spending your credits. By default it's set to 10 credits…”

   proof ↗⚑ flag

   Slate—none0/10

   Slate's docs describe parallel subagent orchestration within a live session (background agents you keep talking to, spin up more agents to parallelize tasks) but there is no evidence of scheduling, event/webhook triggers, or persistent always-on agents that run autonomously outside an active session to maintain/fix software over time.

   • [claimed-docs] “Those agents show up as a grid of inline subagent cards, one per agent.”
   • [claimed-docs] “While one or more agents run in the background, you can keep talking with Slate: plan next steps, queue up additional tasks, or spin up more…”
   • [claimed-docs] “`goal` and `deep-research` are built-in programs. They are user-visible workflows, not something you need to author before using Slate.”

   proof ↗⚑ flag

Debugging

1. developerDebug a live running web application directly from my coding assistant

   weight 1 · round to Devin
   Devin~partialclaimed6/10

   Devin's computer-use/desktop mode gives it a full browser and desktop environment (mouse, keyboard, screenshots) plus VPN access to internal services, and docs explicitly mention reproducing and fixing bugs, which together support interacting with and debugging a live running app. However, there's no explicit documentation of dev-tools-style debugging features (breakpoints, console/log inspection, network tracing) or a dedicated 'live app debugging' workflow, and community evidence doesn't corroborate this specific use case. Missing for 10: explicit live-debugging tooling (breakpoints/console/log inspection), independent hands-on confirmation of debugging a running app.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Devin has access to a full desktop environment — not just a browser. It can move the mouse, click on UI elements, type on the keyboard, take…”
   • [claimed-docs] “Devin can connect to a VPN from inside its workspace, so sessions can reach internal services such as package registries, databases, and int…”
   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”

   proof ↗⚑ flag

   Slate—none0/10

   No evidence in the pack mentions debugging live running applications, attaching to running processes, browser/runtime debugging, or any live-app inspection capability; Slate's docs focus on codebase review, shell commands, orchestration, and skills, none of which address live debugging.

   ⚑ flag

2. developerDebug issues and troubleshoot using natural-language queries

   weight 2 · round to Devin
   Devin~partialcommunity7/10

   Docs clearly support NL-driven debugging: 'repro and fix bugs', 'Ask Devin can answer questions about code structure...help you scope and plan tasks', and tagging Devin in Slack/Teams about a bug thread. However, hands-on community reports describe practical caveats—Devin adding extraneous changes it can't undo, getting stuck without escalating, requiring manual babysitting—that temper reliability for troubleshooting workflows. Missing for 10: independent benchmark/case study specifically on debugging accuracy, and resolution of the 'getting stuck on bugs' community complaint.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Tagging Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “Ask Devin can answer questions about code structure and dependencies, and help you scope and plan tasks before implementation.”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “Devin does ask for help when it can't do something, but it really hates asking for help if it's a skill issue - it would prefer running in c…”

   proof ↗⚑ flag

   Slate~partialclaimed5/10

   Slate's docs show it operates via natural-language prompts, executes shell commands (`!`), references files (`@filename`), and can run integration tests and review codebase architecture in NL form, which implies it could be used for debugging and troubleshooting queries. However there is no explicit example, workflow, or documentation section dedicated to debugging/troubleshooting via natural language, and community evidence is skeptical/unrelated to this specific capability. Missing for 10: explicit debugging-focused examples or docs, independent verification that NL-based debugging works well, dedicated troubleshooting workflow beyond generic agent capabilities.

   • [claimed-docs] “Execute shell commands directly with `!`”
   • [claimed-docs] “Use `@filename` references”
   • [claimed-docs] “Slate is one of the few agents capable of performing integration tests manually.”
   • [claimed-docs] “Please review the architecture of my entire codebase creating an ARCH.md and then give me ways I can improve it.”

   proof ↗⚑ flag

Feature implementation

1. developerTurn a tracked issue into a complete pull request end-to-end

   weight 3 · round to Devin
   Devin~partialcommunity6/10

   Devin's docs explicitly describe taking Linear/Jira tickets and implementing full features, with Devin Review/Auto-Fix looping PRs toward merge-ready status without human involvement, covering the issue-to-PR pipeline end-to-end. However, hands-on community reports describe practical friction — extraneous unrelated changes that can break things, inability to easily undo them, and agents getting stuck for days rather than asking for help — casting doubt on how cleanly the 'complete' PR is delivered without oversight. Missing for 10: independent verification of a clean ticket→merged-PR flow without manual intervention, and resolution of the reported reliability/quality issues.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Devin Review provides automated first-pass reviews on pull requests, checking for correctness and conformance with organizational best pract…”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Code migrations, refactors, and modernization”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “Devin does ask for help when it can't do something, but it really hates asking for help if it's a skill issue - it would prefer running in c…”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

   Slate—none0/10

   The evidence pack covers Slate's session management, orchestration, skills, and permissions but contains no mention of issue-tracker integration (e.g., GitHub issues) or automated pull-request creation/submission. Without evidence of ingesting a tracked issue and producing a PR end-to-end, this capability is unshown.

   ⚑ flag

2. developerDescribe a feature or bug in plain language and have the agent implement or fix it across multiple files

   weight 3 · round drawn
   Devin!disputedcontradicted5/10

   Devin's docs strongly claim end-to-end feature/bug implementation across a full repo (Linear/Jira tickets, multi-file fixes, code migrations) with full workspace/VM access [devin-docs-1, devin-docs-13, devin-docs-21, devin-docs-27], but hands-on community reports concretely contradict smooth delivery: it 'always adds extraneous changes beyond the task that can break other things, and can't undo those changes if asked' and required constant supervision/termination to get real work done [devin-comm-1, devin-comm-5], with another user noting it can run for days without an escape hatch when stuck [devin-comm-2]. Missing for 10: independent benchmark data on multi-file correctness, and resolution of the extraneous-change/undo failure mode reported by users.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “Code migrations, refactors, and modernization”
   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

   Slate!disputedcontradicted5/10

   Docs imply broad multi-file code work (e.g. the quickstart example asks Slate to review an entire codebase and produce ARCH.md, plus orchestration features for parallelizing tasks across files/agents), suggesting Slate can act on plain-language requests across a codebase. However, independent community scrutiny of a specific real-world claim (a 'ported library' from one sentence) found it was actually a trivial JS->TS rename, excluded tests, lacked a verifiable repo, and drew explicit skepticism about the quality/usefulness of the generated code — concretely contradicting the marketed multi-file code-generation capability. Missing for 10: first-party documentation of a genuine multi-file bug-fix/feature-implementation workflow with verifiable before/after results, and independent hands-on confirmation that resolves the community dispute.

   • [claimed-docs] “Please review the architecture of my entire codebase creating an ARCH.md and then give me ways I can improve it.”
   • [claimed-docs] “While one or more agents run in the background, you can keep talking with Slate: plan next steps, queue up additional tasks, or spin up more…”
   • [claimed-docs] “Parallelize working and orchestration of many tasks at once.”
   • [community] “Blog post claimed porting a library with one sentence, but critic noted it was JS->TS (trivial rename) not Python->TS, excluded tests/exampl…”
   • [community] “"Why trumpet code that is so ready for the garbage that you wouldn't even bother to publish it" - skepticism about the quality/usefulness of…”

   proof ↗⚑ flag

Maintenance automation

1. developerHave the agent write tests, fix lint errors, resolve merge conflicts, and update dependencies for me

   weight 3 · round to Devin
   Devin~partialcommunity6/10

   Docs show Devin is a general-purpose coding agent that can fix bugs, implement features, iterate on CI failures, respond to review comments (Auto-Fix), and handle code migrations/refactors/modernization, which plausibly covers lint fixes and CI-related work, but none of the docs explicitly mention writing tests, resolving merge conflicts, or updating dependencies as named capabilities. Community reports (devin-comm-1) also note Devin can introduce extraneous changes and struggles to cleanly undo them, tempering confidence in reliably delivering these specific maintenance tasks. Missing for 10: explicit documentation/evidence of test-writing, lint-fixing, merge-conflict resolution, and dependency-update workflows, plus independent hands-on confirmation of these specific tasks succeeding.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Code migrations, refactors, and modernization”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”

   proof ↗⚑ flag

   Slate~partialcommunity3/10

   Slate is documented as a general-purpose coding agent with shell execution, file editing, permissioning, and orchestration of multiple sub-agents (random-labs-docs-9, random-labs-docs-13, random-labs-docs-16), which implies it could perform tasks like running tests or lint/dependency commands, but the evidence never explicitly documents test-writing, lint-fixing, merge-conflict resolution, or dependency updates as capabilities. Community commentary raises quality concerns about generated code but doesn't specifically address these tasks. Missing for 10: explicit documentation or examples of writing/fixing tests, resolving lint errors, resolving merge conflicts, and updating dependencies.

   • [claimed-docs] “Execute shell commands directly with `!`”
   • [claimed-docs] “While one or more agents run in the background, you can keep talking with Slate: plan next steps, queue up additional tasks, or spin up more…”
   • [claimed-docs] “Slate is one of the few agents capable of performing integration tests manually.”
   • [community] “"Why trumpet code that is so ready for the garbage that you wouldn't even bother to publish it" - skepticism about the quality/usefulness of…”

   proof ↗⚑ flag

Multimodal generation

1. ai-native userGenerate a working app from a sketch, image, or PDF design

   weight 2 · round drawn
   Devin—none0/10

   No evidence anywhere in the pack that Devin accepts a sketch/image/PDF design as input and generates a working app from it; documentation focuses on text-based tasks, tickets, Slack threads, code review, and CLI/desktop environment features with no multimodal design-to-app capability mentioned.

   ⚑ flag

   Slate—none0/10

   The evidence describes Slate as a terminal-based CLI agent for coding sessions, orchestration, skills, and permissions, but nothing in the docs or community evidence mentions accepting sketches, images, or PDF designs as input to generate an app. missing for 10: any mention of image/sketch/PDF input, multimodal design-to-code capability, or UI mockup ingestion.

   • [claimed-docs] “npm i -g @randomlabs/slate”
   • [claimed-docs] “Please review the architecture of my entire codebase creating an ARCH.md and then give me ways I can improve it.”
   • [claimed-docs] “Skills are markdown instruction packages that give the agent domain-specific knowledge and behavior.”
   • [claimed-docs] “description: "Create distinctive, production-grade frontend interfaces."”

   proof ↗⚑ flag

Codebase mapping

1. developerUnderstand how a codebase fits together to find where to start making changes

   weight 3 · round to Devin
   Devin✓fullclaimed7/10

   Devin explicitly indexes repositories to power 'Ask Devin' and 'DeepWiki', which answer questions about code structure and dependencies and help developers scope/plan where to start making changes, directly matching the story. missing for 10: independent/hands-on evidence validating DeepWiki/Ask Devin's accuracy on real codebases (community evidence only covers task execution, not codebase-understanding features).

   • [claimed-docs] “Use DeepWiki to navigate architecture and code with auto-generated documentation.”
   • [claimed-docs] “Ask Devin can answer questions about code structure and dependencies, and help you scope and plan tasks before implementation.”
   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”

   proof ↗⚑ flag

   Slate~partialclaimed5/10

   The quickstart example explicitly shows Slate producing an ARCH.md architecture review of an entire codebase with improvement suggestions, directly supporting codebase-understanding use cases, and @filename references plus workspace management help navigate a repo. However there's no dedicated codebase-mapping/search feature (e.g., symbol index, dependency graph) documented beyond this one example, and no independent evidence confirming quality of such architecture summaries. missing for 10: dedicated code-navigation/search tooling, independent validation of architecture-summary accuracy, more than a single example of codebase-understanding workflow.

   • [claimed-docs] “Please review the architecture of my entire codebase creating an ARCH.md and then give me ways I can improve it.”
   • [claimed-docs] “Use `@filename` references”
   • [claimed-docs] “Use `/workspace` to open the workspace manager, where you can review and remove workspace directories.”

   proof ↗⚑ flag

2. developerHave the agent map and explain an entire unfamiliar codebase without manually selecting context files

   weight 3 · round to Devin
   Devin✓fullclaimed7/10

   Docs describe repo indexing that lets Devin understand the codebase and power 'Ask Devin' and DeepWiki auto-generated architecture docs, directly enabling exploration/explanation of an unfamiliar codebase without manual file selection (devin-docs-27, devin-docs-15, devin-docs-16). No community evidence contradicts this specific capability. Missing for 10: independent/hands-on verification of codebase-mapping accuracy and no concrete example of DeepWiki output quality.

   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”
   • [claimed-docs] “Use DeepWiki to navigate architecture and code with auto-generated documentation.”
   • [claimed-docs] “Ask Devin can answer questions about code structure and dependencies, and help you scope and plan tasks before implementation.”

   proof ↗⚑ flag

   Slate~partialcommunity6/10

   Docs show Slate's quickstart example explicitly demonstrates asking it to 'review the architecture of my entire codebase' and generate an ARCH.md without manual file selection, and it has orchestration/subagent features for broad exploration. However there's no independent/hands-on verification that this codebase-mapping actually works well on large unfamiliar repos, and community evidence raises quality skepticism about other generated outputs. missing for 10: independent hands-on validation of full-codebase mapping accuracy, evidence of handling very large/unfamiliar codebases without manual curation, detail on how context is auto-selected under the hood.

   • [claimed-docs] “Please review the architecture of my entire codebase creating an ARCH.md and then give me ways I can improve it.”
   • [claimed-docs] “Those agents show up as a grid of inline subagent cards, one per agent.”
   • [claimed-docs] “While one or more agents run in the background, you can keep talking with Slate: plan next steps, queue up additional tasks, or spin up more…”
   • [community] “"Why trumpet code that is so ready for the garbage that you wouldn't even bother to publish it" - skepticism about the quality/usefulness of…”

   proof ↗⚑ flag

Context management

1. developerHave the agent build and recall memory automatically across sessions

   weight 2 · round to Devin
   Devin~partialclaimed5/10

   Devin supports persistent cross-session context via "Knowledge" (instructions referenced in all sessions), AGENTS.md, and environment blueprints that let every session boot into a known-good state, which enables some recall across sessions. However, these mechanisms are largely user-configured/onboarded rather than autonomously built by the agent from its own experience, and there's no evidence of automatic memory creation or recall behavior demonstrated in practice. Missing for 10: evidence the agent automatically extracts/updates memory from its own task experience without manual setup, and independent confirmation that recalled memory improves subsequent session performance.

   • [claimed-docs] “Knowledge is a collection of instructions and advice that Devin can reference in all sessions.”
   • [claimed-docs] “Devin supports AGENTS.md - a simple, open standard for providing context and instructions to AI agents.”
   • [claimed-docs] “Configure it once, and every session boots into that known-good state.”
   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”

   proof ↗⚑ flag

   Slate—none0/10

   Docs describe session switching (/sessions), long multi-hour session support, and diagnostic context attachment, but there is no evidence of automatic cross-session memory building or recall — sessions appear to be manually selected/switched contexts, not an automatic memory system. Missing for higher verdict: any documentation of persistent memory storage, automatic recall of past codebase context, or memory summarization across sessions.

   • [claimed-docs] “Slate works with you across long, multi-hour sessions.”
   • [claimed-docs] “Use `/sessions` to switch between existing sessions”
   • [claimed-docs] “Slate automatically attaches relevant diagnostic information (OS, version, session context) to your report.”

   proof ↗⚑ flag

2. developerInclude multiple project directories in a single session for broader context

   weight 2 · round to Slate
   Devin—none0/10

   No evidence describes attaching or indexing multiple project directories/repositories within a single Devin session; docs discuss single-repo indexing, knowledge, and environment setup but not multi-directory context in one session.

   ⚑ flag

   Slate~partialclaimed6/10

   Docs mention a `/workspace` manager for reviewing and removing 'workspace directories' (plural), implying support for multiple project directories in one session, but there's no detailed documentation on how directories are added or how context is merged across them, and no independent/hands-on confirmation. Missing for 10: explicit instructions/examples for adding multiple directories, and independent verification that broader multi-directory context actually works in practice.

   • [claimed-docs] “Use `/workspace` to open the workspace manager, where you can review and remove workspace directories.”

   proof ↗⚑ flag

3. developerAdd a project instructions file to set coding standards and conventions the agent follows

   weight 3 · round to Devin
   Devin✓fullclaimed9/10

   Devin explicitly supports AGENTS.md, an open standard for providing context and instructions to AI agents, plus a separate 'Knowledge' feature for instructions/advice referenced across all sessions, directly covering project-level coding standards/conventions. Missing for 10: independent/hands-on confirmation that these instructions are reliably followed in practice.

   • [claimed-docs] “Devin supports AGENTS.md - a simple, open standard for providing context and instructions to AI agents.”
   • [claimed-docs] “Knowledge is a collection of instructions and advice that Devin can reference in all sessions.”

   proof ↗⚑ flag

   Slate~partialclaimed6/10

   Docs confirm Slate 'respects agent rules' in a defined precedence order and supports Skills (markdown instruction packages, including Claude Code-compatible `.claude/skills/` paths), which cover project-level conventions/instructions, but there's no explicit example of a single top-level 'instructions file' analogous to AGENTS.md/CLAUDE.md being demonstrated end-to-end. missing for 10: explicit naming/format of the project instructions file, a worked example showing the agent following custom conventions from it, and independent/community confirmation it works as documented.

   • [claimed-docs] “Slate by default respects agent rules in the following order”
   • [claimed-docs] “Skills are markdown instruction packages that give the agent domain-specific knowledge and behavior.”
   • [claimed-docs] “`.claude/skills/` | Claude Code compatibility”

   proof ↗⚑ flag

Issue diagnosis

1. developerReproduce issues, narrow down root causes, and verify fixes

   weight 3 · round to Devin
   Devin~partialcommunity6/10

   Docs explicitly claim Devin can 'repro and fix bugs' (devin-docs-1), use Ask Devin/DeepWiki to narrow root causes via codebase understanding (devin-docs-15, devin-docs-16, devin-docs-27), and verify fixes through CI iteration/Auto-Fix loops (devin-docs-18). However, hands-on community testimony reports Devin often adds extraneous changes beyond the task scope and cannot reliably undo them when asked, undermining clean verification of fixes (devin-comm-1), and lacks an escape hatch when stuck on root-cause diagnosis (devin-comm-2, devin-comm-3). Missing for 10: independent verification of successful bug reproduction/root-cause narrowing at scale, and resolution of the reported inability to cleanly revert unwanted changes during fix verification.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Use DeepWiki to navigate architecture and code with auto-generated documentation.”
   • [claimed-docs] “Ask Devin can answer questions about code structure and dependencies, and help you scope and plan tasks before implementation.”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “One thing that surprised me is there doesn't seem to be an 'ask for help' escape hatch in Devin - it would work away for literally days on a…”
   • [community] “Devin does ask for help when it can't do something, but it really hates asking for help if it's a skill issue - it would prefer running in c…”

   proof ↗⚑ flag

   Slate~partialclaimed4/10

   Slate documents shell execution (`!`), file references, and being 'one of the few agents capable of performing integration tests manually,' which are plausible building blocks for debugging workflows, but there's no explicit documentation of a reproduce→diagnose→verify-fix workflow. Missing for 10: explicit debugging/root-cause-analysis workflow documentation, evidence of test-driven verification loops, and independent hands-on confirmation that Slate helps developers actually reproduce and fix bugs.

   • [claimed-docs] “Slate is one of the few agents capable of performing integration tests manually.”
   • [claimed-docs] “Execute shell commands directly with `!`”
   • [claimed-docs] “Use `@filename` references”
   • [claimed-docs] “While one or more agents run in the background, you can keep talking with Slate: plan next steps, queue up additional tasks, or spin up more…”

   proof ↗⚑ flag

Marketplace

1. developerEquip the agent with custom skills to perform specialized tasks

   weight 1 · round to Slate
   Devin~partialclaimed6/10

   Devin supports several skill-like extension mechanisms — 'Knowledge' (persistent instructions/advice for all sessions), AGENTS.md for structured agent instructions, and 'Playbooks' exposed via MCP — plus an open-source 'Devin Handoff' explicitly described as a 'plugin and skill'. This gives developers real levers to encode specialized, reusable task behavior, but the docs don't show a dedicated skill-authoring UI/marketplace or detailed examples of building a complex custom skill, and there is no independent/community evidence confirming this works well in practice. Missing for 10: concrete examples/tutorials of authoring a non-trivial custom skill, a discoverable skills registry, and independent hands-on corroboration.

   • [claimed-docs] “Knowledge is a collection of instructions and advice that Devin can reference in all sessions.”
   • [claimed-docs] “Devin supports AGENTS.md - a simple, open standard for providing context and instructions to AI agents.”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “Devin Handoff is an open-source plugin and skill that brings the same handoff workflow to any coding agent — Claude Code, Codex, Cursor, and…”

   proof ↗⚑ flag

   Slate✓fullclaimed7/10

   Slate has a documented Skills system: markdown instruction packages that give the agent domain-specific knowledge/behavior, with example skill definitions and compatibility with Claude Code's `.claude/skills/` format, letting developers equip the agent with custom specialized capabilities. Missing for 10: independent/hands-on verification that custom skills work as documented, and more detail on skill authoring/discovery beyond the single example.

   • [claimed-docs] “Skills are markdown instruction packages that give the agent domain-specific knowledge and behavior.”
   • [claimed-docs] “description: "Create distinctive, production-grade frontend interfaces."”
   • [claimed-docs] “`.claude/skills/` | Claude Code compatibility”

   proof ↗⚑ flag

2. engineering-leadIntegrate third-party partner-built agent apps into my workflows

   weight 1 · round to Devin
   Devin~partialprobed4/10

   Devin exposes an MCP server so external MCP-compatible agents/IDEs can access its session management and tools, and its open-source 'Devin Handoff' plugin interoperates with other coding agents (Claude Code, Codex, Cursor), showing some cross-agent workflow integration. However there is no evidence of a partner/marketplace ecosystem of third-party agent apps being integrated into Devin's own workflows. Missing for 10: a documented partner-app marketplace or catalog, evidence of installing/configuring third-party agent apps within Devin, and any case study of an engineering-lead orchestrating partner-built agents through Devin.

   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “Devin Handoff is an open-source plugin and skill that brings the same handoff workflow to any coding agent — Claude Code, Codex, Cursor, and…”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [probe] “official MCP server documented at https://docs.devin.ai/work-with-devin/devin-mcp”

   proof ↗⚑ flag

   Slate—none0/10

   Slate is a coding CLI agent focused on subagents, skills, sessions, and model orchestration; there is no evidence of an ecosystem for integrating third-party partner-built agent apps (e.g., a marketplace, app store, or partner integration framework). Skills compatibility with Claude Code is mentioned but that is file-format compatibility, not partner app integration.

   • [claimed-docs] “Skills are markdown instruction packages that give the agent domain-specific knowledge and behavior.”
   • [claimed-docs] “`.claude/skills/` | Claude Code compatibility”

   proof ↗⚑ flag

Team knowledge

1. engineering-leadCreate a shared workspace from my docs and repos as a common source of truth for the team

   weight 1 · round to Devin
   Devin~partialclaimed6/10

   Devin offers building blocks for a team-wide source of truth — repo indexing that powers 'Ask Devin' and DeepWiki architecture docs, org-wide 'Knowledge' referenced in all sessions, and AGENTS.md support for shared context — which collectively let a lead centralize docs/repo knowledge for the team. However, there's no explicit product feature framed as a 'shared workspace' UI for team-wide docs/repo browsing outside of Devin's own agent sessions. Missing for 10: a dedicated shared workspace/knowledge-base product surface for humans to browse, independent evidence of teams using it as a collaborative source of truth, and clarity on cross-repo doc aggregation beyond per-session knowledge.

   • [claimed-docs] “Indexing your repositories allows Devin to understand your codebase and enables powerful features like Ask Devin and DeepWiki”
   • [claimed-docs] “Knowledge is a collection of instructions and advice that Devin can reference in all sessions.”
   • [claimed-docs] “Devin supports AGENTS.md - a simple, open standard for providing context and instructions to AI agents.”
   • [claimed-docs] “Use DeepWiki to navigate architecture and code with auto-generated documentation.”
   • [claimed-docs] “Devin inspects your repository, figures out which tools, runtimes, and dependencies are needed, and generates the blueprint for you.”

   proof ↗⚑ flag

   Slate—none0/10

   Slate is a CLI coding agent focused on individual sessions, workspaces (local directories), skills, and orchestration of subagents—there is no evidence of a shared team workspace or collaborative source-of-truth feature built from docs and repos. The 'workspace' concept here refers to local directory management (/workspace), not a shared team hub.

   • [claimed-docs] “Use `/workspace` to open the workspace manager, where you can review and remove workspace directories.”
   • [claimed-docs] “Slate by default respects agent rules in the following order”
   • [claimed-docs] “Skills are markdown instruction packages that give the agent domain-specific knowledge and behavior.”

   proof ↗⚑ flag

Tool integration

1. developerConnect the agent to workflow tools like Jira, Slack, and Google Drive to extend its context

   weight 3 · round to Devin
   Devin~partialclaimed6/10

   Devin can be tagged in Slack/Teams threads and work Jira/Linear tickets, giving it direct workflow-tool integration, and its MCP server plus API enable further extension to other tools. However there is no explicit documented Google Drive integration, and no independent/hands-on corroboration of these integrations actually working in practice. missing for 10: Google Drive connector evidence, independent verification of Jira/Slack integration reliability.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Tagging Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”

   proof ↗⚑ flag

   Slate—none0/10

   No evidence anywhere in the docs pack mentions integrations with Jira, Slack, Google Drive, or any workflow tools/MCP connectors; the docs focus on CLI usage, sessions, skills, and permissions with no mention of external tool connectivity.

   ⚑ flag

2. developerKick off agent tasks directly from GitHub, GitLab, Linear, or Slack

   weight 2 · round to Devin
   Devin~partialclaimed6/10

   Docs confirm Devin can be invoked from Linear/Jira tickets and Slack/Teams threads, plus API integration for building custom workflow triggers, but there's no explicit mention of GitHub or GitLab issue/PR-based task kickoff in the evidence pack. Community evidence doesn't directly contradict the integration claims, only general effectiveness concerns. Missing for 10: explicit GitHub/GitLab-triggered task creation documentation, independent hands-on confirmation of these specific integrations working.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Tagging Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”

   proof ↗⚑ flag

   Slate—none0/10

   No evidence of any integration with GitHub, GitLab, Linear, or Slack for triggering agent tasks; Slate's documentation covers CLI usage, sessions, skills, and configuration but nothing about ecosystem/platform triggers.

   ⚑ flag

Cross device continuity

1. developerStart a task on one device and continue it later from another device or browser

   weight 2 · round to Devin
   Devin✓fullclaimed7/10

   Devin's cloud sessions run in a dedicated VM that persists independent of the local device (docs-13), can be started from Slack/Teams, IDE, CLI, or API and continued/taken over in the embedded IDE or web UI (docs-3, docs-4, docs-7, docs-24), and Devin Handoff explicitly lets you start work locally and continue in a cloud session accessible from a browser (docs-38). missing for 10: no explicit first-party walkthrough of resuming the same session from a different browser/device login, and no independent/community confirmation of cross-device continuity.

   • [claimed-docs] “Devin is designed to be a conversational user interface, and allows you to follow and take over Devin's development process in the embedded …”
   • [claimed-docs] “Devin CLI, a local command-line coding agent with deep Devin Cloud integration.”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “Tag Devin on a Slack or Teams thread about a bug you're discussing with coworkers”
   • [claimed-docs] “Hand a task off to a cloud Devin session and keep working locally.”

   proof ↗⚑ flag

   Slate—none0/10

   Docs show session management within Slate (e.g. `/sessions` to switch sessions, `Ctrl+X N` for new session) but only describe local session switching, not any cloud sync or cross-device/browser continuation mechanism. Slate appears to be a terminal-only CLI tool with no mention of a browser interface or account-based sync for resuming tasks elsewhere.

   • [claimed-docs] “Use `/sessions` to switch between existing sessions”
   • [claimed-docs] “Ctrl+X then N New session”
   • [claimed-docs] “npm i -g @randomlabs/slate”

   proof ↗⚑ flag

Ide integration

1. developerChat with the coding assistant directly inside my IDE for contextual help

   weight 3 · round to Slate
   Devin~partialclaimed5/10

   Devin exposes a conversational interface via its own embedded IDE within cloud sessions (devin-docs-3) and its Desktop app that imports VS Code/Cursor settings (devin-docs-9), plus an MCP server letting 'any MCP-compatible AI agent or IDE' access sessions (devin-docs-6/23). However there's no evidence of a native extension that lets a developer chat with Devin directly inside their own existing IDE (e.g., a VS Code/JetBrains plugin) — Devin's model is its own IDE/Desktop environment or MCP bridging rather than embedding in the user's IDE. Missing for 10: a first-party IDE extension for VS Code/JetBrains enabling in-IDE chat, and independent confirmation of this workflow working well in practice.

   • [claimed-docs] “Devin is designed to be a conversational user interface, and allows you to follow and take over Devin's development process in the embedded …”
   • [claimed-docs] “Import VS Code or Cursor settings, configure themes, and start coding with AI-powered assistance.”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”

   proof ↗⚑ flag

   Slate~partialclaimed6/10

   Slate is documented as a terminal-based coding agent with session management, `@filename` references, shell execution, and workspace context — providing contextual chat help that developers can run alongside their editor in a terminal. However, there is no evidence of a native IDE extension/panel (e.g., VS Code/JetBrains plugin) that embeds Slate directly inside the IDE UI itself. missing for 10: dedicated IDE extension/panel integration, evidence of in-editor chat UI beyond terminal, independent corroboration of IDE workflow usage.

   • [claimed-docs] “Use `/sessions` to switch between existing sessions”
   • [claimed-docs] “Execute shell commands directly with `!`”
   • [claimed-docs] “Use `@filename` references”
   • [claimed-docs] “Use `/workspace` to open the workspace manager, where you can review and remove workspace directories.”
   • [claimed-docs] “Ctrl+X then N New session”

   proof ↗⚑ flag

Session management

1. developerReview diffs visually and run multiple sessions side by side in a desktop app

   weight 2 · round to Devin
   Devin~partialclaimed4/10

   Devin has a documented Desktop app (docs-9) and supports running multiple independent sessions in parallel (docs-25), but there is no evidence of a visual diff review feature or explicit side-by-side session UI within the desktop app itself. Missing for 10: explicit documentation of an in-app diff viewer, UI showing multiple sessions simultaneously in one window, and any hands-on/community confirmation of this desktop workflow.

   • [claimed-docs] “Import VS Code or Cursor settings, configure themes, and start coding with AI-powered assistance.”
   • [claimed-docs] “Devin is designed to be a conversational user interface, and allows you to follow and take over Devin's development process in the embedded …”
   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”

   proof ↗⚑ flag

   Slate—none0/10

   Slate is documented as a terminal/CLI tool (npm install, terminal-background onboarding, hotkeys, `/sessions` switching, subagent grid) with no mention of a desktop GUI or visual diff review; session switching is terminal-based, not side-by-side desktop windows. Missing for 10: any evidence of a desktop application, a visual diff viewer, or GUI-based side-by-side session comparison.

   • [claimed-docs] “npm i -g @randomlabs/slate”
   • [claimed-docs] “Onboarding asks for your terminal background, multiline input preference, and model source: your ChatGPT/Codex subscription, SuperGrok subsc…”
   • [claimed-docs] “Use `/sessions` to switch between existing sessions”
   • [claimed-docs] “Those agents show up as a grid of inline subagent cards, one per agent.”
   • [claimed-docs] “Ctrl+X then N New session”

   proof ↗⚑ flag

2. engineering-leadManage multiple agent-driven coding sessions from one unified workspace

   weight 2 · round to Slate
   Devin~partialcommunity6/10

   Devin supports running multiple parallel sessions ('carve out independent tasks and run them simultaneously'), an API to create sessions on behalf of users, and an embedded IDE/CLI/desktop app to interact with sessions, which together enable a lead-like workspace for managing several agent sessions. However, there's no dedicated 'unified workspace' dashboard evidence for an engineering-lead specifically monitoring/managing a team's multiple concurrent sessions, and community feedback highlights session reliability issues that would complicate multi-session oversight. missing for 10: explicit multi-session dashboard/UI for a lead role, team-level session oversight features, independent corroboration of smooth multi-session management at scale.

   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “Devin is designed to be a conversational user interface, and allows you to follow and take over Devin's development process in the embedded …”
   • [claimed-docs] “Devin CLI, a local command-line coding agent with deep Devin Cloud integration.”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”
   • [community] “I've learned to just click 'Terminate Session' immediately after spotting Devin doing something hopeless. I've managed to get real work done…”

   proof ↗⚑ flag

   Slate✓fullclaimed8/10

   Docs describe first-class multi-session/multi-agent workspace features: `/sessions` to switch sessions, `/workspace` manager, new-session hotkey, and orchestration showing a grid of inline subagent cards while continuing to chat, queue tasks, or spin up more parallel agents — directly matching the engineering-lead's need to manage multiple concurrent agent sessions from one place. Missing for 10: independent/hands-on verification of this workspace at scale and any lead-specific team-management features beyond individual session switching.

   • [claimed-docs] “Use `/sessions` to switch between existing sessions”
   • [claimed-docs] “Use `/workspace` to open the workspace manager, where you can review and remove workspace directories.”
   • [claimed-docs] “Those agents show up as a grid of inline subagent cards, one per agent.”
   • [claimed-docs] “While one or more agents run in the background, you can keep talking with Slate: plan next steps, queue up additional tasks, or spin up more…”
   • [claimed-docs] “Ctrl+X then N New session”

   proof ↗⚑ flag

Terminal workflow

1. developerRun a coding agent locally from my terminal

   weight 3 · round drawn
   Devin✓fullprobed8/10

   Devin CLI is explicitly documented as a local command-line coding agent that can be invoked from terminal (e.g. `devin -- check out this code...`), with local sandboxing (--sandbox flag) and deep integration with Devin Cloud for handoff. Missing for 10: independent/hands-on community verification specifically of the CLI experience (community evidence only covers the cloud/browser Devin product, not the local CLI).

   • [claimed-docs] “Devin CLI, a local command-line coding agent with deep Devin Cloud integration.”
   • [claimed-docs] “devin -- check out this code and suggest a feasible, helpful feature”
   • [claimed-docs] “a local command-line coding agent with deep Devin Cloud integration”
   • [claimed-docs] “The --sandbox flag runs the CLI with OS-level isolation, enforcing writable paths and deny rules at the operating-system level and optionall…”
   • [claimed-docs] “The `--sandbox` flag runs the CLI with OS-level isolation, enforcing writable paths and `deny` rules at the operating-system level”
   • [probe] “official CLI documented at https://docs.devin.ai/cli/index”

   proof ↗⚑ flag

   Slate✓fullprobed8/10

   Slate ships as a global npm CLI (`npm i -g @randomlabs/slate`) that runs interactively in the terminal, with documented terminal-native features like hotkeys, shell command execution (`!`), file references (`@filename`), session management (`/sessions`), and configuration via `slate.json` — all consistent with a locally-run terminal coding agent. Missing for 10: independent hands-on confirmation of local terminal usage (community evidence only discusses porting-quality skepticism, not terminal operation itself) and no evidence of offline/non-terminal fallback limitations.

   • [claimed-docs] “npm i -g @randomlabs/slate”
   • [claimed-docs] “Onboarding asks for your terminal background, multiline input preference, and model source: your ChatGPT/Codex subscription, SuperGrok subsc…”
   • [claimed-docs] “Use `/sessions` to switch between existing sessions”
   • [claimed-docs] “Press Tab to queue the current message so it runs after the current turn finishes.”
   • [claimed-docs] “Execute shell commands directly with `!`”
   • [claimed-docs] “Use `@filename` references”
   • [claimed-docs] “Ctrl+X then N New session”
   • [probe] “official CLI documented at https://docs.randomlabs.ai/en/getting-started/quickstart”

   proof ↗⚑ flag

2. developerRun the agent non-interactively in scripts for workflow automation

   weight 2 · round to Devin
   Devin✓fullclaimed7/10

   Devin exposes a documented API for creating sessions programmatically (including on behalf of users) and explicit CI/CD pipeline integration to auto-respond to static-analysis findings, plus a CLI (`devin -- <prompt>`) that can be invoked headlessly, all pointing to non-interactive, scriptable automation. Missing for 10: independent/hands-on confirmation of headless CLI scripting in real CI pipelines and more detail on CLI exit codes/output for scripting.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Devin CLI, a local command-line coding agent with deep Devin Cloud integration.”
   • [claimed-docs] “devin -- check out this code and suggest a feasible, helpful feature”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “a local command-line coding agent with deep Devin Cloud integration”

   proof ↗⚑ flag

   Slate—none0/10

   The evidence shows Slate is a CLI-based interactive agent (npm install, onboarding, in-session commands like /sessions, !, @filename) but nowhere documents a non-interactive/headless mode, flags for scripted execution, or CI/automation usage; --dangerously-skip-permissions bypasses prompts but is not shown as enabling scripted/non-interactive invocation. Missing for 10: documentation of a non-interactive/print/exec mode, exit-code or piping behavior, or any CI/scripting examples.

   • [claimed-docs] “npm i -g @randomlabs/slate”
   • [claimed-docs] “We support `--dangerously-skip-permissions` (alias: `--yolo`) to bypass permission prompts.”
   • [probe] “official CLI documented at https://docs.randomlabs.ai/en/getting-started/quickstart”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Devin
   Devin~partialprobed5/10

   Devin exposes a documented API (session creation, create_as_user_id) and an MCP server giving 'full access to session management, playbooks, knowledge, and scheduling', suggesting broad programmatic parity, but there's no explicit claim of full UI/API feature parity, and probes for an OpenAPI spec all 404'd, indicating the full API surface isn't transparently documented. Missing for 10: explicit parity statement covering UI-only features like Devin Review/Auto-Fix/Computer Use/desktop app settings, a discoverable OpenAPI schema, and independent confirmation that all UI actions are API-reachable.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [claimed-docs] “it gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [claimed-docs] “gives any MCP-compatible AI agent or IDE full access to session management, playbooks, knowledge, and scheduling”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

   Slate—none0/10

   No evidence of any public API for Slate — the openapi.json/swagger.json probes returned 404s and no docs reference programmatic endpoints; Slate is documented purely as a CLI/terminal agent with slash-commands, hotkeys, and config files, not an API-driven product with UI/API parity.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.randomlabs.ai/openapi.json, https://docs.randomlabs.ai/swagger.json, https://docs.rando…”
   • [probe] “official CLI documented at https://docs.randomlabs.ai/en/getting-started/quickstart”
   • [claimed-docs] “npm i -g @randomlabs/slate”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round drawn
   Devin—none0/10

   No evidence of a data export feature or open-format export/portability mechanism for Devin sessions, knowledge, or artifacts; the docs cover API, CLI, MCP, and infrastructure but nothing about exporting all user data in open formats to leave the platform.

   ⚑ flag

   Slate—none0/10

   No evidence in the docs or elsewhere describes any data export functionality, open-format export, or data portability mechanism for Slate. Sessions, workspace history, and configurations appear stored locally but no documented export/leave path is mentioned. Missing for 10: any documentation of export commands, data format specifications, or account/data portability guarantees.

   ⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round drawn
   Devin—none0/10

   Devin is a closed, proprietary commercial product; no evidence indicates its source code is available under an open license. The only mention of 'open-source' is for the separate Devin Handoff plugin/skill, not Devin itself.

   • [claimed-docs] “Devin Handoff is an open-source plugin and skill that brings the same handoff workflow to any coding agent — Claude Code, Codex, Cursor, and…”

   proof ↗⚑ flag

   Slate—none0/10

   No evidence of an open-source license or public source repository for Slate; the CLI is distributed via npm install with no mention of source availability. missing for 10: open-source license declaration, public source repo link, license file/OSS registry evidence.

   ⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round drawn
   Devin—none0/10

   Devin is a cloud-based SaaS agent; Outposts lets you run sessions on your own infrastructure but the core Devin model/orchestration itself remains Cognition-hosted, and there's no evidence of a self-hostable core product/model package. No mention of on-prem/self-hosted deployment of the core Devin engine.

   • [claimed-docs] “Outposts lets you run Devin sessions inside infrastructure you control — your own VMs, containers, Kubernetes clusters, or even a Mac Mini o…”

   proof ↗⚑ flag

   Slate—none0/10

   No evidence anywhere in the docs of Slate being open-source or offering a self-hosted deployment option; it's installed via npm as a CLI that connects to model subscriptions/credits, implying a hosted/service model rather than self-hostable core infrastructure. Missing for 10: any mention of self-hosting instructions, open-source repo, or on-prem deployment option.

   • [claimed-docs] “npm i -g @randomlabs/slate”
   • [claimed-docs] “Onboarding asks for your terminal background, multiline input preference, and model source: your ChatGPT/Codex subscription, SuperGrok subsc…”

   proof ↗⚑ flag

Authentication

1. developerAuthenticate with an API key instead of an account login

   weight 2 · round to Devin
   Devin~partialprobed6/10

   Devin has a documented API (devin-docs-7) intended for programmatic integration, implying API key auth as an alternative to account login, and even supports creating sessions on behalf of other users (devin-docs-8), suggesting a service-level credential model. However, no explicit documentation of API key generation/management or authentication mechanics is present in the evidence pack. Missing for 10: explicit API key creation/management docs, confirmation that API key auth fully replaces login flows, and independent/community verification of this workflow.

   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”
   • [claimed-docs] “you can create sessions on behalf of any user in your organization using the create_as_user_id parameter”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.devin.ai/openapi.json, https://docs.devin.ai/swagger.json, https://docs.devin.ai/api/op…”

   proof ↗⚑ flag

   Slate—none0/10

   No evidence pack mentions API key authentication as an alternative to account login; onboarding docs only describe choosing a model source (ChatGPT/Codex, SuperGrok, or Slate credits subscription), not API-key auth. No mention of an API key mechanism anywhere, and the openapi probe returned 404s, giving no indication of an API-key based auth path.

   • [claimed-docs] “Onboarding asks for your terminal background, multiline input preference, and model source: your ChatGPT/Codex subscription, SuperGrok subsc…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.randomlabs.ai/openapi.json, https://docs.randomlabs.ai/swagger.json, https://docs.rando…”

   proof ↗⚑ flag

2. engineering-leadAuthenticate through an enterprise identity or cloud platform for compliance and scalability

   weight 2 · round drawn
   Devin—none0/10

   No evidence of SSO/SAML, OIDC, or enterprise identity provider integration (e.g., Okta, Azure AD, Google Workspace) for authentication; the only related item is API session creation on behalf of a user, which is not enterprise identity/SSO authentication.

   ⚑ flag

   Slate—none0/10

   No evidence of SSO/SAML/OIDC enterprise identity integration or cloud-platform authentication for compliance; onboarding only mentions choosing a model source (ChatGPT/Codex, SuperGrok, or Slate credits), not enterprise identity federation.

   • [claimed-docs] “Onboarding asks for your terminal background, multiline input preference, and model source: your ChatGPT/Codex subscription, SuperGrok subsc…”

   proof ↗⚑ flag

3. developerSign in with my existing product subscription plan to use the coding agent

   weight 2 · round to Slate
   Devin—none0/10

   The evidence describes Devin's own subscription tiers (Pro, Max, Core/Team/Enterprise) but nothing about letting a developer sign in using an existing external subscription (e.g., an existing LLM provider or IDE subscription) to access the agent. No mention of SSO-linked billing or bring-your-own-subscription support.

   • [claimed-docs] “Max is for individual users who consistently exceed the Pro quota. It includes everything in Pro, plus a significantly larger weekly usage q…”
   • [claimed-docs] “Power users who need more quota”

   proof ↗⚑ flag

   Slate✓fullclaimed8/10

   Docs explicitly state onboarding lets you choose your model source as your existing ChatGPT/Codex subscription or SuperGrok subscription (in addition to Slate credits), directly matching the story of signing in with an existing subscription plan to use the agent. Missing for 10: independent/hands-on confirmation that subscription sign-in actually works end-to-end and any detail on limitations of that mode vs credits.

   • [claimed-docs] “Onboarding asks for your terminal background, multiline input preference, and model source: your ChatGPT/Codex subscription, SuperGrok subsc…”

   proof ↗⚑ flag

4. developerSign in with a personal account to get free-tier access without managing API keys

   weight 1 · round to Slate
   Devin—none0/10

   Evidence only describes paid tiers (Pro, Max, 'power users who need more quota') and API key based integration; there's no mention of a free tier accessible via personal account sign-in without API key management.

   • [claimed-docs] “Max is for individual users who consistently exceed the Pro quota. It includes everything in Pro, plus a significantly larger weekly usage q…”
   • [claimed-docs] “Power users who need more quota”
   • [claimed-docs] “The Devin API enables you to integrate Devin into your applications, automate workflows, and build powerful tools.”

   proof ↗⚑ flag

   Slate~partialclaimed4/10

   Docs show onboarding lets users choose a model source including an existing ChatGPT/Codex or SuperGrok subscription instead of managing API keys, implying account-based auth is supported, but there's no explicit mention of a free tier or of signing in with a personal Slate account for free credits without a paid subscription. Missing for 10: explicit free-tier account sign-in flow, confirmation that 'Slate credits' option requires no payment, and any account-based (not subscription-based) login mechanism.

   • [claimed-docs] “Onboarding asks for your terminal background, multiline input preference, and model source: your ChatGPT/Codex subscription, SuperGrok subsc…”

   proof ↗⚑ flag

Model choice

1. developerLet the tool automatically pick the best model for each task

   weight 1 · round to Slate
   Devin—none0/10

   No evidence that Devin offers automatic model selection or model-routing based on task complexity; docs discuss quotas, tiers (Pro/Max), and features but never mention choosing among multiple underlying models per task.

   ⚑ flag

   Slate~partialclaimed6/10

   Docs explicitly claim Slate 'automatically selects the right model for the job' and also allow developers to set preferred default models per slot via `/models` or `slate.json`, suggesting a hybrid automatic+manual approach relevant to pricing/limits tradeoffs. However, there's no detail on the selection logic, cost-awareness, or independent verification that auto-selection actually optimizes for task/price. Missing for 10: independent hands-on confirmation of auto-selection quality, explanation of selection criteria (cost vs capability), and evidence of pricing-limit awareness in model choice.

   • [claimed-docs] “Slate automatically selects the right model for the job.”
   • [claimed-docs] “Set preferred default models for each slot with the `/models` dialog or `slate.json` under `models`.”

   proof ↗⚑ flag

2. developerChoose which underlying AI model powers my session from multiple providers

   weight 2 · round to Slate
   Devin—none0/10

   No evidence in the pack mentions selecting or switching between multiple underlying AI model providers (e.g., choosing between different LLMs) for a Devin session; documentation covers agent capabilities, environments, and billing tiers but not model selection.

   ⚑ flag

   Slate~partialclaimed6/10

   Docs confirm model source can be chosen at onboarding (ChatGPT/Codex, SuperGrok, or Slate credits) and that default models per 'slot' can be set via `/models` or slate.json, showing multi-provider flexibility. However, this is framed around subscription/credit sources rather than a clear list of many independent model providers, and there's no independent/hands-on verification of switching providers mid-session. missing for 10: independent corroboration of provider switching, a full list of supported model providers, and confirmation this works reliably in practice.

   • [claimed-docs] “Onboarding asks for your terminal background, multiline input preference, and model source: your ChatGPT/Codex subscription, SuperGrok subsc…”
   • [claimed-docs] “Set preferred default models for each slot with the `/models` dialog or `slate.json` under `models`.”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round drawn
   Devin—none0/10

   No evidence of data residency/region selection controls for where Devin stores data; the pack covers deployment options (Outposts, VPN, sandbox isolation) but nothing about choosing storage region/jurisdiction.

   ⚑ flag

   Slate—none0/10

   Slate is a CLI coding agent tool; the evidence pack contains no mention of data residency, region selection, or storage location controls. Missing for 10: any documentation of data residency options, regional storage configuration, or compliance controls.

   ⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   Devin—none0/10

   The evidence pack contains no mention of data-training opt-out, data usage policy, or privacy controls for AI model training; nothing in the docs or community sources addresses this axis.

   ⚑ flag

   Slate—none0/10

   No evidence in the pack addresses data-training opt-out, privacy policy, or any control over model training use; the documentation covers CLI usage, orchestration, and skills but nothing about data privacy posture. Missing for 10: any privacy policy statement, opt-out settings, or data usage terms regarding AI training.

   ⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round drawn
   Devin—none0/10

   No evidence pack items address data retention policies, data deletion controls, or privacy/data-handling settings for user data or session data; the closest items concern sandboxing/isolation of execution environments, not retention/deletion of stored data.

   ⚑ flag

   Slate—none0/10

   No evidence pack items mention data retention policies, deletion controls, or privacy settings for user data/sessions; docs cover workspace management and permissions but not data retention/deletion. Missing for 10: any documentation of data retention periods, deletion mechanisms, or export/erase controls.

   ⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Devin—none0/10

   No evidence pack item mentions telemetry opt-out, usage tracking settings, or privacy controls of that kind; the docs cover features, sandboxing, and infrastructure but not data-collection opt-out mechanisms.

   ⚑ flag

   Slate—none0/10

   No evidence pack item mentions telemetry, usage tracking, analytics, or an opt-out setting anywhere in Slate's docs or community coverage; the closest item (diagnostic attachment on bug reports) doesn't address general telemetry opt-out. Missing for 10: any mention of telemetry collection, a privacy policy, or a documented opt-out flag/setting.

   • [claimed-docs] “Slate automatically attaches relevant diagnostic information (OS, version, session context) to your report.”

   proof ↗⚑ flag

Data governance

1. engineering-leadOpt out of having my code and prompts used for AI model training

   weight 1 · round drawn
   Devin—none0/10

   No evidence in the pack addresses data usage, training opt-out, or privacy/data-handling policies for Devin; nothing documents an opt-out mechanism for code/prompt use in model training.

   ⚑ flag

   Slate—none0/10

   No evidence in the pack addresses data usage, training opt-out, or privacy policy for prompts/code submitted to Slate or its model providers.

   ⚑ flag

Pr review

1. developerHave the agent stage changes, write commit messages, create branches, and open pull requests

   weight 3 · round to Devin
   Devin~partialcommunity6/10

   Devin's docs describe it implementing features/fixing bugs and producing pull requests that get automated review and iteration (devin-docs-17, devin-docs-18), implying it handles the full git workflow (branch, commit, PR) autonomously, but no doc explicitly details staging, commit-message generation, or branch creation as discrete steps. Community feedback (devin-comm-1) also notes it can add extraneous changes it can't cleanly undo, a real caveat on commit hygiene. Missing for 10: explicit documentation of commit/staging/branch mechanics and independent confirmation that generated commits/PRs are clean and reviewable.

   • [claimed-docs] “Ask Devin to tackle Linear/Jira tickets, implement entirely new features, repro and fix bugs, build internal tools, and more!”
   • [claimed-docs] “Devin Review provides automated first-pass reviews on pull requests, checking for correctness and conformance with organizational best pract…”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Carve out independent tasks and run them simultaneously.”
   • [community] “Trialed Devin, it's quite impressive when it understands code formatting and local test setup, but it always adds extraneous changes beyond …”

   proof ↗⚑ flag

   Slate—none0/10

   No evidence in the pack mentions git operations like staging, committing, branching, or opening pull requests; documentation covers sessions, orchestration, skills, permissions, and CLI setup but not any git/PR workflow. Absence of evidence for this applicable capability means the verdict is none.

   ⚑ flag

2. developerInspect diffs and run checks to catch problems before merging

   weight 3 · round to Devin
   Devin✓fullclaimed8/10

   Devin's docs describe Devin Review, which performs automated first-pass PR reviews checking correctness and org conformance, plus CI/CD integration to respond to static-analysis findings (SonarQube, Fortify, Veracode) and Auto-Fix iterating on CI failures — directly enabling developers to inspect diffs and run checks before merge. This is corroborated by explicit SDLC integration workflow docs, not just a single mention. missing for 10: independent/hands-on evidence confirming Devin Review's diff-inspection quality in practice, and detail on how diffs are surfaced/inspected by the developer (UI specifics) beyond docs claims.

   • [claimed-docs] “Devin Review provides automated first-pass reviews on pull requests, checking for correctness and conformance with organizational best pract…”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Enable Devin Review with Auto-Fix so Devin automatically responds to code review comments, fixe”

   proof ↗⚑ flag

   Slate~partialclaimed3/10

   Slate's docs mention it can perform integration tests manually (random-labs-docs-16), implying some check-running capability, but there is no evidence of diff inspection, git diff review, PR-style change summaries, or pre-merge validation workflows. missing for 10: diff/change inspection UI or command, explicit pre-merge check/test running workflow, and any corroborating hands-on evidence of catching problems before merge.

   • [claimed-docs] “Slate is one of the few agents capable of performing integration tests manually.”

   proof ↗⚑ flag

Safe execution

1. engineering-leadControl which external tools and integrations the agent is allowed to access

   weight 2 · round drawn
   Devin~partialclaimed6/10

   Devin exposes some admin-level controls over its access—an org-wide 'Enable desktop mode' toggle for Computer Use, a CLI --sandbox flag enforcing OS-level writable-path/deny rules and network restriction, and Outposts for running sessions in infra you control—giving leads levers to constrain what Devin can reach or do. However there's no documented centralized policy/allowlist for specific external integrations (e.g., disabling Slack, Jira, GitHub, VPN access per-tool) or granular permission/audit management for engineering leads. missing for 10: a unified integration-permission/allowlist admin panel, per-tool enable/disable controls beyond desktop mode, and independent verification of these controls in practice.

   • [claimed-docs] “Computer Use is controlled by the Enable desktop mode toggle in your organization's customization options.”
   • [claimed-docs] “The --sandbox flag runs the CLI with OS-level isolation, enforcing writable paths and deny rules at the operating-system level and optionall…”
   • [claimed-docs] “The `--sandbox` flag runs the CLI with OS-level isolation, enforcing writable paths and `deny` rules at the operating-system level”
   • [claimed-docs] “Outposts lets you run Devin sessions inside infrastructure you control — your own VMs, containers, Kubernetes clusters, or even a Mac Mini o…”
   • [claimed-docs] “Devin can connect to a VPN from inside its workspace, so sessions can reach internal services such as package registries, databases, and int…”

   proof ↗⚑ flag

   Slate~partialclaimed6/10

   Slate's configuration docs describe a permission system where each permission key maps to allow/ask/deny actions or fine-grained pattern objects, which supports controlling what tools/actions the agent can perform, and a `--yolo` flag exists to bypass these prompts entirely. However there's no explicit documentation of controlling specific external integrations (e.g., MCP servers, API connectors) or org/team-level lockdown for an engineering lead specifically. Missing for 10: explicit external-integration/MCP allowlist docs, engineering-lead/team-level enforcement (vs individual config), and independent verification that permission enforcement can't be trivially bypassed.

   • [claimed-docs] “Each permission key maps to an action ("allow", "ask", or "deny"), or a pattern object for fine-grained control.”
   • [claimed-docs] “We support `--dangerously-skip-permissions` (alias: `--yolo`) to bypass permission prompts.”

   proof ↗⚑ flag

2. engineering-leadHave the agent operate inside a sandbox when interacting with code, tools, and network resources

   weight 2 · round to Devin
   Devin✓fullclaimed8/10

   Devin runs cloud sessions in isolated VMs (devin-docs-13) and provides an explicit CLI --sandbox flag enforcing OS-level isolation, writable path restrictions, deny rules, and optional network restriction (devin-docs-12, devin-docs-37), directly matching the sandboxed code/tool/network isolation story. Missing for 10: independent/hands-on verification of sandbox robustness and more detail on network isolation guarantees beyond docs claims.

   • [claimed-docs] “The --sandbox flag runs the CLI with OS-level isolation, enforcing writable paths and deny rules at the operating-system level and optionall…”
   • [claimed-docs] “The `--sandbox` flag runs the CLI with OS-level isolation, enforcing writable paths and `deny` rules at the operating-system level”
   • [claimed-docs] “The cloud session gets its own VM with a shell, browser, and full repo access, so it can keep going after you close your laptop.”
   • [claimed-docs] “Devin can connect to a VPN from inside its workspace, so sessions can reach internal services such as package registries, databases, and int…”

   proof ↗⚑ flag

   Slate—none0/10

   The evidence shows a permission system (allow/ask/deny actions) and a --yolo flag to bypass prompts, but there is no mention of sandboxed execution, containerization, or network isolation for the agent's code/tool interactions. missing for 10: any documentation of sandbox/container execution, network isolation controls, or filesystem confinement mechanisms.

   • [claimed-docs] “Each permission key maps to an action ("allow", "ask", or "deny"), or a pattern object for fine-grained control.”
   • [claimed-docs] “We support `--dangerously-skip-permissions` (alias: `--yolo`) to bypass permission prompts.”

   proof ↗⚑ flag

Security checks

1. engineering-leadSee license and public-code matching references for AI-suggested code

   weight 1 · round drawn
   Devin—none0/10

   No evidence of license or public-code matching/attribution references for AI-suggested code anywhere in the docs; Devin Review covers correctness and best-practice conformance but nothing about license/provenance matching.

   ⚑ flag

   Slate—none0/10

   No evidence anywhere in the pack mentions license compliance checks, public-code/OSS matching, or provenance references for AI-suggested code; the docs cover orchestration, skills, permissions, and CLI usage but nothing about code provenance/license scanning.

   ⚑ flag

2. developerGet contextual explanations and automatic fixes for security vulnerabilities

   weight 2 · round to Devin
   Devin✓fullclaimed7/10

   Devin's docs describe Devin Review giving automated PR reviews with explanations for correctness/best-practice issues, Auto-Fix automatically responding to review comments and fixing flagged bugs/CI failures, and CI/CD integration to respond to findings from security scanners like SonarQube, Fortify, and Veracode — directly matching contextual explanation plus automatic fixing of vulnerabilities. Missing for 10: independent/hands-on evidence specifically validating security-vulnerability fixes (community evidence only discusses general reliability/scope-creep issues, not security-fix accuracy).

   • [claimed-docs] “Devin Review provides automated first-pass reviews on pull requests, checking for correctness and conformance with organizational best pract…”
   • [claimed-docs] “With Auto-Fix enabled, Devin automatically responds to code review comments, fixes flagged bugs, and iterates on CI failures — creating a cl…”
   • [claimed-docs] “Integrate Devin into your CI/CD pipeline to respond to findings from static analysis tools like SonarQube, Fortify, or Veracode.”
   • [claimed-docs] “Ask Devin can answer questions about code structure and dependencies, and help you scope and plan tasks before implementation.”

   proof ↗⚑ flag

   Slate—none0/10

   No evidence in the pack mentions security vulnerability detection, explanations, or automatic fixes; documentation covers session management, orchestration, skills, and configuration but nothing about security review or vulnerability remediation. Missing for 10: any mention of vulnerability scanning, security explanations, or auto-fix capability.

   ⚑ flag