Foreloop vs Factory

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to Factory
   Foreloop—none0/10

   No evidence pack item mentions llms.txt or an agent-oriented docs endpoint that agents can be pointed at; docs reference CLI, MCP endpoints, and skills but not an llms.txt-style discoverable doc file.

   ⚑ flag

   Factory✓fullprobed8/10

   Factory hosts an actual llms.txt at docs.factory.ai/llms.txt (HTTP 200) confirmed by direct probe, and its docs describe an agent-native platform with structured agent-oriented documentation (droid-cli, agent-readiness, missions) that an agent could be pointed at. missing for 10: independent/hands-on confirmation that an agent successfully consumes llms.txt in practice, and no explicit vendor statement encouraging users to point agents at llms.txt.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.factory.ai/llms.txt # Factory Documentation > Documentation for Factory, the agent-native software…”
   • [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
   • [claimed-docs] “Run the /readiness-report slash command in a Factory App web or desktop session, or in the Droid CLI, to evaluate a repository's readiness l…”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to Factory
   Foreloop~partialprobed6/10

   Foreloop ships a global CLI with a stable, additive `--json` output on every command, and docs explicitly state it can be run 'from anything that can run one, including coding agents and cron,' which is direct evidence of headless/CI-style automation support. However, there's no CI-specific documentation (e.g. GitHub Actions example, env-based auth for non-interactive contexts) and the approval-gate design ('nothing runs until a person approves a contract') implies a human-in-the-loop checkpoint that could block fully unattended pipelines. missing for 10: CI/pipeline-specific setup docs or examples, confirmation that approval gates can be bypassed or pre-authorized for headless runs, independent/hands-on verification of CI usage.

   • [claimed-docs] “`--json` works on any command and has a stable shape. Additive changes only.”
   • [claimed-docs] “Projects, loops, intentions, stories and tasks from a terminal — and from anything that can run one, including coding agents and cron.”
   • [claimed-docs] “--json works on any command and has a stable shape. Additive changes only.”
   • [claimed-docs] “Nothing runs until a person approves a contract. Approval is the only gate, and the gate is deliberate: an agent is about to change your rep…”
   • [probe] “official CLI documented at https://foreloop.com/install”

   proof ↗⚑ flag

   Factory✓fullprobed8/10

   Factory documents droid exec as a one-shot CLI command explicitly designed for CI/CD pipelines, shell scripts, and batch processing, with tiered autonomy controls for unattended operation. This directly satisfies headless/CI automation. Missing for 10: independent/hands-on third-party verification of CI usage and more detail on exit codes/output formats for pipeline integration.

   • [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
   • [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
   • [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
   • [probe] “official CLI documented at https://docs.factory.ai/droid-cli/quickstart”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round to Factory
   Foreloop—none0/10

   All MCP-related evidence describes Foreloop exposing its own MCP server endpoint (/api/app/public/v1/mcp) so that external MCP clients (agents) can call into it — this is the reverse of the story, which asks whether the user can plug external MCP servers into Foreloop so it can use their tools. No evidence shows Foreloop acting as an MCP client that consumes other servers' tools.

   • [claimed-docs] “Foreloop speaks MCP at /api/app/public/v1/mcp over streamable HTTP. An MCP client needs that URL and an API key and nothing else”
   • [claimed-docs] “Foreloop speaks MCP at /api/app/public/v1/mcp over streamable HTTP. An MCP client needs that URL and an API key and nothing else.”

   proof ↗⚑ flag

   Factory~partialclaimed6/10

   Docs explicitly state Droid CLI can connect MCP tools alongside Jira/Notion/Slack/Linear/PagerDuty integrations, confirming MCP client support. However, there is no detail on setup/configuration process, supported transport types, or independent/hands-on confirmation of MCP tool usage in practice. missing for 10: configuration details for adding MCP servers, examples of MCP tools being invoked, independent verification of functionality.

   • [claimed-docs] “Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems”
   • [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”

   proof ↗⚑ flag

4. ai-native userConnect an agent via an official MCP server

   weight 3 · round to Foreloop
   Foreloop✓fullclaimed8/10

   Foreloop documents a first-party MCP server endpoint (/api/app/public/v1/mcp over streamable HTTP) that any MCP client can connect to with just a URL and API key, explicitly enabling agents to claim tasks and work in repos. This is first-party documentation only, with no independent/hands-on corroboration of the MCP connection working in practice. Missing for 10: independent verification that an external MCP client successfully connects and operates against this endpoint.

   • [claimed-docs] “Foreloop speaks MCP at /api/app/public/v1/mcp over streamable HTTP. An MCP client needs that URL and an API key and nothing else”
   • [claimed-docs] “Foreloop speaks MCP at /api/app/public/v1/mcp over streamable HTTP. An MCP client needs that URL and an API key and nothing else.”
   • [claimed-docs] “You run an agent — Claude Code or Codex — in a checkout of your repository on your own machine. That agent asks Foreloop for approved tasks …”

   proof ↗⚑ flag

   Factory~partialprobed5/10

   Factory is primarily a coding agent (Droid), and factory-docs-4 shows it can connect OUT to MCP tools (client-side), which wouldn't count under the na rule — but factory-probe-3 explicitly indicates an official MCP server is documented at docs.factory.ai/harness/mcp, flipping this to the applicable server-side capability. However, the evidence pack only confirms the page's existence via a probe hit with no detail on what the server exposes or how other agents connect to it. Missing for 10: detailed documentation of the MCP server's tool surface, setup instructions for external agents, and independent confirmation of a working connection.

   • [probe] “official MCP server documented at https://docs.factory.ai/harness/mcp”
   • [claimed-docs] “Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems”

   proof ↗⚑ flag

5. ai-native userUse an official CLI

   weight 2 · round to Factory
   Foreloop✓fullprobed8/10

   Foreloop ships an official globally-installable CLI (npm i -g foreloop) that manages projects, loops, intentions, stories, and tasks, supports scripting via `--json` with a stable additive schema, and is explicitly designed to be run 'from anything that can run one, including coding agents and cron,' making it clearly AI-native/agentic-workflow friendly. missing for 10: independent/hands-on corroboration beyond vendor docs, and more detail on full command surface.

   • [claimed-docs] “npm i -g foreloop # installs globally, so the CLI works in any directory”
   • [claimed-docs] “`--json` works on any command and has a stable shape. Additive changes only.”
   • [claimed-docs] “Projects, loops, intentions, stories and tasks from a terminal — and from anything that can run one, including coding agents and cron.”
   • [claimed-docs] “--json works on any command and has a stable shape. Additive changes only.”
   • [probe] “official CLI documented at https://foreloop.com/install”

   proof ↗⚑ flag

   Factory✓fullprobed9/10

   Factory ships an official Droid CLI with documented quickstart, terminal/editor/Git integration, bash mode, one-shot exec for CI/CD, and slash-command features—clearly an official CLI for AI-native workflows. Missing for 10: independent/hands-on third-party corroboration beyond vendor docs.

   • [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
   • [claimed-docs] “Press ! to toggle bash mode and run shell commands directly without AI interpretation.”
   • [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
   • [probe] “official CLI documented at https://docs.factory.ai/droid-cli/quickstart”

   proof ↗⚑ flag

6. ai-native userDrive the product through a documented public API

   weight 3 · round to Foreloop
   Foreloop✓fullprobed8/10

   Foreloop documents a public MCP endpoint (/api/app/public/v1/mcp over streamable HTTP with just a URL and API key) plus a CLI with stable, additive `--json` output usable from any scriptable context including cron and coding agents, giving AI-native users two documented programmatic surfaces. missing for 10: no independent/third-party corroboration of the API's stability or real-world agent integration beyond first-party docs, and no formal OpenAPI/REST reference is shown alongside the MCP/CLI docs.

   • [claimed-docs] “Foreloop speaks MCP at /api/app/public/v1/mcp over streamable HTTP. An MCP client needs that URL and an API key and nothing else”
   • [claimed-docs] “Foreloop speaks MCP at /api/app/public/v1/mcp over streamable HTTP. An MCP client needs that URL and an API key and nothing else.”
   • [claimed-docs] “`--json` works on any command and has a stable shape. Additive changes only.”
   • [claimed-docs] “Projects, loops, intentions, stories and tasks from a terminal — and from anything that can run one, including coding agents and cron.”
   • [claimed-docs] “You run an agent — Claude Code or Codex — in a checkout of your repository on your own machine. That agent asks Foreloop for approved tasks …”
   • [probe] “official CLI documented at https://foreloop.com/install”

   proof ↗⚑ flag

   Factory~partialprobed6/10

   Factory documents a public API for creating and driving Droid sessions (factory-docs-13) plus CLI/exec automation paths (droid exec, CI/CD) that give programmatic control, but there's no discoverable OpenAPI/swagger spec (probe found only 404s), suggesting the API reference is thinner or less formally specified than a full REST API story would require. missing for 10: a published OpenAPI/swagger schema, broader API endpoint coverage beyond sessions, independent third-party confirmation of API usage.

   • [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
   • [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
   • [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.factory.ai/openapi.json, https://docs.factory.ai/swagger.json, https://docs.factory.ai/…”

   proof ↗⚑ flag

7. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round to Foreloop
   Foreloop~partialclaimed3/10

   The docs mention a 'write-only' API key type (flpk_YOUR_WRITE_ONLY_KEY) which implies some credential scoping, and the MCP endpoint for agents requires 'an API key', but there is no explicit documentation of issuing least-privilege or role-scoped credentials specifically for agent access to repos/tasks. missing for 10: explicit agent-scoped API key creation/management flow, documented permission levels (read/write/task-only), and evidence of per-agent credential rotation or revocation.

   • [claimed-docs] “await initForeloopFeedback({ apiKey: "flpk_YOUR_WRITE_ONLY_KEY", reporterName: signedInUser?.email, });”
   • [claimed-docs] “Foreloop speaks MCP at /api/app/public/v1/mcp over streamable HTTP. An MCP client needs that URL and an API key and nothing else.”
   • [claimed-docs] “Foreloop speaks MCP at /api/app/public/v1/mcp over streamable HTTP. An MCP client needs that URL and an API key and nothing else”

   proof ↗⚑ flag

   Factory—none0/10

   No evidence of scoped or least-privilege API credential/token issuance for agents; docs mention API sessions and integrations (Jira, Slack, MCP) but nothing about credential scoping, permission tiers for API keys, or least-privilege access control. Tiered autonomy (factory-docs-7) governs action confirmation, not credential scoping.

   ⚑ flag

8. ai-native userBuild against official SDKs

   weight 2 · round to Foreloop
   Foreloop~partialprobed6/10

   Foreloop exposes an official CLI, a JS feedback SDK (initForeloopFeedback), and an MCP endpoint for agent clients, giving AI-native builders official integration surfaces beyond just docs. However, this is all first-party vendor documentation with no independent corroboration or SDK versioning/language-coverage details (e.g. only JS snippet shown, no Python/other language SDKs). Missing for 10: independent/hands-on verification of SDK usage, multi-language SDK coverage, and evidence of SDK stability/versioning beyond the JS snippet.

   • [claimed-docs] “Foreloop speaks MCP at /api/app/public/v1/mcp over streamable HTTP. An MCP client needs that URL and an API key and nothing else”
   • [claimed-docs] “await initForeloopFeedback({ apiKey: "flpk_YOUR_WRITE_ONLY_KEY", reporterName: signedInUser?.email, });”
   • [claimed-docs] “Foreloop speaks MCP at /api/app/public/v1/mcp over streamable HTTP. An MCP client needs that URL and an API key and nothing else.”
   • [claimed-docs] “`--json` works on any command and has a stable shape. Additive changes only.”
   • [probe] “official CLI documented at https://foreloop.com/install”

   proof ↗⚑ flag

   Factory~partialprobed4/10

   Factory documents an API reference for creating/driving Droid sessions (factory-docs-13) and MCP/CLI integration points, suggesting some programmatic access, but there is no evidence of an official language SDK (Python/JS/etc.) and the OpenAPI spec probe returned 404s, indicating thin or unpublished API tooling. missing for 10: no named official SDK packages, no OpenAPI/schema availability, no code samples showing SDK usage.

   • [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.factory.ai/openapi.json, https://docs.factory.ai/swagger.json, https://docs.factory.ai/…”
   • [probe] “official MCP server documented at https://docs.factory.ai/harness/mcp”

   proof ↗⚑ flag

9. ai-native userSubscribe to events via webhooks

   weight 2 · round drawn
   Foreloop—none0/10

   The evidence pack documents MCP connectivity, CLI, and feedback widget integrations, but nowhere mentions webhooks or event subscriptions that an AI-native user could subscribe to; no push/callback mechanism is described.

   ⚑ flag

   Factory—none0/10

   No evidence of webhook subscription support; docs mention API sessions, MCP tools, and integrations (Jira, Slack, etc.) but nothing about webhooks for event notifications. missing for 10: any webhook documentation, event types, subscription endpoints, or third-party confirmation of webhook support.

   ⚑ flag

Agentic features

1. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Factory
   Foreloop~partialclaimed5/10

   Foreloop supports background-style automation via its CLI (usable from cron or other schedulers) and agents that autonomously claim and work tasks in a repo, but it explicitly gates all agent actions behind a mandatory human approval step ('Nothing runs until a person approves a contract'), meaning true unattended autonomy is limited. Missing for 10: concrete documentation of scheduled/recurring automation setups, evidence of fully unattended runs without approval gating, and details on monitoring/alerting for background tasks.

   • [claimed-docs] “Projects, loops, intentions, stories and tasks from a terminal — and from anything that can run one, including coding agents and cron.”
   • [claimed-docs] “Nothing runs until a person approves a contract. Approval is the only gate, and the gate is deliberate: an agent is about to change your rep…”
   • [claimed-docs] “You run an agent — Claude Code or Codex — in a checkout of your repository on your own machine. That agent asks Foreloop for approved tasks …”
   • [claimed-docs] “Approve the big calls from your phone.”

   proof ↗⚑ flag

   Factory~partialclaimed6/10

   droid exec supports one-shot autonomous runs with tiered autonomy suited for CI/CD, scripts, and batch automation, and the API supports creating/driving Droid sessions programmatically, which enables background automation setups. However, there's no evidence of a scheduling/trigger system (e.g., cron-like or event-driven automations) or a dedicated 'automations' dashboard for persistent background jobs. Missing for 10: native scheduling/triggers for autonomous background runs, independent hands-on confirmation of unattended long-running automations, and a dedicated automations management UI.

   • [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
   • [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
   • [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
   • [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”

   proof ↗⚑ flag

2. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round to Factory
   Foreloop—none0/10

   Foreloop's documented model relies on external agents (Claude Code or Codex) running on the user's own machine and connecting via MCP to claim tasks — there is no built-in AI assistant shipped inside Foreloop itself that a user delegates tasks to directly.

   • [claimed-docs] “You run an agent — Claude Code or Codex — in a checkout of your repository on your own machine. That agent asks Foreloop for approved tasks …”
   • [claimed-docs] “Foreloop speaks MCP at /api/app/public/v1/mcp over streamable HTTP. An MCP client needs that URL and an API key and nothing else”
   • [claimed-docs] “Foreloop speaks MCP at /api/app/public/v1/mcp over streamable HTTP. An MCP client needs that URL and an API key and nothing else.”

   proof ↗⚑ flag

   Factory✓fullclaimed8/10

   Factory's core product is built around delegating tasks to Droid agents via App, CLI, or exec, with clear docs on task delegation, custom droids, and session management. missing for 10: independent/hands-on corroboration beyond vendor docs, and quantified reliability of delegated task completion.

   • [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
   • [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
   • [claimed-docs] “delegate scoped tasks to Custom Droids via /droids, and package reusable procedures as Skills with /skills or /create-skill”
   • [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
   • [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”

   proof ↗⚑ flag

3. ai-native userOperate the product with natural-language commands

   weight 2 · round to Factory
   Foreloop~partialclaimed5/10

   Foreloop lets users describe problems in plain language (e.g., "Users can't tell which invoices are overdue") to kick off an intention that an agent turns into tasks, showing some natural-language entry point, but its actual operating surface is a structured CLI (with --json flags) and an MCP interface for agents, not a general natural-language command interface for the product as a whole. Missing for 10: evidence of NL commands controlling arbitrary product actions (not just initial feedback/intention creation), and any hands-on/independent confirmation of this NL flow working.

   • [claimed-docs] “"Users can't tell which invoices are overdue" is enough to start; you do not need to know which files change.”
   • [claimed-docs] “An intention starts from the report, and its agent reads the submission, fills in the intention, and proposes the tasks”
   • [claimed-docs] “Projects, loops, intentions, stories and tasks from a terminal — and from anything that can run one, including coding agents and cron.”

   proof ↗⚑ flag

   Factory✓fullprobed8/10

   Factory's Droid CLI and App let users delegate tasks, run commands, and manage workflows via natural-language prompts, with bash-mode fallback and slash commands (/droids, /skills, /readiness-report) layered on top of NL interaction, showing deep agentic natural-language operation across CLI, exec, and Missions. missing for 10: independent/hands-on user reports confirming NL command quality and reliability outside vendor docs.

   • [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
   • [claimed-docs] “Press ! to toggle bash mode and run shell commands directly without AI interpretation.”
   • [claimed-docs] “delegate scoped tasks to Custom Droids via /droids, and package reusable procedures as Skills with /skills or /create-skill”
   • [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
   • [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”
   • [probe] “official CLI documented at https://docs.factory.ai/droid-cli/quickstart”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   Foreloop—none0/10

   The evidence pack shows CLI docs, MCP endpoint info, and install snippets, but nowhere describes an interactive API reference (e.g., Swagger/OpenAPI explorer, try-it-now console) with runnable examples.

   ⚑ flag

   Factory—none0/10

   Factory does have an API reference section (docs.factory.ai/api-reference/sessions) but there's no evidence of an interactive, runnable-example explorer — the probe for standard OpenAPI/Swagger interactive docs (openapi.json, swagger.json, etc.) returned 404 at all candidate paths, suggesting no live 'try it' interface exists.

   • [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.factory.ai/openapi.json, https://docs.factory.ai/swagger.json, https://docs.factory.ai/…”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   Foreloop—none0/10

   Evidence shows Foreloop exposes an MCP endpoint and CLI with --json output, but there is no mention of a downloadable OpenAPI spec or any machine-readable API specification document for its REST/API surface.

   ⚑ flag

   Factory—none0/10

   Factory has an API reference (sessions endpoints) but probes for standard OpenAPI/swagger spec locations all returned 404, indicating no downloadable machine-readable spec is published; missing for 10: any accessible OpenAPI/swagger JSON file or equivalent machine-readable spec.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.factory.ai/openapi.json, https://docs.factory.ai/swagger.json, https://docs.factory.ai/…”
   • [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”

   proof ↗⚑ flag

3. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round to Foreloop
   Foreloop~partialclaimed3/10

   The docs mention a stability guarantee for CLI `--json` output ('has a stable shape. Additive changes only') and the MCP/API endpoint path includes a 'v1' segment, implying some versioning discipline, but there is no explicit documented deprecation policy, versioning scheme, or migration guidance for the API itself. missing for 10: explicit API version numbering scheme, documented deprecation timeline/process, changelog or migration guides, evidence beyond CLI JSON stability.

   • [claimed-docs] “`--json` works on any command and has a stable shape. Additive changes only.”
   • [claimed-docs] “--json works on any command and has a stable shape. Additive changes only.”
   • [claimed-docs] “Foreloop speaks MCP at /api/app/public/v1/mcp over streamable HTTP. An MCP client needs that URL and an API key and nothing else”
   • [claimed-docs] “Foreloop speaks MCP at /api/app/public/v1/mcp over streamable HTTP. An MCP client needs that URL and an API key and nothing else.”

   proof ↗⚑ flag

   Factory—none0/10

   There is an API reference (sessions) but no evidence of API versioning scheme or a documented deprecation policy; the OpenAPI spec probe even returned 404s across candidate paths, suggesting no formal versioned spec is exposed.

   • [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.factory.ai/openapi.json, https://docs.factory.ai/swagger.json, https://docs.factory.ai/…”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Factory
   Foreloop—none0/10

   Foreloop's docs describe per-task workflows (claim one task, agent does the work, opens one PR) and a CLI/MCP interface, but nothing in the evidence describes performing an operation across many items/tasks simultaneously (e.g., batch approve, bulk edit, multi-task claim). The axis is plausible for a CLI/automation tool but no evidence of bulk operation support exists.

   • [claimed-docs] “You run an agent — Claude Code or Codex — in a checkout of your repository on your own machine. That agent asks Foreloop for approved tasks …”
   • [claimed-docs] “Development, under Agents, lists one row per worker: which agent it is running as, the name that checkout gave itself, the task it holds, an…”
   • [claimed-docs] “`--json` works on any command and has a stable shape. Additive changes only.”
   • [claimed-docs] “Projects, loops, intentions, stories and tasks from a terminal — and from anything that can run one, including coding agents and cron.”

   proof ↗⚑ flag

   Factory~partialclaimed6/10

   droid exec is explicitly documented as a one-shot command 'ideal for CI/CD pipelines, shell scripts, and batch processing,' and the API lets users programmatically create/drive many Droid sessions, both enabling bulk automation across items. However, there's no explicit example, docs, or tooling demonstrating a native 'run across many files/repos/tickets at once' bulk operation feature — it relies on the user scripting droid exec in loops rather than a built-in bulk-operation UI/command. Missing for 10: a dedicated bulk-operation command or documented multi-item batch workflow example, and independent/hands-on evidence of it working at scale.

   • [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
   • [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
   • [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to Factory
   Foreloop—none0/10

   Foreloop's docs describe human-approved task workflows (agents claim tasks, but 'nothing runs until a person approves a contract'), not a rules engine for automatic event-triggered actions; the closest mention is CLI usability from cron, which is not evidence of a native rules/trigger system. missing for 10: any documented rule-definition syntax, event-trigger configuration, or automation-without-approval capability.

   • [claimed-docs] “Nothing runs until a person approves a contract. Approval is the only gate, and the gate is deliberate: an agent is about to change your rep…”
   • [claimed-docs] “Projects, loops, intentions, stories and tasks from a terminal — and from anything that can run one, including coding agents and cron.”

   proof ↗⚑ flag

   Factory~partialclaimed4/10

   Factory's droid exec is explicitly designed to run as a one-shot command in CI/CD pipelines, shell scripts, and batch jobs, which implies it can be wired to external events (e.g., git hooks, CI triggers) with tiered autonomy controlling what runs unattended — but this is an execution mode, not a native rule/trigger definition system where a user declares 'on event X, do Y'. Missing for 10: explicit rule/trigger authoring UI or config, built-in event listeners (e.g., webhook triggers, issue-created triggers), and any documented automation-rules engine beyond CI invocation.

   • [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
   • [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
   • [claimed-docs] “Open Software Factory to see your delivery lifecycle as an automation coverage map.”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round to Foreloop
   Foreloop~partialclaimed3/10

   The only relevant evidence is a single line noting the CLI can be run 'from anything that can run one, including coding agents and cron,' implying users could wire up their own cron-based scheduling — but Foreloop itself documents no native recurring-job/workflow scheduler, retry policy, or schedule management UI. Missing for 10: a first-party scheduling feature, documented recurring-workflow configuration, and evidence of built-in cron/interval management within the product.

   • [claimed-docs] “Projects, loops, intentions, stories and tasks from a terminal — and from anything that can run one, including coding agents and cron.”

   proof ↗⚑ flag

   Factory—none0/10

   Evidence shows droid exec for CI/CD one-shot automation and API session management, but no documentation of scheduling or recurring/cron-style job execution exists anywhere in the pack. missing for 10: any mention of scheduling, cron, recurring triggers, or timed/repeated workflow execution.

   ⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round to Factory
   Foreloop~partialclaimed4/10

   Foreloop's workflow requires human approval before any agent-driven change runs (foreloop-docs-7) and changes are delivered as GitHub pull requests (foreloop-docs-3, foreloop-docs-15), which inherently gives code-level versioning and review via git. However, there is no documentation of versioning, reviewing, or rolling back the automations/tasks/loops themselves (as opposed to the resulting code diffs) — no rollback command, task history, or automation-versioning feature is evidenced. Missing for 10: explicit automation/task versioning, a documented rollback mechanism for tasks or loops, and audit/history views beyond the live agent status list.

   • [claimed-docs] “Nothing runs until a person approves a contract. Approval is the only gate, and the gate is deliberate: an agent is about to change your rep…”
   • [claimed-docs] “You run an agent — Claude Code or Codex — in a checkout of your repository on your own machine. That agent asks Foreloop for approved tasks …”
   • [claimed-docs] “Connecting GitHub is how Foreloop reads your code, and how an agent opens a pull request against it.”
   • [claimed-docs] “Development, under Agents, lists one row per worker: which agent it is running as, the name that checkout gave itself, the task it holds, an…”

   proof ↗⚑ flag

   Factory~partialclaimed5/10

   Factory supports reviewing diffs and merging via git workflow (factory-docs-1, factory-docs-2), and packages automations as reusable Custom Droids/Skills (factory-docs-5), which implies some git-based versioning, but there is no explicit documentation of a version-history feature for Droids/Skills nor an explicit rollback mechanism for automations themselves. missing for 10: explicit versioning UI/history for Skills/Droids, dedicated rollback command or feature distinct from generic git revert.

   • [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
   • [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
   • [claimed-docs] “delegate scoped tasks to Custom Droids via /droids, and package reusable procedures as Skills with /skills or /create-skill”

   proof ↗⚑ flag

End to end feature delivery

1. ai-native userHave an agent automatically generate and run tests to validate its own code changes before proposing them

   weight 2 · round drawn
   Foreloop—none0/10

   No evidence describes Foreloop's agents generating or running tests to validate code changes before proposing them; the docs describe task claiming, PR opening, and human approval gates but never mention automated test generation/execution as a validation step.

   ⚑ flag

   Factory—none0/10

   Docs mention integration with 'tests' as part of Git workflow and bash-mode shell execution, plus readiness-report/readiness-fix commands, but none of this describes an agent autonomously generating and running tests to validate its own code changes before proposing a diff. Missing for 10: explicit documentation of automated test generation, self-validation loop, or evidence droid runs tests as a pre-proposal gate.

   • [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
   • [claimed-docs] “Press ! to toggle bash mode and run shell commands directly without AI interpretation.”
   • [claimed-docs] “Run the /readiness-report slash command in a Factory App web or desktop session, or in the Droid CLI, to evaluate a repository's readiness l…”
   • [claimed-docs] “Automatically fix failing criteria with the /readiness-fix slash command in a Factory App web or desktop session, or in the Droid CLI”

   proof ↗⚑ flag

2. developerHave an agent autonomously diagnose and fix a reported bug

   weight 3 · round to Foreloop
   Foreloop✓fullclaimed7/10

   Foreloop's documented workflow directly supports this: a bug report captured via the feedback widget (with page, metadata, screenshot) becomes an intention/task that an agent (Claude Code or Codex) claims, works on in a repo checkout, and resolves by opening a PR, without the reporter needing to know which files change. Human approval gates task initiation but the diagnosis-and-fix work itself is agent-driven end to end. Missing for 10: independent/hands-on evidence that agents actually diagnose root causes correctly, and more detail on how the agent investigates ambiguous bugs beyond the general task pipeline description.

   • [claimed-docs] “You run an agent — Claude Code or Codex — in a checkout of your repository on your own machine. That agent asks Foreloop for approved tasks …”
   • [claimed-docs] “"Users can't tell which invoices are overdue" is enough to start; you do not need to know which files change.”
   • [claimed-docs] “An intention starts from the report, and its agent reads the submission, fills in the intention, and proposes the tasks”
   • [claimed-docs] “Customers point at the element. The widget adds the page, its metadata, and an optional screenshot, so the model fixes the right thing.”
   • [claimed-docs] “Connecting GitHub is how Foreloop reads your code, and how an agent opens a pull request against it.”

   proof ↗⚑ flag

   Factory~partialclaimed6/10

   Factory's Droid CLI/exec and delegated task workflow support autonomous code changes (diff review and merge) and integrations like Jira/Linear/PagerDuty for bug tracking, plus tiered autonomy for unattended operation, which together plausibly support autonomous bug diagnosis and fixing. However, no evidence explicitly describes an end-to-end 'diagnose a reported bug from ticket to verified fix' workflow or hands-on validation of bug-fixing accuracy. Missing for 10: explicit bug-diagnosis workflow documentation, independent/hands-on evidence of successful autonomous bug fixes, and details on root-cause diagnosis capability.

   • [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
   • [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
   • [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
   • [claimed-docs] “Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems”
   • [claimed-docs] “Automatically fix failing criteria with the /readiness-fix slash command in a Factory App web or desktop session, or in the Droid CLI”

   proof ↗⚑ flag

3. product-managerGo from a mockup or design to a working implementation without an engineering handoff

   weight 2 · round to Factory
   Foreloop~partialclaimed6/10

   Foreloop lets a non-engineer describe a need in plain language (e.g. "Users can't tell which invoices are overdue") or flag a page via a feedback widget with screenshot, and an agent turns that into tasks and a PR without the PM needing to specify files — this covers a 'no engineering handoff' description-to-implementation flow. However, the evidence never shows ingesting an actual mockup/design file (e.g. Figma) as the starting artifact, and a human still must approve the contract before code changes run, so it's not a fully autonomous design-to-code path. Missing for 10: direct mockup/design-file import as an input, evidence of end-to-end completion without any review gate, and a demonstrated design-to-code case study.

   • [claimed-docs] “"Users can't tell which invoices are overdue" is enough to start; you do not need to know which files change.”
   • [claimed-docs] “Customers point at the element. The widget adds the page, its metadata, and an optional screenshot, so the model fixes the right thing.”
   • [claimed-docs] “An intention starts from the report, and its agent reads the submission, fills in the intention, and proposes the tasks”
   • [claimed-docs] “Nothing runs until a person approves a contract. Approval is the only gate, and the gate is deliberate: an agent is about to change your rep…”
   • [claimed-docs] “You run an agent — Claude Code or Codex — in a checkout of your repository on your own machine. That agent asks Foreloop for approved tasks …”

   proof ↗⚑ flag

   Factory✓fullclaimed7/10

   Factory's agent-readiness docs explicitly describe the exact scenario: "A designer shares a mockup, and the system implements it without handoffs" (factory-docs-8), backed by delegation/review/merge workflow (factory-docs-1) and readiness tooling (factory-docs-9, factory-docs-10) that lets non-engineers trigger and validate implementation. Missing for 10: independent or hands-on corroboration of a PM-specific end-to-end mockup-to-merge case study, and detail on how a non-technical PM reviews/approves the diff without engineering involvement.

   • [claimed-docs] “A designer shares a mockup, and the system implements it without handoffs.”
   • [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
   • [claimed-docs] “Run the /readiness-report slash command in a Factory App web or desktop session, or in the Droid CLI, to evaluate a repository's readiness l…”
   • [claimed-docs] “Automatically fix failing criteria with the /readiness-fix slash command in a Factory App web or desktop session, or in the Droid CLI”

   proof ↗⚑ flag

4. developerHave an agent implement a requested feature end-to-end, including writing tests

   weight 3 · round to Factory
   Foreloop~partialclaimed6/10

   Foreloop orchestrates tasks and lets an agent (Claude Code/Codex) claim a task, do the work, and open a PR in the user's repo, which covers end-to-end feature implementation via an external coding agent — but Foreloop itself doesn't write code or tests; it delegates that entirely to the connected agent, and there's no explicit mention of test-writing behavior or verification. missing for 10: explicit evidence that generated PRs include tests, independent/hands-on confirmation of successful end-to-end feature delivery, and detail on how test coverage is verified before approval.

   • [claimed-docs] “You run an agent — Claude Code or Codex — in a checkout of your repository on your own machine. That agent asks Foreloop for approved tasks …”
   • [claimed-docs] “"Users can't tell which invoices are overdue" is enough to start; you do not need to know which files change.”
   • [claimed-docs] “Nothing runs until a person approves a contract. Approval is the only gate, and the gate is deliberate: an agent is about to change your rep…”
   • [claimed-docs] “Connecting GitHub is how Foreloop reads your code, and how an agent opens a pull request against it.”

   proof ↗⚑ flag

   Factory~partialclaimed7/10

   Factory's docs show agents can be delegated end-to-end feature tasks (delegate, diff review, merge), run in terminal/CI with test execution and git workflow, and orchestrate multi-feature projects via Missions, but no explicit first-party evidence confirms the agent autonomously writes tests as part of implementation. missing for 10: explicit documentation of test-writing behavior, independent/hands-on verification of end-to-end feature delivery.

   • [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
   • [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
   • [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
   • [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”

   proof ↗⚑ flag

Environment setup

1. developerHave an agent automatically clone the repo, install dependencies, and configure its own working environment

   weight 2 · round drawn
   Foreloop~partialclaimed4/10

   Foreloop's docs describe the developer running an agent 'in a checkout of your repository on your own machine' and using `foreloop init` to install a task-executor skill into that existing checkout — implying the human, not the agent, does the initial clone/setup, with the agent only claiming and working tasks afterward. There's no documentation of the agent autonomously cloning the repo, installing dependencies, or provisioning its own environment from scratch. Missing for 10: evidence of agent-initiated repo cloning, dependency installation steps, and autonomous environment configuration (e.g., a sandboxed or cloud checkout flow) rather than reliance on a pre-existing local checkout.

   • [claimed-docs] “foreloop init installs the task-executor skill into the checkout, which is what lets an agent working in that directory pick up the project'…”
   • [claimed-docs] “You run an agent — Claude Code or Codex — in a checkout of your repository on your own machine. That agent asks Foreloop for approved tasks …”
   • [claimed-docs] “Development, under Agents, lists one row per worker: which agent it is running as, the name that checkout gave itself, the task it holds, an…”

   proof ↗⚑ flag

   Factory~partialclaimed4/10

   Factory's Droid CLI/exec can run shell commands autonomously (bash mode, tiered autonomy for unconfirmed operations) and operates within a repo's terminal/Git workflow, which implies it could run clone/install commands, but no evidence explicitly describes an agent autonomously cloning a repo or bootstrapping its own dev environment from scratch. missing for 10: explicit documentation of automated repo cloning, dependency installation, or environment provisioning as a first-class capability, and any hands-on example showing this workflow.

   • [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
   • [claimed-docs] “Press ! to toggle bash mode and run shell commands directly without AI interpretation.”
   • [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
   • [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”

   proof ↗⚑ flag

Interactive takeover

1. developerTake over an in-progress agent task in my editor, terminal, or browser to finish or redirect the work

   weight 2 · round to Factory
   Foreloop~partialclaimed4/10

   Foreloop lets agents run locally in a checkout and claim/report tasks, and humans can approve contracts or view agent status via a dashboard row per worker, but there's no evidence of a live handoff mechanism to pause, take over, or redirect an in-progress task from editor/terminal/browser mid-execution. missing for 10: explicit takeover/interrupt UX, editor-integrated control of a running agent, terminal command to seize an active task, and any hands-on confirmation of mid-task redirection.

   • [claimed-docs] “You run an agent — Claude Code or Codex — in a checkout of your repository on your own machine. That agent asks Foreloop for approved tasks …”
   • [claimed-docs] “Nothing runs until a person approves a contract. Approval is the only gate, and the gate is deliberate: an agent is about to change your rep…”
   • [claimed-docs] “Development, under Agents, lists one row per worker: which agent it is running as, the name that checkout gave itself, the task it holds, an…”
   • [claimed-docs] “Approve the big calls from your phone.”

   proof ↗⚑ flag

   Factory~partialclaimed6/10

   Factory explicitly supports multi-surface access (App/web, terminal via Droid CLI, browser) and delegating tasks, reviewing diffs, and merging from any of these surfaces, which implies continuity across surfaces. However, there's no explicit documentation of a 'takeover mid-task' handoff flow (e.g., pausing an in-progress session in one surface and resuming/redirecting it live in another) — the evidence shows task delegation and review/merge but not explicit interactive takeover semantics. Missing for 10: explicit documentation of resuming/redirecting an in-progress session across surfaces, and independent/hands-on confirmation of this handoff working smoothly.

   • [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
   • [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
   • [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”

   proof ↗⚑ flag

2. developerSend follow-up instructions to an active agent session to steer its work without restarting

   weight 2 · round to Factory
   Foreloop—none0/10

   The evidence describes agents claiming tasks, opening PRs, and reporting status, plus approval gates and skill-writing to correct recurring mistakes — but nothing describes sending live follow-up/steering instructions to an already-running agent session mid-task without restarting it.

   ⚑ flag

   Factory~partialclaimed6/10

   Factory's API reference explicitly supports creating and driving Droid sessions including sending messages within an active session (factory-docs-13), and the CLI is interactive by nature, implying follow-up prompts are possible without restarting. However, there is no explicit documentation describing mid-task interruption/steering while the agent is actively executing a long-running task (e.g., Missions or droid exec), so the steering-while-running behavior is not directly evidenced. Missing for 10: explicit docs on interrupting/redirecting an in-progress autonomous run, and independent/hands-on confirmation that follow-up messages actually steer ongoing work rather than queue for the next turn.

   • [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
   • [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
   • [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”

   proof ↗⚑ flag

Sandbox execution

1. developerHave an agent safely execute code and install dependencies inside an isolated sandbox

   weight 3 · round drawn
   Foreloop—none0/10

   Foreloop's docs explicitly describe agents running 'in a checkout of your repository on your own machine' (foreloop-docs-3), with no mention of sandboxing, containerization, or isolated execution environments for code/dependency installation. Missing for 10: any documentation of sandbox/container isolation, dependency install safety, or resource/network restrictions during agent execution.

   • [claimed-docs] “You run an agent — Claude Code or Codex — in a checkout of your repository on your own machine. That agent asks Foreloop for approved tasks …”

   proof ↗⚑ flag

   Factory—none0/10

   Evidence describes tiered autonomy, bash mode, and CI/CD execution (droid exec) but never mentions an isolated sandbox environment for code execution or dependency installation; no container/VM isolation is documented.

   ⚑ flag

Approval controls

1. developerConfigure an agent to auto-approve all its actions instead of confirming each one

   weight 2 · round to Factory
   Foreloop—none0/10

   Foreloop's docs explicitly state that human approval of a contract is a deliberate, mandatory gate before any agent action runs ('Nothing runs until a person approves a contract. Approval is the only gate'), with no mention of a setting to bypass or auto-approve this gate. This indicates the product does not offer an auto-approve configuration, so the story's capability is unsupported.

   • [claimed-docs] “Nothing runs until a person approves a contract. Approval is the only gate, and the gate is deliberate: an agent is about to change your rep…”
   • [claimed-docs] “Approve the big calls from your phone.”

   proof ↗⚑ flag

   Factory~partialclaimed6/10

   Factory's docs confirm 'tiered autonomy' in droid exec that controls what operations run without manual confirmation, implying a configurable auto-approve mode for CI/batch use, but there's no explicit documentation of a full 'auto-approve all actions' toggle or its exact configuration options/flags. missing for 10: explicit config syntax/flag for full auto-approval, independent confirmation of behavior, coverage of auto-approve in interactive (non-exec) sessions.

   • [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
   • [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”

   proof ↗⚑ flag

2. product-managerApprove key agent decisions from my phone while agents continue working

   weight 1 · round to Foreloop
   Foreloop~partialclaimed5/10

   Foreloop explicitly markets phone-based approval of 'big calls' and documents that no agent work runs until a human approves a contract, which matches the core ask. However, evidence is limited to marketing taglines with no detail on the mobile UI/notification flow, how approvals interact with agents continuing to work in parallel, or independent corroboration. Missing for 10: concrete mobile-app/notification workflow details, evidence of multi-agent concurrent operation while awaiting approval, and third-party confirmation.

   • [claimed-docs] “Approve the big calls from your phone.”
   • [claimed-docs] “Nothing runs until a person approves a contract. Approval is the only gate, and the gate is deliberate: an agent is about to change your rep…”
   • [claimed-docs] “Development, under Agents, lists one row per worker: which agent it is running as, the name that checkout gave itself, the task it holds, an…”

   proof ↗⚑ flag

   Factory~partialclaimed3/10

   Docs mention delegating tasks, reviewing diffs, and merging 'from the App or your terminal' and tiered autonomy that gates operations needing confirmation, implying some human-in-the-loop review outside the terminal, but there is no explicit evidence of a mobile/phone app or of approving in-flight agent decisions remotely while agents keep working. Missing for 10: explicit mobile app/phone interface, evidence of asynchronous approval while agent continues running, and independent confirmation of this workflow.

   • [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
   • [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”

   proof ↗⚑ flag

3. engineering-leadSet tiered autonomy levels controlling what an agent can do without manual confirmation

   weight 3 · round to Factory
   Foreloop—none0/10

   Foreloop documents a single approval gate before any agent work starts ('Nothing runs until a person approves a contract') and lets you approve from your phone, but there is no evidence of configurable tiered autonomy levels (e.g., auto-approving certain task types while requiring confirmation for others).

   • [claimed-docs] “Nothing runs until a person approves a contract. Approval is the only gate, and the gate is deliberate: an agent is about to change your rep…”
   • [claimed-docs] “Approve the big calls from your phone.”

   proof ↗⚑ flag

   Factory~partialclaimed6/10

   Factory explicitly documents 'tiered autonomy' in droid exec to control which operations run without manual confirmation, directly matching the story, but this is scoped to the CI/CD-oriented droid exec mode rather than a broader, configurable set of autonomy tiers across all agent surfaces. missing for 10: detail on specific tier levels/permissions, configuration UI or granular controls, and evidence this applies uniformly across App/CLI sessions, not just droid exec.

   • [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
   • [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”

   proof ↗⚑ flag

Visibility monitoring

1. developerWatch what a running agent is doing in real time, including its current status

   weight 3 · round to Foreloop
   Foreloop~partialclaimed6/10

   The docs describe a Development/Agents view listing each worker's agent identity, checkout name, current task, and last message — a real-time-ish status view (foreloop-docs-8). However, there's no evidence of live streaming updates, logs, or granular in-progress activity beyond a status line, and no independent/hands-on confirmation of real-time refresh behavior. missing for 10: evidence of live/streaming updates or detailed activity logs, independent confirmation of real-time refresh, granularity beyond a single status line per agent.

   • [claimed-docs] “Development, under Agents, lists one row per worker: which agent it is running as, the name that checkout gave itself, the task it holds, an…”
   • [claimed-docs] “You run an agent — Claude Code or Codex — in a checkout of your repository on your own machine. That agent asks Foreloop for approved tasks …”

   proof ↗⚑ flag

   Factory~partialclaimed5/10

   Factory's session API lets you create and manage Droid sessions—including lifecycle, settings, and messages—implying some ability to track a running agent's state, and the App/CLI let you review diffs as work progresses. However, there is no explicit documentation of a live status dashboard, streaming logs, or real-time progress view of an in-flight agent. Missing for 10: dedicated real-time monitoring UI/stream, explicit 'live status' feature documentation, independent confirmation of live tracking.

   • [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
   • [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
   • [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”

   proof ↗⚑ flag

2. developerGet notified when an agent completes a task or needs my input

   weight 2 · round to Factory
   Foreloop~partialclaimed3/10

   Foreloop provides a dashboard showing agent status ('last thing it said') and mentions approving 'big calls from your phone', implying some notification/approval flow, but there is no explicit documentation of push/email/Slack notifications when a task completes or needs input. missing for 10: explicit notification channel (email/Slack/push) documentation, confirmation of real-time alerts rather than manual dashboard polling, evidence of notification content for 'needs input' events.

   • [claimed-docs] “Development, under Agents, lists one row per worker: which agent it is running as, the name that checkout gave itself, the task it holds, an…”
   • [claimed-docs] “Approve the big calls from your phone.”
   • [claimed-docs] “Nothing runs until a person approves a contract. Approval is the only gate, and the gate is deliberate: an agent is about to change your rep…”

   proof ↗⚑ flag

   Factory~partialclaimed4/10

   Factory integrates with Slack and PagerDuty and supports tiered autonomy that pauses for manual confirmation, implying some mechanism for alerting developers when input is needed, but there's no explicit documentation of a notification feature for task completion or input requests. missing for 10: explicit notification/alert documentation, evidence of completion pings, confirmation of Slack/PagerDuty being used specifically for task-status alerts.

   • [claimed-docs] “Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems”
   • [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
   • [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”

   proof ↗⚑ flag

Natural language task intake

1. developerDescribe a feature or bug in plain language and have it automatically turned into a scoped implementation task

   weight 3 · round to Foreloop
   Foreloop✓fullclaimed8/10

   Docs describe plain-language input ('Users can't tell which invoices are overdue') turning into scoped, approvable tasks via 'intentions' that an agent fills in and proposes as tasks, gated by human approval before an agent acts on the repo. This directly matches intent-to-spec workflow with first-party documentation across multiple pages. Missing for 10: independent/hands-on verification of the resulting task's scoping quality and no example of the full generated task artifact.

   • [claimed-docs] “"Users can't tell which invoices are overdue" is enough to start; you do not need to know which files change.”
   • [claimed-docs] “An intention starts from the report, and its agent reads the submission, fills in the intention, and proposes the tasks”
   • [claimed-docs] “Nothing runs until a person approves a contract. Approval is the only gate, and the gate is deliberate: an agent is about to change your rep…”
   • [claimed-docs] “You run an agent — Claude Code or Codex — in a checkout of your repository on your own machine. That agent asks Foreloop for approved tasks …”

   proof ↗⚑ flag

   Factory~partialclaimed6/10

   Factory supports delegating tasks described in plain language (via droid CLI, App, or Missions) which get turned into scoped, executable work with diffs to review and merge, and readiness tooling helps scope repo work automatically. However, there's no explicit documented workflow for turning a raw bug/feature description into a formal 'scoped implementation task' spec artifact (e.g., structured ticket/spec generation before execution) distinct from just running the agent. missing for 10: explicit intent-to-spec artifact generation/preview step, independent/hands-on evidence of accurate scoping from vague input, and detail on how ambiguous requests are clarified before execution.

   • [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
   • [claimed-docs] “delegate scoped tasks to Custom Droids via /droids, and package reusable procedures as Skills with /skills or /create-skill”
   • [claimed-docs] “A designer shares a mockup, and the system implements it without handoffs.”
   • [claimed-docs] “Run the /readiness-report slash command in a Factory App web or desktop session, or in the Droid CLI, to evaluate a repository's readiness l…”
   • [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”

   proof ↗⚑ flag

2. product-managerConvert user feedback submissions into structured tasks with proposed scope

   weight 2 · round to Foreloop
   Foreloop✓fullclaimed7/10

   Docs show a feedback-collection widget that captures user submissions, and an agent-driven 'intention' workflow that reads the submission, fills in an intention, and proposes tasks (foreloop-docs-12), with human approval gating execution (foreloop-docs-7). This directly matches converting feedback into structured tasks with proposed scope, though evidence is entirely vendor docs with no independent/hands-on corroboration of output quality or scope structure. Missing for 10: independent/hands-on verification of the feedback-to-task pipeline, and concrete example of the 'proposed scope' structure/fields.

   • [claimed-docs] “An intention starts from the report, and its agent reads the submission, fills in the intention, and proposes the tasks”
   • [claimed-docs] “Nothing runs until a person approves a contract. Approval is the only gate, and the gate is deliberate: an agent is about to change your rep…”
   • [claimed-docs] “Customers point at the element. The widget adds the page, its metadata, and an optional screenshot, so the model fixes the right thing.”
   • [claimed-docs] “"Users can't tell which invoices are overdue" is enough to start; you do not need to know which files change.”

   proof ↗⚑ flag

   Factory—none0/10

   Factory documents task delegation, Missions planning, and integrations with Jira/Linear/Notion, but nothing in the evidence describes ingesting user feedback submissions and outputting a structured task with proposed scope — this specific PM workflow is not documented.

   ⚑ flag

3. developerAttach a marked-up screenshot or mockup to a task so the agent implements the correct visual change

   weight 2 · round to Foreloop
   Foreloop~partialclaimed4/10

   The feedback widget lets a customer point at an element and attach an optional screenshot with metadata so 'the model fixes the right thing' (foreloop-docs-21), which is adjacent to the story but is customer-bug-report flow, not a developer attaching an annotated mockup directly to a task. There's no documentation of markup/annotation tools, mockup uploads, or a developer-facing task-attachment workflow for visual specs. Missing for 10: developer-facing screenshot/mockup attachment to tasks, annotation/markup capability, and evidence the agent parses visual markup to implement precise UI changes.

   • [claimed-docs] “Customers point at the element. The widget adds the page, its metadata, and an optional screenshot, so the model fixes the right thing.”

   proof ↗⚑ flag

   Factory—none0/10

   Evidence only mentions a designer sharing a mockup conceptually (factory-docs-8), with no documented capability to attach an image/screenshot with markup annotations to a task or session, nor any multimodal image-input feature described.

   • [claimed-docs] “A designer shares a mockup, and the system implements it without handoffs.”

   proof ↗⚑ flag

Plan approval

1. developerReview and approve an agent's implementation plan before any code changes are made

   weight 3 · round to Foreloop
   Foreloop✓fullclaimed8/10

   Docs explicitly describe a 'contract' approval gate before any agent touches the repository ('Nothing runs until a person approves a contract... an agent is about to change your repository'), and mention approving 'big calls' from a phone, indicating a review/approve workflow prior to code changes. Missing for 10: independent/hands-on confirmation of the approval UI showing the actual implementation plan content and evidence of what happens if a plan is rejected/revised.

   • [claimed-docs] “Nothing runs until a person approves a contract. Approval is the only gate, and the gate is deliberate: an agent is about to change your rep…”
   • [claimed-docs] “Approve the big calls from your phone.”
   • [claimed-docs] “You run an agent — Claude Code or Codex — in a checkout of your repository on your own machine. That agent asks Foreloop for approved tasks …”

   proof ↗⚑ flag

   Factory~partialclaimed4/10

   Factory's tiered autonomy in droid exec implies some operations require manual confirmation before running, and Missions offer 'plan and execute' orchestration, suggesting a planning phase exists, but there is no explicit documentation of a discrete implementation-plan artifact that a developer reviews and approves before any code changes are made — the described workflow (docs-1) instead centers on reviewing the diff/output after changes. missing for 10: explicit plan-approval UI/step description, evidence of a pre-execution plan artifact, confirmation that no code is touched until plan is approved.

   • [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
   • [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”
   • [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”

   proof ↗⚑ flag

2. engineering-leadApprove a task's scope and contract before an agent is allowed to modify the repository

   weight 2 · round to Foreloop
   Foreloop✓fullclaimed8/10

   Foreloop's docs explicitly state approval of a contract is a mandatory gate before any agent touches the repository ('Nothing runs until a person approves a contract... an agent is about to change your repository'), and agents only claim 'approved tasks' from the repo. This directly matches the engineering-lead approval-gate story, reinforced by mobile approval flow. Missing for 10: independent/hands-on verification that the gate is enforced in practice (not just documented) and detail on what 'scope and contract' specifically contains.

   • [claimed-docs] “Nothing runs until a person approves a contract. Approval is the only gate, and the gate is deliberate: an agent is about to change your rep…”
   • [claimed-docs] “You run an agent — Claude Code or Codex — in a checkout of your repository on your own machine. That agent asks Foreloop for approved tasks …”
   • [claimed-docs] “Approve the big calls from your phone.”

   proof ↗⚑ flag

   Factory~partialclaimed4/10

   Factory documents tiered autonomy that limits what operations run without manual confirmation and diff review before merge, which implies some human-gate mechanics, but there's no explicit evidence of an engineering-lead approving a task's defined scope/contract *before* the agent is permitted to touch the repository — reviews are framed as post-hoc diff review rather than pre-execution scope sign-off. Missing for 10: explicit scope/contract definition step, an approval gate that blocks agent execution until lead sign-off, and evidence this is lead-specific rather than generic autonomy tiering.

   • [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
   • [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”

   proof ↗⚑ flag

Ticket driven tasking

1. developerAssign a coding task to an agent directly from an existing issue or ticket

   weight 3 · round to Factory
   Foreloop—none0/10

   Foreloop's workflow starts from user feedback reports or free-text intentions that get turned into tasks an agent claims (foreloop-docs-4, foreloop-docs-12), but there is no evidence of importing or linking an existing GitHub issue/Jira ticket as the task source for an agent to pick up.

   ⚑ flag

   Factory~partialclaimed4/10

   Factory documents integrations with Jira and Linear ('Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems'), implying tickets can feed into agent workflows, but there is no explicit documentation of assigning/triggering a droid task directly from an issue or ticket UI. missing for 10: explicit workflow/screenshot showing task creation from a Jira/Linear ticket, confirmation that ticket assignment auto-triggers an agent session, independent corroboration.

   • [claimed-docs] “Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Foreloop
   Foreloop~partialprobed6/10

   Foreloop ships a CLI covering projects, loops, intentions, stories and tasks with a stable `--json` output on every command, plus an MCP endpoint for agent clients, suggesting broad API/CLI parity with the UI. However, there's no explicit statement or independent confirmation that literally every UI action (e.g., approvals, feedback widget config, agent monitoring dashboard views) is exposed via API/CLI. missing for 10: explicit UI-to-API parity documentation or audit, independent/hands-on verification, coverage of approval workflow and dashboard views via API.

   • [claimed-docs] “Projects, loops, intentions, stories and tasks from a terminal — and from anything that can run one, including coding agents and cron.”
   • [claimed-docs] “`--json` works on any command and has a stable shape. Additive changes only.”
   • [claimed-docs] “--json works on any command and has a stable shape. Additive changes only.”
   • [claimed-docs] “Foreloop speaks MCP at /api/app/public/v1/mcp over streamable HTTP. An MCP client needs that URL and an API key and nothing else”
   • [claimed-docs] “Foreloop speaks MCP at /api/app/public/v1/mcp over streamable HTTP. An MCP client needs that URL and an API key and nothing else.”
   • [probe] “official CLI documented at https://foreloop.com/install”

   proof ↗⚑ flag

   Factory~partialprobed5/10

   Factory documents a real API (session creation/lifecycle) alongside its CLI and UI, showing some functional overlap, but there's no OpenAPI spec (404s on all candidate URLs) and no evidence that UI-exclusive features like Missions, Software Factory's coverage map, or readiness-report/fix slash commands are exposed via the API. missing for 10: documented API parity for Missions, Software Factory, and readiness commands; a public OpenAPI/schema confirming full surface coverage.

   • [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
   • [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”
   • [claimed-docs] “Open Software Factory to see your delivery lifecycle as an automation coverage map.”
   • [claimed-docs] “Run the /readiness-report slash command in a Factory App web or desktop session, or in the Droid CLI, to evaluate a repository's readiness l…”
   • [claimed-docs] “Automatically fix failing criteria with the /readiness-fix slash command in a Factory App web or desktop session, or in the Droid CLI”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.factory.ai/openapi.json, https://docs.factory.ai/swagger.json, https://docs.factory.ai/…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round drawn
   Foreloop—none0/10

   No evidence of a bulk data export feature or open-format export of projects/tasks/intentions for migration away from Foreloop; the closest hints (--json CLI output, MCP API access) are for programmatic access, not a documented full-data export/leave capability.

   ⚑ flag

   Factory—none0/10

   No evidence addresses data export or portability in open formats, nor any account-deletion/data-takeout mechanism; the docs focus on session management, CLI, and integrations, not exporting user data to leave the platform.

   ⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round drawn
   Foreloop—none0/10

   No evidence of an open-source license or public source code repository; Foreloop is presented as a closed SaaS product with a CLI/widget/MCP server, but nothing indicates the source is available under an open license.

   ⚑ flag

   Factory—none0/10

   No evidence of an open-source license or publicly available source code for Factory/Droid; all evidence points to closed docs and a proprietary CLI/platform. missing for 10: any license file, GitHub repo, or open-source statement covering the product's source code.

   ⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round drawn
   Foreloop—none0/10

   All evidence describes Foreloop as a hosted SaaS (with a CLI/MCP endpoint at foreloop.com/api) that agents connect to remotely; there is no mention of a self-hostable core server, on-prem deployment, or open-source release. missing for 10: any documentation of a self-hosted deployment option, container/server image, or license permitting self-hosting.

   ⚑ flag

   Factory—none0/10

   Factory is presented as a cloud-hosted platform (Factory App, Droid CLI connecting to hosted services, API sessions) with no evidence of a self-hostable core server or on-prem deployment option anywhere in the docs or probes.

   ⚑ flag

Enterprise licensing

1. engineering-leadLicense an enterprise deployment with SSO and commercial support for organization-wide rollout

   weight 2 · round drawn
   Foreloop—none0/10

   No evidence in the pack mentions enterprise licensing, SSO, or commercial support tiers; all citations focus on CLI usage, MCP integration, and agent workflows. This is a plausible axis for a SaaS product like Foreloop (organizations often need SSO/enterprise support), so absence of evidence yields 'none' rather than 'na'.

   ⚑ flag

   Factory—none0/10

   No evidence pack items mention enterprise licensing, SSO, or commercial support offerings; documentation only covers product features like Droid CLI, MCP integration, and readiness reports. Missing for 10: any mention of SSO/SAML, enterprise tier, commercial support SLAs, or org-wide licensing terms.

   ⚑ flag

Model flexibility

1. engineering-leadBring my own LLM or API key so agents run on the model of my choice

   weight 2 · round drawn
   Foreloop—none0/10

   The docs describe running third-party agents (Claude Code or Codex) locally against Foreloop's task API, but there's no evidence Foreloop lets a user configure or supply their own LLM/API key for Foreloop's own model usage (e.g., intention generation, feedback triage). Missing for 10: any documentation of model/provider selection, API key configuration UI, or support for alternate LLM backends.

   • [claimed-docs] “You run an agent — Claude Code or Codex — in a checkout of your repository on your own machine. That agent asks Foreloop for approved tasks …”
   • [claimed-docs] “An intention starts from the report, and its agent reads the submission, fills in the intention, and proposes the tasks”

   proof ↗⚑ flag

   Factory—none0/10

   No evidence pack item mentions bringing your own LLM, custom API keys, or model selection/configuration options; all docs focus on CLI, integrations, and workflow features.

   ⚑ flag

Usage quotas

1. engineering-leadSee and manage plan-based daily task and concurrency limits for agent workflows

   weight 2 · round drawn
   Foreloop—none0/10

   No evidence in the pack mentions plan-based daily task limits, concurrency limits, or any pricing/plan management dashboard for engineering leads; the docs focus on agent workflow mechanics (task claiming, approvals, MCP) with no reference to usage caps or plan tiers.

   ⚑ flag

   Factory—none0/10

   No evidence of plan-based daily task limits, concurrency caps, or admin controls for managing such limits; docs cover CLI, sessions, missions, and integrations but nothing on usage/plan limit visibility or management.

   ⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round drawn
   Foreloop—none0/10

   No evidence anywhere in the pack mentions data residency, region selection, or storage location options; Foreloop's docs cover CLI, MCP, agents, and feedback widgets but never address where data is stored or any regional controls.

   ⚑ flag

   Factory—none0/10

   No evidence in the pack addresses data residency, regional storage options, or compliance/data-location controls; all evidence covers agent workflows, CLI, and integrations unrelated to data storage location.

   ⚑ flag

2. ai-native userControl data retention and deletion

   weight 2 · round drawn
   Foreloop—none0/10

   No evidence pack items mention data retention policies, deletion controls, data export, or privacy settings for user/feedback data collected by Foreloop; all evidence focuses on agent workflows, CLI usage, and MCP integration.

   ⚑ flag

   Factory—none0/10

   No evidence pack items address data retention policies, deletion controls, or privacy settings for user data; all citations relate to product features like CLI, MCP integrations, and agent workflows rather than privacy posture.

   ⚑ flag

3. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Foreloop—none0/10

   No evidence in the pack mentions telemetry, usage tracking, analytics collection, or an opt-out mechanism for Foreloop; the docs cover CLI usage, agent workflows, and feedback widgets but never address data collection or privacy controls.

   ⚑ flag

   Factory—none0/10

   No evidence in the pack mentions telemetry, usage tracking, data collection, or any opt-out/privacy settings for Factory or the Droid CLI.

   ⚑ flag

Chat integration

1. developerTag an agent in a chat thread to discuss and delegate a bug or task

   weight 2 · round to Factory
   Foreloop—none0/10

   Foreloop's workflow is built around agents polling for approved tasks/contracts and reporting status via a dashboard (docs-3, docs-7, docs-8), not around tagging or chatting with agents in a thread. No evidence describes a chat-thread interface or an @-mention mechanism to delegate work conversationally.

   • [claimed-docs] “You run an agent — Claude Code or Codex — in a checkout of your repository on your own machine. That agent asks Foreloop for approved tasks …”
   • [claimed-docs] “Nothing runs until a person approves a contract. Approval is the only gate, and the gate is deliberate: an agent is about to change your rep…”
   • [claimed-docs] “Development, under Agents, lists one row per worker: which agent it is running as, the name that checkout gave itself, the task it holds, an…”

   proof ↗⚑ flag

   Factory~partialclaimed3/10

   Factory's docs mention connecting Slack as an integration to 'keep development synchronized with team systems,' implying some chat-based interaction, but there is no explicit evidence of an @-mention/tagging mechanism in a chat thread to discuss or delegate a specific bug/task to the agent. Missing for 10: explicit Slack @droid tagging workflow, thread-based task delegation UI, and confirmation that discussion happens inline in chat rather than just triggering external actions.

   • [claimed-docs] “Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems”

   proof ↗⚑ flag

Knowledge context

1. developerAdd a context file describing my codebase conventions so agents generate more relevant plans and code

   weight 3 · round to Foreloop
   Foreloop~partialclaimed4/10

   Foreloop's docs describe a 'skill' mechanism ('Write a skill when you notice yourself giving an agent the same correction twice') and an installed task-executor skill in the checkout, which loosely maps to encoding project-specific conventions for agents, but there's no explicit documentation of a dedicated context file (e.g., conventions.md) for describing codebase style/conventions. Missing for 10: explicit context-file feature, guidance on structuring codebase conventions, evidence that agents consume such a file specifically for improving plan/code relevance.

   • [claimed-docs] “foreloop init installs the task-executor skill into the checkout, which is what lets an agent working in that directory pick up the project'…”
   • [claimed-docs] “Write a skill when you notice yourself giving an agent the same correction twice.”

   proof ↗⚑ flag

   Factory—none0/10

   The evidence pack does not mention any context file mechanism (e.g., AGENTS.md, .factory config, or similar) for describing codebase conventions to guide agent behavior; it covers CLI usage, integrations, readiness reports, and missions but nothing about persistent repo-convention context files.

   ⚑ flag

Project management integration

1. product-managerConnect issue trackers like Jira, Linear, ClickUp, or Monday.com so agents can manage tickets directly

   weight 3 · round to Factory
   Foreloop—none0/10

   Foreloop's evidence covers GitHub integration, its own task/loop tracking, MCP server, and feedback widgets, but there is no mention of connecting to Jira, Linear, ClickUp, or Monday.com at all — Foreloop appears to use its own internal issue/task system rather than integrating external issue trackers.

   • [claimed-docs] “Connecting GitHub is how Foreloop reads your code, and how an agent opens a pull request against it.”
   • [claimed-docs] “Projects, loops, intentions, stories and tasks from a terminal — and from anything that can run one, including coding agents and cron.”
   • [claimed-docs] “You run an agent — Claude Code or Codex — in a checkout of your repository on your own machine. That agent asks Foreloop for approved tasks …”

   proof ↗⚑ flag

   Factory~partialclaimed5/10

   Docs confirm Jira and Linear integration explicitly (factory-docs-4), but ClickUp and Monday.com are not mentioned anywhere in the evidence, and there's no detail on ticket management workflows (creating/updating tickets) beyond 'connecting' systems to sync development. Missing for 10: ClickUp integration, Monday.com integration, concrete ticket-management/CRUD workflows via these connectors, independent corroboration.

   • [claimed-docs] “Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems”

   proof ↗⚑ flag

Version control integration

1. developerConnect a GitHub repository so an agent can access the code and open pull requests against it

   weight 3 · round to Foreloop
   Foreloop✓fullclaimed7/10

   Docs explicitly describe connecting GitHub so Foreloop can read code and agents can open PRs against it, with a full workflow (agent claims task in a checkout, does work, opens a PR from the user's GitHub account, gated by human approval). Missing for 10: independent/hands-on verification of the GitHub connection flow and no screenshots or third-party confirmation that PRs are actually opened successfully.

   • [claimed-docs] “Connecting GitHub is how Foreloop reads your code, and how an agent opens a pull request against it.”
   • [claimed-docs] “You run an agent — Claude Code or Codex — in a checkout of your repository on your own machine. That agent asks Foreloop for approved tasks …”
   • [claimed-docs] “Nothing runs until a person approves a contract. Approval is the only gate, and the gate is deliberate: an agent is about to change your rep…”
   • [claimed-docs] “foreloop init installs the task-executor skill into the checkout, which is what lets an agent working in that directory pick up the project'…”

   proof ↗⚑ flag

   Factory~partialclaimed5/10

   Docs indicate Factory works within Git workflows (delegate tasks, review diffs, merge from App/terminal) and can run in CI/CD via droid exec, implying repo access and PR-opening capability, but there is no explicit documentation describing connecting/authorizing a GitHub repository or an explicit PR-creation feature. Missing for 10: explicit GitHub repo connection/auth flow documentation, explicit 'open pull request' feature description, and independent/hands-on confirmation of PR creation.

   • [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
   • [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
   • [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”

   proof ↗⚑ flag

2. developerGrant an agent access to my repositories with a one-click install, without complex setup

   weight 2 · round to Foreloop
   Foreloop~partialclaimed5/10

   Docs confirm connecting GitHub gives Foreloop read/PR access to repos (foreloop-docs-15) and that agents run against a checkout, but achieving this requires installing a global CLI, running foreloop init, configuring MCP with an API key, and running an agent locally — multiple manual steps rather than a true one-click flow. missing for 10: explicit one-click OAuth install flow, evidence of a simple GitHub App install button, independent confirmation of setup simplicity.

   • [claimed-docs] “Connecting GitHub is how Foreloop reads your code, and how an agent opens a pull request against it.”
   • [claimed-docs] “npm i -g foreloop # installs globally, so the CLI works in any directory”
   • [claimed-docs] “foreloop init installs the task-executor skill into the checkout, which is what lets an agent working in that directory pick up the project'…”
   • [claimed-docs] “Foreloop speaks MCP at /api/app/public/v1/mcp over streamable HTTP. An MCP client needs that URL and an API key and nothing else”
   • [claimed-docs] “Foreloop speaks MCP at /api/app/public/v1/mcp over streamable HTTP. An MCP client needs that URL and an API key and nothing else.”

   proof ↗⚑ flag

   Factory—none0/10

   Evidence shows Factory integrates with Git workflows, CLI, and external tools like Jira/Slack/MCP, but there is no mention of a one-click GitHub/repo install or simplified OAuth-based repo authorization flow. Setup appears to require CLI installation and configuration rather than a one-click grant.

   ⚑ flag

Ci remediation

1. engineering-leadHave failed CI workflows automatically diagnosed and fixed with a proposed pull request

   weight 3 · round to Factory
   Foreloop—none0/10

   Foreloop's documented workflow is task/intention-based (feedback → task → agent → PR), with no evidence of CI pipeline integration, failed-build detection, or automated diagnosis-and-fix triggered by CI failures. The axis is plausible for an agent-driven PR tool, but nothing in the evidence pack shows Foreloop watching or reacting to CI status.

   ⚑ flag

   Factory~partialclaimed4/10

   Factory's droid exec is explicitly designed to run in CI/CD pipelines as a one-shot task with tiered autonomy, and Droid CLI integrates with Git workflows, suggesting the building blocks exist for automating CI fixes. However, there is no direct evidence of a dedicated feature that detects a failed CI workflow, diagnoses the failure, and automatically opens a proposed pull request end-to-end. missing for 10: explicit CI-failure detection/trigger integration, automatic diagnosis-to-PR workflow documentation, and any hands-on/independent proof of this specific use case.

   • [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
   • [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
   • [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”

   proof ↗⚑ flag

2. developerTrigger an agent from CI/CD pipelines to fix a broken build or failing test

   weight 2 · round to Factory
   Foreloop—none0/10

   Foreloop's docs describe agents claiming human-approved tasks and opening PRs from a local checkout, with a CLI usable from "anything that can run one, including coding agents and cron" (foreloop-docs-16), but there is no evidence of CI/CD pipeline integration or of the agent being triggered specifically to fix broken builds or failing tests — its workflow centers on approved feature/task work, not automated CI failure remediation.

   • [claimed-docs] “You run an agent — Claude Code or Codex — in a checkout of your repository on your own machine. That agent asks Foreloop for approved tasks …”
   • [claimed-docs] “Nothing runs until a person approves a contract. Approval is the only gate, and the gate is deliberate: an agent is about to change your rep…”
   • [claimed-docs] “Projects, loops, intentions, stories and tasks from a terminal — and from anything that can run one, including coding agents and cron.”

   proof ↗⚑ flag

   Factory✓fullprobed8/10

   Factory explicitly documents `droid exec` as a one-shot CLI command designed for CI/CD pipelines, shell scripts, and batch processing, with tiered autonomy controls for unattended operation — directly enabling triggering an agent from CI to fix builds/tests. Missing for 10: a concrete worked example of a CI pipeline invoking droid exec on a failing test/build, and independent/hands-on corroboration beyond vendor docs.

   • [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
   • [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
   • [probe] “official CLI documented at https://docs.factory.ai/droid-cli/quickstart”

   proof ↗⚑ flag

Diff review

1. developerConfigure an agent to automatically open a pull request when its task completes

   weight 2 · round to Foreloop
   Foreloop✓fullclaimed8/10

   Docs clearly describe the agent workflow: it claims a task, does the work, and opens a pull request from the user's GitHub account, with GitHub connection enabling PR creation, gated by human approval of the contract before work starts. Missing for 10: independent/hands-on confirmation that PRs are reliably opened on task completion, and no detail on configuring PR-specific settings (e.g., target branch, draft PRs).

   • [claimed-docs] “You run an agent — Claude Code or Codex — in a checkout of your repository on your own machine. That agent asks Foreloop for approved tasks …”
   • [claimed-docs] “Connecting GitHub is how Foreloop reads your code, and how an agent opens a pull request against it.”
   • [claimed-docs] “Nothing runs until a person approves a contract. Approval is the only gate, and the gate is deliberate: an agent is about to change your rep…”

   proof ↗⚑ flag

   Factory~partialclaimed3/10

   Docs show git workflow integration (droid-cli git workflow, delegate task then review diff and merge, droid exec for CI/CD pipelines) which implies PR-related automation, but there is no explicit documentation of a feature to automatically open a pull request when a task completes. Missing for 10: explicit config/flag for auto-PR creation, first-party example of a droid opening a PR on completion, independent/hands-on confirmation.

   • [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
   • [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
   • [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”

   proof ↗⚑ flag

2. developerReview a diff of an agent's changes and approve it before it becomes a pull request

   weight 3 · round to Factory
   Foreloop—none0/10

   Foreloop's 'approval' gate (foreloop-docs-7) is a pre-work contract approval, not a diff review after the agent finishes changes — the agent then 'opens a pull request from your GitHub account' directly (foreloop-docs-3), with no documented step where a person reviews the diff before the PR is created. No evidence describes a diff-preview/approve-before-PR workflow.

   • [claimed-docs] “You run an agent — Claude Code or Codex — in a checkout of your repository on your own machine. That agent asks Foreloop for approved tasks …”
   • [claimed-docs] “Nothing runs until a person approves a contract. Approval is the only gate, and the gate is deliberate: an agent is about to change your rep…”

   proof ↗⚑ flag

   Factory~partialclaimed6/10

   Docs explicitly describe the delegate-review-merge workflow ('Delegate a task, review the diff, and merge from the App or your terminal') and tiered autonomy controls that gate what runs without confirmation, supporting a review-before-PR gate. However, there is no detailed documentation of the diff review UI itself, approval mechanics, or independent/hands-on confirmation of this exact flow. Missing for 10: dedicated diff-review UI documentation, explicit approval-step mechanics, and independent verification of the review-before-PR gate.

   • [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
   • [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”

   proof ↗⚑ flag

Pr review automation

1. ai-native userHave incoming issues automatically triaged with severity suggested and routed to the right owner

   weight 2 · round drawn
   Foreloop—none0/10

   Foreloop's docs describe turning user feedback into intentions/tasks that agents pick up and claim (foreloop-docs-12, foreloop-docs-4), but there is no evidence of automatic severity classification or routing to a specific human owner — tasks are self-claimed by agents rather than triaged and assigned. missing for 10: severity scoring, owner/routing logic, any triage workflow evidence.

   • [claimed-docs] “"Users can't tell which invoices are overdue" is enough to start; you do not need to know which files change.”
   • [claimed-docs] “An intention starts from the report, and its agent reads the submission, fills in the intention, and proposes the tasks”

   proof ↗⚑ flag

   Factory—none0/10

   Factory documents integrations with issue trackers like Jira, Linear, and PagerDuty (factory-docs-4) and generic custom droid/skill automation (factory-docs-5), but there is no evidence of an automated triage workflow that suggests severity or routes issues to an owner. missing for 10: any documented triage feature, severity classification logic, or owner-routing capability.

   • [claimed-docs] “Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems”
   • [claimed-docs] “delegate scoped tasks to Custom Droids via /droids, and package reusable procedures as Skills with /skills or /create-skill”

   proof ↗⚑ flag

Readiness checks

1. engineering-leadAutomatically fix failing agent-readiness criteria in my repository

   weight 1 · round to Factory
   Foreloop—none0/10

   Foreloop's docs describe a task/feedback pipeline where agents claim approved tasks and open PRs, but nothing in the evidence pack mentions detecting or automatically remediating 'agent-readiness criteria' (e.g., AGENTS.md, tool-permission manifests, readiness scoring) in a repository. Missing for 10: any readiness-criteria scanning/grading feature, evidence of automatic detection of failing criteria, and evidence of automated fixes tied to such criteria.

   ⚑ flag

   Factory✓fullclaimed8/10

   Factory explicitly documents a /readiness-fix slash command that automatically fixes failing agent-readiness criteria, usable from the Factory App or Droid CLI, complementing the /readiness-report diagnostic command. This directly matches the story's ask, though only first-party docs support it with no independent/hands-on corroboration. Missing for 10: independent or hands-on verification that /readiness-fix reliably resolves criteria, and detail on which criteria types it can/cannot auto-fix.

   • [claimed-docs] “Run the /readiness-report slash command in a Factory App web or desktop session, or in the Droid CLI, to evaluate a repository's readiness l…”
   • [claimed-docs] “Automatically fix failing criteria with the /readiness-fix slash command in a Factory App web or desktop session, or in the Droid CLI”

   proof ↗⚑ flag

2. engineering-leadRun a readiness report that evaluates how ready my repository is for autonomous agents

   weight 2 · round to Factory
   Foreloop—none0/10

   No evidence in the pack describes a 'readiness report' or any repository-readiness scoring/assessment feature for autonomous agents; Foreloop's docs cover task claiming, MCP connectivity, feedback widgets, and approval gates, but nothing about evaluating repo readiness.

   ⚑ flag

   Factory✓fullclaimed8/10

   Factory explicitly documents a `/readiness-report` slash command (in App or Droid CLI) that evaluates a repository's agent-readiness level, plus a companion `/readiness-fix` command to auto-remediate failing criteria, directly matching the engineering-lead's story. Missing for 10: independent/hands-on validation of report contents and no sample report output shown.

   • [claimed-docs] “Run the /readiness-report slash command in a Factory App web or desktop session, or in the Droid CLI, to evaluate a repository's readiness l…”
   • [claimed-docs] “Automatically fix failing criteria with the /readiness-fix slash command in a Factory App web or desktop session, or in the Droid CLI”
   • [claimed-docs] “A designer shares a mockup, and the system implements it without handoffs.”

   proof ↗⚑ flag

Security remediation

1. engineering-leadHave security alerts automatically validated and remediated with an opened pull request

   weight 2 · round drawn
   Foreloop—none0/10

   Foreloop's evidence describes a general feedback/task-to-PR agent workflow (bug reports, feature requests) with human approval gates, but nothing in the pack mentions ingesting security alerts (e.g., from SAST/SCA/dependency scanners), validating them, or triggering remediation PRs specifically for vulnerabilities. The axis is plausible for a task-automation platform, but no evidence shows this specific capability.

   ⚑ flag

   Factory—none0/10

   Evidence covers task delegation, CLI automation, readiness reports/fixes, and CI/CD execution, but nothing describes ingesting security alerts (e.g., from a scanner like Dependabot/Snyk) and automatically validating and remediating them via an opened PR. No security-alert-specific workflow or integration is documented.

   ⚑ flag

Concurrent execution

1. engineering-leadRun many agent tasks concurrently to scale delivery throughput

   weight 3 · round to Factory
   Foreloop~partialclaimed4/10

   The docs show that multiple agent workers can each claim tasks and run in parallel checkouts, with a dashboard listing one row per worker (foreloop-docs-8), implying some capacity for concurrent agent execution across a team. However, there is no explicit product feature for orchestrating, scaling, or managing many concurrent tasks at once, no throughput/scale claims, and no evidence of built-in concurrency limits or fleet management tooling. missing for 10: explicit support/documentation for running many agents simultaneously at scale, concurrency controls, throughput metrics, or a scheduler/queue for parallel task dispatch.

   • [claimed-docs] “You run an agent — Claude Code or Codex — in a checkout of your repository on your own machine. That agent asks Foreloop for approved tasks …”
   • [claimed-docs] “Development, under Agents, lists one row per worker: which agent it is running as, the name that checkout gave itself, the task it holds, an…”

   proof ↗⚑ flag

   Factory~partialclaimed7/10

   Factory provides several building blocks for scaling agent work: droid exec is designed for one-shot, CI/CD, and batch processing use cases, the Sessions API lets you create and manage multiple Droid sessions programmatically, and Missions support orchestrating large multi-feature projects with structured coordination. Together these imply the ability to run many concurrent tasks, but no evidence explicitly states a documented concurrency limit, dashboard for tracking many simultaneous droids, or independent case study proving throughput scaling. Missing for 10: explicit concurrency/parallelism guarantees or limits, a multi-task monitoring UI description, and independent/hands-on validation of running many tasks simultaneously.

   • [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
   • [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”
   • [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
   • [claimed-docs] “Open Software Factory to see your delivery lifecycle as an automation coverage map.”

   proof ↗⚑ flag

2. engineering-leadCreate agent sessions on behalf of other users in my organization

   weight 2 · round drawn
   Foreloop—none0/10

   The evidence describes agents run locally by an individual user in their own repo checkout, claiming tasks and opening PRs from their own GitHub account, but nothing describes an engineering-lead provisioning or launching agent sessions on behalf of other org members. Missing for 10: any admin/org-level session creation API or UI, delegated-session documentation, or multi-user session management evidence.

   • [claimed-docs] “You run an agent — Claude Code or Codex — in a checkout of your repository on your own machine. That agent asks Foreloop for approved tasks …”
   • [claimed-docs] “Development, under Agents, lists one row per worker: which agent it is running as, the name that checkout gave itself, the task it holds, an…”
   • [claimed-docs] “Connecting GitHub is how Foreloop reads your code, and how an agent opens a pull request against it.”

   proof ↗⚑ flag

   Factory—none0/10

   The API reference (factory-docs-13) shows session creation/management exists, but nothing in the evidence indicates an org-admin or lead can create/manage sessions on behalf of other specific users within an organization. missing for 10: evidence of org-level user impersonation, admin controls for delegating sessions to teammates, or role-based session ownership across users.

   • [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”

   proof ↗⚑ flag

Deployment flexibility

1. developerUse a managed cloud offering to run agents without operating my own backend infrastructure

   weight 2 · round to Factory
   Foreloop—none0/10

   Foreloop's own docs state the opposite of this story: agents (Claude Code or Codex) run 'in a checkout of your repository on your own machine,' with Foreloop only coordinating tasks and approvals — there is no evidence of a managed cloud runtime that executes agents on Foreloop's infrastructure.

   • [claimed-docs] “You run an agent — Claude Code or Codex — in a checkout of your repository on your own machine. That agent asks Foreloop for approved tasks …”
   • [claimed-docs] “Development, under Agents, lists one row per worker: which agent it is running as, the name that checkout gave itself, the task it holds, an…”

   proof ↗⚑ flag

   Factory✓fullclaimed7/10

   Factory's docs describe a hosted 'Factory App' (web/desktop) where users delegate tasks and review diffs without local setup, plus an API for creating and managing Droid sessions server-side, indicating a managed backend rather than self-hosted infra. Missing for 10: independent/hands-on confirmation of the cloud service's scalability and infrastructure details beyond first-party docs.

   • [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
   • [claimed-docs] “Run the /readiness-report slash command in a Factory App web or desktop session, or in the Droid CLI, to evaluate a repository's readiness l…”
   • [claimed-docs] “Automatically fix failing criteria with the /readiness-fix slash command in a Factory App web or desktop session, or in the Droid CLI”
   • [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”

   proof ↗⚑ flag

Headless automation

1. developerRun an agent headlessly inside CI/CD pipelines and shell scripts

   weight 2 · round to Factory
   Foreloop~partialclaimed5/10

   Docs mention the CLI is invocable from 'anything that can run one, including coding agents and cron' and support scriptable --json output, suggesting headless invocation is possible (foreloop-docs-16, foreloop-docs-9). However, other docs state that 'nothing runs until a person approves a contract' (foreloop-docs-7), implying a human-in-the-loop gate that complicates fully unattended CI/CD execution, and there is no explicit CI/CD pipeline example or corroboration. Missing for 10: explicit CI/CD/pipeline example, evidence of non-interactive approval flow, independent confirmation of headless operation.

   • [claimed-docs] “Projects, loops, intentions, stories and tasks from a terminal — and from anything that can run one, including coding agents and cron.”
   • [claimed-docs] “`--json` works on any command and has a stable shape. Additive changes only.”
   • [claimed-docs] “Nothing runs until a person approves a contract. Approval is the only gate, and the gate is deliberate: an agent is about to change your rep…”

   proof ↗⚑ flag

   Factory✓fullclaimed9/10

   Factory explicitly documents droid exec as a one-shot headless command designed for CI/CD pipelines, shell scripts, and batch processing, with tiered autonomy controls for unattended operation. Missing for 10: independent/hands-on confirmation of real-world CI pipeline usage beyond first-party docs.

   • [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
   • [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
   • [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”

   proof ↗⚑ flag