Factory vs HumanLayer
HumanLayer wins · 21–22 (27 drawn)
Agenticness — how well agents can access and operate the productAgenticness
How well agents can access and operate the product
Agent access
ai-native userPoint an agent at llms.txt or agent-oriented docs
weight 2 · round to FactoryFactory hosts an actual llms.txt at docs.factory.ai/llms.txt (HTTP 200) confirmed by direct probe, and its docs describe an agent-native platform with structured agent-oriented documentation (droid-cli, agent-readiness, missions) that an agent could be pointed at. missing for 10: independent/hands-on confirmation that an agent successfully consumes llms.txt in practice, and no explicit vendor statement encouraging users to point agents at llms.txt.
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.factory.ai/llms.txt # Factory Documentation > Documentation for Factory, the agent-native software…”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
- [claimed-docs] “Run the /readiness-report slash command in a Factory App web or desktop session, or in the Droid CLI, to evaluate a repository's readiness l…”
ai-native userRun the product headlessly / in CI for automation
weight 2 · round drawnFactory documents droid exec as a one-shot CLI command explicitly designed for CI/CD pipelines, shell scripts, and batch processing, with tiered autonomy controls for unattended operation. This directly satisfies headless/CI automation. Missing for 10: independent/hands-on third-party verification of CI usage and more detail on exit codes/output formats for pipeline integration.
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
- [probe] “official CLI documented at https://docs.factory.ai/droid-cli/quickstart”
Docs explicitly describe running HumanLayer headlessly via `humanlayer automation run` from CI jobs, cron machines, or scripts, plus launch tokens for non-interactive/non-PTY environments, and remote daemon hosts on cloud VMs or servers, directly matching the CI/automation story. Missing for 10: independent/hands-on confirmation of CI usage and more detail on auth/config specifics for automated pipelines.
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [probe] “official CLI documented at https://docs.humanlayer.com/guide/remote-daemons”
ai-native userPlug MCP servers into this product so it can use their tools
weight 3 · round to FactoryDocs explicitly state Droid CLI can connect MCP tools alongside Jira/Notion/Slack/Linear/PagerDuty integrations, confirming MCP client support. However, there is no detail on setup/configuration process, supported transport types, or independent/hands-on confirmation of MCP tool usage in practice. missing for 10: configuration details for adding MCP servers, examples of MCP tools being invoked, independent verification of functionality.
- [claimed-docs] “Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
HumanLayernone0/10No evidence anywhere in the pack mentions MCP servers or the ability to plug external tool servers into HumanLayer; integrations mentioned (Jira, Slack, GitHub, Linear) are task-source connectors, not MCP tool servers. Missing for 10: any mention of MCP protocol support, MCP server configuration, or tool-plugin mechanism.
ai-native userConnect an agent via an official MCP server
weight 3 · round to FactoryFactory is primarily a coding agent (Droid), and factory-docs-4 shows it can connect OUT to MCP tools (client-side), which wouldn't count under the na rule — but factory-probe-3 explicitly indicates an official MCP server is documented at docs.factory.ai/harness/mcp, flipping this to the applicable server-side capability. However, the evidence pack only confirms the page's existence via a probe hit with no detail on what the server exposes or how other agents connect to it. Missing for 10: detailed documentation of the MCP server's tool surface, setup instructions for external agents, and independent confirmation of a working connection.
- [probe] “official MCP server documented at https://docs.factory.ai/harness/mcp”
- [claimed-docs] “Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems”
HumanLayernone0/10HumanLayer is a platform/control-plane for running and overseeing coding-agent sessions (Claude Code, Codex) rather than itself being an agent, so an official MCP server is a fair, applicable axis. The evidence pack documents many integrations (Jira, Slack, GitHub, Linear), a CLI, and remote daemons, but no MCP server offering is mentioned anywhere, and API/OpenAPI probes returned 404s. missing for 10: any first-party MCP server documentation, endpoint, or 'mcp serve' style capability.
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.humanlayer.com/openapi.json, https://docs.humanlayer.com/swagger.json, https://docs.hum…”
- [probe] “PROBE llms.txt: HTTP 404 at https://docs.humanlayer.com/llms.txt”
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
ai-native userUse an official CLI
weight 2 · round to FactoryFactory ships an official Droid CLI with documented quickstart, terminal/editor/Git integration, bash mode, one-shot exec for CI/CD, and slash-command features—clearly an official CLI for AI-native workflows. Missing for 10: independent/hands-on third-party corroboration beyond vendor docs.
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
- [claimed-docs] “Press ! to toggle bash mode and run shell commands directly without AI interpretation.”
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [probe] “official CLI documented at https://docs.factory.ai/droid-cli/quickstart”
HumanLayer documents an official CLI (e.g. `humanlayer automation run`, launch tokens, remote-daemon control) used for CI, cron, and scripted agentic workflows, confirmed by a dedicated CLI doc page. missing for 10: no independent/hands-on verification of the CLI, no full command reference, and no evidence of broader CLI feature parity with the app.
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [probe] “official CLI documented at https://docs.humanlayer.com/guide/remote-daemons”
ai-native userDrive the product through a documented public API
weight 3 · round to FactoryFactory documents a public API for creating and driving Droid sessions (factory-docs-13) plus CLI/exec automation paths (droid exec, CI/CD) that give programmatic control, but there's no discoverable OpenAPI/swagger spec (probe found only 404s), suggesting the API reference is thinner or less formally specified than a full REST API story would require. missing for 10: a published OpenAPI/swagger schema, broader API endpoint coverage beyond sessions, independent third-party confirmation of API usage.
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.factory.ai/openapi.json, https://docs.factory.ai/swagger.json, https://docs.factory.ai/…”
HumanLayer documents a CLI (`humanlayer automation run`, launch tokens, remote daemons) that lets automation environments drive sessions programmatically, which is a form of documented programmatic control, but there is no evidence of a documented public REST/OpenAPI API — probes explicitly found openapi.json/swagger.json/llms.txt all 404. missing for 10: a documented HTTP/OpenAPI public API spec, SDK/client library docs, and independent confirmation of API usage.
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.humanlayer.com/openapi.json, https://docs.humanlayer.com/swagger.json, https://docs.hum…”
- [probe] “PROBE llms.txt: HTTP 404 at https://docs.humanlayer.com/llms.txt”
- [probe] “official CLI documented at https://docs.humanlayer.com/guide/remote-daemons”
ai-native userIssue scoped/least-privilege API credentials for an agent
weight 2 · round to HumanLayerFactorynone0/10No evidence of scoped or least-privilege API credential/token issuance for agents; docs mention API sessions and integrations (Jira, Slack, MCP) but nothing about credential scoping, permission tiers for API keys, or least-privilege access control. Tiered autonomy (factory-docs-7) governs action confirmation, not credential scoping.
The only relevant evidence is a mention of a 'launch token' scoped to a single non-interactive command, which implies some least-privilege token issuance, but there is no documentation of a broader credential/permission model, scopes, or API key management for agents. missing for 10: explicit least-privilege credential scoping model, permission granularity, revocation/rotation mechanisms, and any independent corroboration.
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
ai-native userBuild against official SDKs
weight 2 · round to FactoryFactory documents an API reference for creating/driving Droid sessions (factory-docs-13) and MCP/CLI integration points, suggesting some programmatic access, but there is no evidence of an official language SDK (Python/JS/etc.) and the OpenAPI spec probe returned 404s, indicating thin or unpublished API tooling. missing for 10: no named official SDK packages, no OpenAPI/schema availability, no code samples showing SDK usage.
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.factory.ai/openapi.json, https://docs.factory.ai/swagger.json, https://docs.factory.ai/…”
- [probe] “official MCP server documented at https://docs.factory.ai/harness/mcp”
HumanLayernone0/10The evidence pack covers HumanLayer's CLI, workspace config, and third-party integrations (Jira, Slack, GitHub, Linear) but contains no mention of an official SDK (Python, TypeScript, etc.) for building against HumanLayer programmatically. Probes for openapi.json and llms.txt both returned 404, further suggesting no discoverable API/SDK surface.
- [probe] “PROBE llms.txt: HTTP 404 at https://docs.humanlayer.com/llms.txt”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.humanlayer.com/openapi.json, https://docs.humanlayer.com/swagger.json, https://docs.hum…”
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
ai-native userSubscribe to events via webhooks
weight 2 · round drawnFactorynone0/10No evidence of webhook subscription support; docs mention API sessions, MCP tools, and integrations (Jira, Slack, etc.) but nothing about webhooks for event notifications. missing for 10: any webhook documentation, event types, subscription endpoints, or third-party confirmation of webhook support.
HumanLayernone0/10No evidence pack item mentions webhooks or event subscription mechanisms; integrations described (Slack, Jira, Linear, GitHub) are inbound task-creation connectors, not outbound webhook events, and API/OpenAPI probes returned 404s.
Agentic features
ai-native userSet up automations that run autonomously in the background
weight 2 · round to HumanLayerdroid exec supports one-shot autonomous runs with tiered autonomy suited for CI/CD, scripts, and batch automation, and the API supports creating/driving Droid sessions programmatically, which enables background automation setups. However, there's no evidence of a scheduling/trigger system (e.g., cron-like or event-driven automations) or a dedicated 'automations' dashboard for persistent background jobs. Missing for 10: native scheduling/triggers for autonomous background runs, independent hands-on confirmation of unattended long-running automations, and a dedicated automations management UI.
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
- [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”
Docs explicitly describe `humanlayer automation run` for running Cloud-visible coding sessions from CI jobs, cron machines, or scripts, plus remote daemons and launch tokens for non-interactive/background execution contexts, directly supporting autonomous background automations. Missing for 10: independent/hands-on verification of long-running background automations, native scheduling UI, and clarity on how human-approval gates interact with continuous autonomous runs.
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [claimed-docs] “Use `.humanlayer/workspace.json` for shared repository or team configuration. Use `.humanlayer/workspace.local.json` for optional user or ma…”
ai-native userDelegate tasks to a built-in AI assistant inside the product
weight 3 · round to FactoryFactory's core product is built around delegating tasks to Droid agents via App, CLI, or exec, with clear docs on task delegation, custom droids, and session management. missing for 10: independent/hands-on corroboration beyond vendor docs, and quantified reliability of delegated task completion.
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
- [claimed-docs] “delegate scoped tasks to Custom Droids via /droids, and package reusable procedures as Skills with /skills or /create-skill”
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
HumanLayer's core product model is task delegation to AI coding agents (Claude Code, Codex, Bedrock-backed Claude) via 'sessions', with workflow modes (Oneshot, RPI, PRD-Oriented, Freeform) and automation runs from CI/cron, directly matching 'delegate tasks to a built-in AI assistant'. missing for 10: independent/hands-on verification of the delegation experience beyond vendor docs, and clarity on how autonomous vs supervised the assistant is in practice.
- [claimed-docs] “This tutorial teaches you how to create and run one small task in HumanLayer. You will use the macOS app and Claude Code.”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
- [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
- [claimed-docs] “HumanLayer registers these RPI sub-agents for Claude Code sessions”
- [claimed-docs] “Select **Oneshot**, **RPI**, **PRD-Oriented**, or **Freeform**.”
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “This guide shows you how to install, authenticate, select, and check Codex for HumanLayer sessions.”
ai-native userOperate the product with natural-language commands
weight 2 · round to FactoryFactory's Droid CLI and App let users delegate tasks, run commands, and manage workflows via natural-language prompts, with bash-mode fallback and slash commands (/droids, /skills, /readiness-report) layered on top of NL interaction, showing deep agentic natural-language operation across CLI, exec, and Missions. missing for 10: independent/hands-on user reports confirming NL command quality and reliability outside vendor docs.
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
- [claimed-docs] “Press ! to toggle bash mode and run shell commands directly without AI interpretation.”
- [claimed-docs] “delegate scoped tasks to Custom Droids via /droids, and package reusable procedures as Skills with /skills or /create-skill”
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”
- [probe] “official CLI documented at https://docs.factory.ai/droid-cli/quickstart”
HumanLayer's core interaction model is giving natural-language instructions to agent sessions (Claude Code, Codex) to create tasks, configure workspaces, and choose workflow phases, as shown by the example NL workspace-config prompt and workflow-selection docs. Missing for 10: independent/hands-on corroboration of NL command robustness and no evidence of a broader NL command surface beyond task/workflow setup.
- [claimed-docs] “Configure a multi-repository workspace with this repository, ../api, and ../web. Make ../web the primary repository. Ask me before you choos…”
- [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
- [claimed-docs] “Select **Oneshot**, **RPI**, **PRD-Oriented**, or **Freeform**.”
- [claimed-docs] “This tutorial teaches you how to create and run one small task in HumanLayer. You will use the macOS app and Claude Code.”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
Api quality
ai-native userExplore an interactive API reference with runnable examples
weight 2 · round drawnFactorynone0/10Factory does have an API reference section (docs.factory.ai/api-reference/sessions) but there's no evidence of an interactive, runnable-example explorer — the probe for standard OpenAPI/Swagger interactive docs (openapi.json, swagger.json, etc.) returned 404 at all candidate paths, suggesting no live 'try it' interface exists.
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.factory.ai/openapi.json, https://docs.factory.ai/swagger.json, https://docs.factory.ai/…”
HumanLayernone0/10No evidence of an interactive API reference or runnable examples; probes explicitly show no OpenAPI/swagger spec and no llms.txt found, and docs are guide/tutorial style rather than an API reference sandbox.
ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)
weight 2 · round drawnFactorynone0/10Factory has an API reference (sessions endpoints) but probes for standard OpenAPI/swagger spec locations all returned 404, indicating no downloadable machine-readable spec is published; missing for 10: any accessible OpenAPI/swagger JSON file or equivalent machine-readable spec.
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.factory.ai/openapi.json, https://docs.factory.ai/swagger.json, https://docs.factory.ai/…”
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
HumanLayernone0/10A direct probe for OpenAPI/swagger specs at all standard locations returned 404s, and no evidence pack item shows a downloadable machine-readable API spec being offered.
ai-native userTest against a sandbox environment without touching production data
weight 1 · round drawnFactorynone0/10Factory's evidence covers coding agent workflows (CLI, sessions, MCP, readiness reports) but nothing addresses a sandbox environment for testing separate from production data. No mention of sandbox mode, staging environment, or data isolation guarantees.
HumanLayernone0/10HumanLayer's docs describe remote daemons, workspaces, and automation sessions, but there is no mention of a sandbox environment, staging/test data isolation, or any mechanism to separate test runs from production data. missing for 10: explicit sandbox/staging environment documentation, data isolation guarantees, evidence of test-vs-production separation.
ai-native userRely on versioned APIs with a documented deprecation policy
weight 2 · round drawnFactorynone0/10There is an API reference (sessions) but no evidence of API versioning scheme or a documented deprecation policy; the OpenAPI spec probe even returned 404s across candidate paths, suggesting no formal versioned spec is exposed.
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.factory.ai/openapi.json, https://docs.factory.ai/swagger.json, https://docs.factory.ai/…”
HumanLayernone0/10No evidence of API versioning scheme or a documented deprecation policy; probes for openapi.json/llms.txt returned 404s and no API reference or changelog covering versioning/deprecation is present.
Automation depth — how much of the product can run unattendedAutomation depth
How much of the product can run unattended
ai-native userPerform bulk operations across many items at once
weight 2 · round to Factorydroid exec is explicitly documented as a one-shot command 'ideal for CI/CD pipelines, shell scripts, and batch processing,' and the API lets users programmatically create/drive many Droid sessions, both enabling bulk automation across items. However, there's no explicit example, docs, or tooling demonstrating a native 'run across many files/repos/tickets at once' bulk operation feature — it relies on the user scripting droid exec in loops rather than a built-in bulk-operation UI/command. Missing for 10: a dedicated bulk-operation command or documented multi-item batch workflow example, and independent/hands-on evidence of it working at scale.
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
- [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”
HumanLayernone0/10HumanLayer's documentation consistently frames work as single tasks/sessions ('one small task', 'one task on a remote machine', 'one Cloud-visible coding session') with per-task review and approval workflows; there is no mention of batch/bulk operations spanning many items at once. missing for 10: any documented bulk-action API/CLI flag, batch approval mechanism, or multi-item automation workflow.
- [claimed-docs] “This tutorial teaches you how to create and run one small task in HumanLayer. You will use the macOS app and Claude Code.”
- [claimed-docs] “This tutorial teaches you how to run one task on a remote machine. You will control the task from app.humanlayer.com on a machine or phone.”
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
ai-native userDefine rules that trigger actions automatically on events
weight 3 · round to HumanLayerFactory's droid exec is explicitly designed to run as a one-shot command in CI/CD pipelines, shell scripts, and batch jobs, which implies it can be wired to external events (e.g., git hooks, CI triggers) with tiered autonomy controlling what runs unattended — but this is an execution mode, not a native rule/trigger definition system where a user declares 'on event X, do Y'. Missing for 10: explicit rule/trigger authoring UI or config, built-in event listeners (e.g., webhook triggers, issue-created triggers), and any documented automation-rules engine beyond CI invocation.
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
- [claimed-docs] “Open Software Factory to see your delivery lifecycle as an automation coverage map.”
HumanLayer integrations (GitHub, Jira, Linear) create tasks automatically from external events like new issues/tickets, and `humanlayer automation run` lets sessions be triggered from CI jobs, cron, or scripts — both are forms of event-driven automation. However, there's no evidence of a general-purpose rules/conditions engine letting users define arbitrary trigger-condition-action logic; the automation is limited to fixed integration hooks and script-based invocation. Missing for 10: a documented rule-definition interface (conditions, filters, custom triggers) beyond fixed platform integrations, and independent confirmation these event-triggers work reliably in practice.
- [claimed-docs] “Connect Jira Cloud so HumanLayer can create tasks from Jira tickets and keep ticket-driven agent work connected to its source.”
- [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
- [claimed-docs] “Connect Linear so HumanLayer can create tasks from Linear issues, sync issue status, and link HumanLayer artifacts back to the source issue.”
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
ai-native userSchedule recurring jobs or workflows
weight 2 · round to HumanLayerFactorynone0/10Evidence shows droid exec for CI/CD one-shot automation and API session management, but no documentation of scheduling or recurring/cron-style job execution exists anywhere in the pack. missing for 10: any mention of scheduling, cron, recurring triggers, or timed/repeated workflow execution.
HumanLayer's `automation run` CLI lets you trigger a Cloud-visible coding session from a cron machine or CI job, implying recurring runs are possible via external schedulers, but there is no documented native scheduling/recurrence feature (no cron syntax, interval config, or job queue) inside HumanLayer itself. missing for 10: built-in recurring scheduler, interval/cron configuration options, evidence of persistent recurring workflow management.
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
ai-native userVersion, review, and roll back my automations
weight 1 · round drawnFactory supports reviewing diffs and merging via git workflow (factory-docs-1, factory-docs-2), and packages automations as reusable Custom Droids/Skills (factory-docs-5), which implies some git-based versioning, but there is no explicit documentation of a version-history feature for Droids/Skills nor an explicit rollback mechanism for automations themselves. missing for 10: explicit versioning UI/history for Skills/Droids, dedicated rollback command or feature distinct from generic git revert.
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
- [claimed-docs] “delegate scoped tasks to Custom Droids via /droids, and package reusable procedures as Skills with /skills or /create-skill”
HumanLayer's task model provides review (comments, PR draft creation) and history that persists across sessions, giving some review/audit capability, but there is no explicit documentation of versioning workflow definitions or rolling back an automation to a prior version. missing for 10: explicit automation versioning/history diffing, a documented rollback mechanism, and independent confirmation these review features extend to full automation lifecycle management.
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
- [claimed-docs] “Draft PR creation — Ask the session agent to open a draft pull request from the GitHub tab or diff view.”
- [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
Autonomous implementation — end-to-end implementation by the agent — multi-file changes, task completionAutonomous implementation
End-to-end implementation by the agent — multi-file changes, task completion
End to end feature delivery
ai-native userHave an agent automatically generate and run tests to validate its own code changes before proposing them
weight 2 · round drawnFactorynone0/10Docs mention integration with 'tests' as part of Git workflow and bash-mode shell execution, plus readiness-report/readiness-fix commands, but none of this describes an agent autonomously generating and running tests to validate its own code changes before proposing a diff. Missing for 10: explicit documentation of automated test generation, self-validation loop, or evidence droid runs tests as a pre-proposal gate.
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
- [claimed-docs] “Press ! to toggle bash mode and run shell commands directly without AI interpretation.”
- [claimed-docs] “Run the /readiness-report slash command in a Factory App web or desktop session, or in the Droid CLI, to evaluate a repository's readiness l…”
- [claimed-docs] “Automatically fix failing criteria with the /readiness-fix slash command in a Factory App web or desktop session, or in the Droid CLI”
developerHave an agent autonomously diagnose and fix a reported bug
weight 3 · round drawnFactory's Droid CLI/exec and delegated task workflow support autonomous code changes (diff review and merge) and integrations like Jira/Linear/PagerDuty for bug tracking, plus tiered autonomy for unattended operation, which together plausibly support autonomous bug diagnosis and fixing. However, no evidence explicitly describes an end-to-end 'diagnose a reported bug from ticket to verified fix' workflow or hands-on validation of bug-fixing accuracy. Missing for 10: explicit bug-diagnosis workflow documentation, independent/hands-on evidence of successful autonomous bug fixes, and details on root-cause diagnosis capability.
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
- [claimed-docs] “Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems”
- [claimed-docs] “Automatically fix failing criteria with the /readiness-fix slash command in a Factory App web or desktop session, or in the Droid CLI”
HumanLayer supports creating tasks directly from GitHub/Jira/Linear issues and then running an agent session (Claude Code/Codex) against the linked repo, with an 'Oneshot' workflow phase designed for small, clear changes — a plausible bug-fix pipeline. However, the product's core design is human-in-the-loop with approval gates rather than fully autonomous action, and there's no end-to-end documented example of an agent independently diagnosing a bug from a ticket and shipping a fix without human review. Missing for 10: a concrete autonomous bug-diagnosis-and-fix walkthrough, and clarity on how much human approval is required mid-flow.
- [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
- [claimed-docs] “Connect Linear so HumanLayer can create tasks from Linear issues, sync issue status, and link HumanLayer artifacts back to the source issue.”
- [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [community] “"I feel much more comfortable with senior developers approving AI changes to our codebase, then letting loose an autonomous agent with no hu…”
product-managerGo from a mockup or design to a working implementation without an engineering handoff
weight 2 · round to FactoryFactory's agent-readiness docs explicitly describe the exact scenario: "A designer shares a mockup, and the system implements it without handoffs" (factory-docs-8), backed by delegation/review/merge workflow (factory-docs-1) and readiness tooling (factory-docs-9, factory-docs-10) that lets non-engineers trigger and validate implementation. Missing for 10: independent or hands-on corroboration of a PM-specific end-to-end mockup-to-merge case study, and detail on how a non-technical PM reviews/approves the diff without engineering involvement.
- [claimed-docs] “A designer shares a mockup, and the system implements it without handoffs.”
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “Run the /readiness-report slash command in a Factory App web or desktop session, or in the Droid CLI, to evaluate a repository's readiness l…”
- [claimed-docs] “Automatically fix failing criteria with the /readiness-fix slash command in a Factory App web or desktop session, or in the Droid CLI”
HumanLayernone0/10HumanLayer's evidence is entirely about developer-facing workflows: running coding sessions, connecting Jira/Linear/GitHub/Slack, managing remote daemons, and CLI automation for engineers overseeing coding agents. Nothing in the pack shows a mockup/design import capability, a no-code interface, or any path for a non-engineer product manager to turn a design into a working implementation without engineering involvement — in fact the workflow language (RPI, PRD-oriented, Oneshot) and workspace configs assume an engineering operator. Missing for evidence of delivering this story: mockup/design ingestion, PM-oriented no-code UI, and any case study of a non-engineer shipping code end-to-end.
- [claimed-docs] “This tutorial teaches you how to create and run one small task in HumanLayer. You will use the macOS app and Claude Code.”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
- [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
- [claimed-docs] “Select **Oneshot**, **RPI**, **PRD-Oriented**, or **Freeform**.”
developerHave an agent implement a requested feature end-to-end, including writing tests
weight 3 · round to FactoryFactory's docs show agents can be delegated end-to-end feature tasks (delegate, diff review, merge), run in terminal/CI with test execution and git workflow, and orchestrate multi-feature projects via Missions, but no explicit first-party evidence confirms the agent autonomously writes tests as part of implementation. missing for 10: explicit documentation of test-writing behavior, independent/hands-on verification of end-to-end feature delivery.
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”
HumanLayer clearly supports end-to-end autonomous coding sessions (Oneshot, RPI, PRD-Oriented workflows) that implement tasks using agents like Claude Code and Codex, including structured phases and sub-agents for research/plan/implement, which implies substantial feature work can be delegated (humanlayer-docs-6, humanlayer-docs-9, humanlayer-docs-10, humanlayer-docs-14). However, no evidence explicitly confirms the agent writes or runs tests as part of the workflow, and no hands-on account demonstrates a full feature-plus-tests delivery. Missing for 10: explicit documentation or example showing test generation/execution as part of the implementation flow, and independent verification of end-to-end feature completion including tests.
- [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
- [claimed-docs] “HumanLayer registers these RPI sub-agents for Claude Code sessions”
- [claimed-docs] “Select **Oneshot**, **RPI**, **PRD-Oriented**, or **Freeform**.”
- [claimed-docs] “This guide shows you how to install, authenticate, select, and check Codex for HumanLayer sessions.”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
Environment setup
developerHave an agent automatically clone the repo, install dependencies, and configure its own working environment
weight 2 · round to HumanLayerFactory's Droid CLI/exec can run shell commands autonomously (bash mode, tiered autonomy for unconfirmed operations) and operates within a repo's terminal/Git workflow, which implies it could run clone/install commands, but no evidence explicitly describes an agent autonomously cloning a repo or bootstrapping its own dev environment from scratch. missing for 10: explicit documentation of automated repo cloning, dependency installation, or environment provisioning as a first-class capability, and any hands-on example showing this workflow.
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
- [claimed-docs] “Press ! to toggle bash mode and run shell commands directly without AI interpretation.”
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
Docs describe workspace configuration (workspace.json, multi-repo setups) and remote hosts that must have access to code/tools/credentials, and one example prompt asks the agent about 'setup commands or local files to copy,' implying some environment configuration ability. However, there is no explicit description of the agent autonomously cloning a repo or installing dependencies end-to-end without human setup of the host/workspace first. Missing for 10: explicit documentation of automatic repo cloning, dependency installation steps, and end-to-end environment bootstrap without prior manual host/workspace configuration.
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [claimed-docs] “Use `.humanlayer/workspace.json` for shared repository or team configuration. Use `.humanlayer/workspace.local.json` for optional user or ma…”
- [claimed-docs] “Configure a multi-repository workspace with this repository, ../api, and ../web. Make ../web the primary repository. Ask me before you choos…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
Interactive takeover
developerTake over an in-progress agent task in my editor, terminal, or browser to finish or redirect the work
weight 2 · round to HumanLayerFactory explicitly supports multi-surface access (App/web, terminal via Droid CLI, browser) and delegating tasks, reviewing diffs, and merging from any of these surfaces, which implies continuity across surfaces. However, there's no explicit documentation of a 'takeover mid-task' handoff flow (e.g., pausing an in-progress session in one surface and resuming/redirecting it live in another) — the evidence shows task delegation and review/merge but not explicit interactive takeover semantics. Missing for 10: explicit documentation of resuming/redirecting an in-progress session across surfaces, and independent/hands-on confirmation of this handoff working smoothly.
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
Docs describe tasks with session/history continuity 'across agents and workstations', remote control from app.humanlayer.com on any machine or phone, and CLI-driven remote daemons for terminal/server contexts, all pointing to genuine hand-off of in-progress work between editor (Claude Code), terminal (remote daemon/CLI), and browser (web app). Live multiplayer draft recovery further shows shared/continuable session state. missing for 10: independent/hands-on confirmation of a live takeover mid-task, and explicit description of an in-editor (IDE plugin) takeover UI rather than just CLI/app/web.
- [claimed-docs] “This tutorial teaches you how to create and run one small task in HumanLayer. You will use the macOS app and Claude Code.”
- [claimed-docs] “This tutorial teaches you how to run one task on a remote machine. You will control the task from app.humanlayer.com on a machine or phone.”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
- [claimed-docs] “Live multiplayer response drafts — Write prompts together with shared text, named cursors, presence, read-only viewing, and draft recovery.”
developerSend follow-up instructions to an active agent session to steer its work without restarting
weight 2 · round to FactoryFactory's API reference explicitly supports creating and driving Droid sessions including sending messages within an active session (factory-docs-13), and the CLI is interactive by nature, implying follow-up prompts are possible without restarting. However, there is no explicit documentation describing mid-task interruption/steering while the agent is actively executing a long-running task (e.g., Missions or droid exec), so the steering-while-running behavior is not directly evidenced. Missing for 10: explicit docs on interrupting/redirecting an in-progress autonomous run, and independent/hands-on confirmation that follow-up messages actually steer ongoing work rather than queue for the next turn.
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
- [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”
HumanLayer's task/session model implies ongoing interaction with agents across a task's lifetime (docs-5) and release notes mention live prompt collaboration during sessions (docs-18), suggesting some capacity to interact with an active session, but there is no explicit documentation of sending mid-session follow-up instructions to steer a running agent without restarting it. missing for 10: explicit docs on injecting new instructions into a live/running session, confirmation the agent incorporates such input without restart, and independent/hands-on verification of this steering behavior.
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
- [claimed-docs] “Live multiplayer response drafts — Write prompts together with shared text, named cursors, presence, read-only viewing, and draft recovery.”
Sandbox execution
developerHave an agent safely execute code and install dependencies inside an isolated sandbox
weight 3 · round drawnFactorynone0/10Evidence describes tiered autonomy, bash mode, and CI/CD execution (droid exec) but never mentions an isolated sandbox environment for code execution or dependency installation; no container/VM isolation is documented.
HumanLayernone0/10HumanLayer docs describe running sessions on remote hosts (cloud VM, workstation, private-network machine) and automation environments, but there is no mention of an isolated/sandboxed execution environment for running code or installing dependencies safely — the host selection is about access/credentials, not isolation guarantees. Missing for full/partial: any explicit sandbox, container, or isolation mechanism; no evidence of dependency-install safety controls.
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
Human oversight — keeping a human in the loop — approvals, checkpoints, interruptsHuman oversight
Keeping a human in the loop — approvals, checkpoints, interrupts
Approval controls
developerConfigure an agent to auto-approve all its actions instead of confirming each one
weight 2 · round to FactoryFactory's docs confirm 'tiered autonomy' in droid exec that controls what operations run without manual confirmation, implying a configurable auto-approve mode for CI/batch use, but there's no explicit documentation of a full 'auto-approve all actions' toggle or its exact configuration options/flags. missing for 10: explicit config syntax/flag for full auto-approval, independent confirmation of behavior, coverage of auto-approve in interactive (non-exec) sessions.
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
HumanLayer's core premise is human-in-the-loop approval, and docs mention workflow phases like 'Oneshot' for low-risk changes and automation sessions (humanlayer automation run) that run non-interactively without confirmation, implying some auto-approve capability exists, but no explicit documentation of a configurable 'auto-approve all actions' toggle or setting is shown. missing for 10: explicit config/flag to disable per-action confirmation entirely, documentation confirming automation sessions skip all human review rather than just running unattended, and independent confirmation this works as intended.
- [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
product-managerApprove key agent decisions from my phone while agents continue working
weight 1 · round to HumanLayerDocs mention delegating tasks, reviewing diffs, and merging 'from the App or your terminal' and tiered autonomy that gates operations needing confirmation, implying some human-in-the-loop review outside the terminal, but there is no explicit evidence of a mobile/phone app or of approving in-flight agent decisions remotely while agents keep working. Missing for 10: explicit mobile app/phone interface, evidence of asynchronous approval while agent continues running, and independent confirmation of this workflow.
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
Docs explicitly describe controlling and continuing a running agent task from a phone via app.humanlayer.com, with tasks providing a shared review/comment history across devices, directly matching the phone-approval-while-agent-continues story; community sentiment corroborates the human-approval-of-agent-actions use case. Missing for 10: a hands-on/independent account specifically confirming the mobile approval UI in practice, and explicit documentation of an 'approve/deny decision' action (vs. general task control) on mobile.
- [claimed-docs] “This tutorial teaches you how to run one task on a remote machine. You will control the task from app.humanlayer.com on a machine or phone.”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [community] “"I feel much more comfortable with senior developers approving AI changes to our codebase, then letting loose an autonomous agent with no hu…”
engineering-leadSet tiered autonomy levels controlling what an agent can do without manual confirmation
weight 3 · round to FactoryFactory explicitly documents 'tiered autonomy' in droid exec to control which operations run without manual confirmation, directly matching the story, but this is scoped to the CI/CD-oriented droid exec mode rather than a broader, configurable set of autonomy tiers across all agent surfaces. missing for 10: detail on specific tier levels/permissions, configuration UI or granular controls, and evidence this applies uniformly across App/CLI sessions, not just droid exec.
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
HumanLayer's workflow-phases doc shows tiered approaches (Oneshot for low-risk changes vs. more review for unclear/risky changes) and skills-workflows lets users select Oneshot/RPI/PRD-Oriented/Freeform modes, which map to different levels of autonomy vs. oversight. However, there's no explicit documentation of a formal 'autonomy level' setting per agent/task with configurable confirmation thresholds, and no independent evidence confirming this tiered control works as an oversight mechanism in practice. missing for 10: explicit named autonomy-tier configuration (e.g., low/medium/high) tied to confirmation gating, evidence of engineering-lead-level policy controls across a team, and independent/hands-on validation that these workflow phases actually reduce unnecessary confirmations without sacrificing safety.
- [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
- [claimed-docs] “Select **Oneshot**, **RPI**, **PRD-Oriented**, or **Freeform**.”
- [claimed-docs] “HumanLayer registers these RPI sub-agents for Claude Code sessions”
Model control
ai-native userHave each task prompt automatically routed to the most suitable underlying model
weight 2 · round drawnFactorynone0/10No evidence describes automatic routing of prompts to the most suitable underlying model; docs mention model-agnostic droid workflows, custom droids, and orchestration but nothing about auto-selecting models per task.
HumanLayernone0/10Evidence shows HumanLayer lets users manually select or configure which model/backend to use (Claude via Bedrock, Codex, RPI sub-agents) but there is no evidence of automatic routing of a task prompt to the 'most suitable' model based on task characteristics.
- [claimed-docs] “HumanLayer registers these RPI sub-agents for Claude Code sessions”
- [claimed-docs] “This guide shows you how to install, authenticate, select, and check Codex for HumanLayer sessions.”
- [claimed-docs] “HumanLayer sessions can run Claude through Amazon Bedrock instead of the Anthropic API.”
engineering-leadSwitch away from automatic model selection to a specific model of my choice
weight 1 · round to HumanLayerFactorynone0/10No evidence in the pack mentions model selection, automatic model routing, or the ability to choose a specific model over an automatic default; the docs cover CLI usage, integrations, missions, and readiness reports but nothing about model choice controls.
Docs show explicit model/backend selection — choosing Codex ("install, authenticate, select, and check Codex") or running Claude via Amazon Bedrock instead of the Anthropic API, plus subagent model registration — indicating an engineering lead can pick a specific model rather than a default. However, there is no explicit documentation of an 'automatic' default-selection mode being toggled off, so the framing of 'switching away from automatic' isn't directly evidenced. Missing for 10: explicit mention of an automatic/default model-selection setting and a documented UI/CLI flag to override it, independent confirmation of model-switching behavior.
- [claimed-docs] “This guide shows you how to install, authenticate, select, and check Codex for HumanLayer sessions.”
- [claimed-docs] “HumanLayer sessions can run Claude through Amazon Bedrock instead of the Anthropic API.”
- [claimed-docs] “HumanLayer registers these RPI sub-agents for Claude Code sessions”
Visibility monitoring
developerWatch what a running agent is doing in real time, including its current status
weight 3 · round to HumanLayerFactory's session API lets you create and manage Droid sessions—including lifecycle, settings, and messages—implying some ability to track a running agent's state, and the App/CLI let you review diffs as work progresses. However, there is no explicit documentation of a live status dashboard, streaming logs, or real-time progress view of an in-flight agent. Missing for 10: dedicated real-time monitoring UI/stream, explicit 'live status' feature documentation, independent confirmation of live tracking.
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
HumanLayer's docs describe remote-daemon control from app.humanlayer.com (including from a phone), live multiplayer session viewing with presence/read-only viewing, and task/session history — all suggesting real-time visibility into agent activity. However, there's no explicit documentation of a dedicated live status/log stream or dashboard showing granular agent state (e.g., current tool call, progress bar) beyond session/task views. missing for 10: explicit real-time status/log streaming documentation, independent hands-on confirmation of live monitoring UX.
- [claimed-docs] “This tutorial teaches you how to run one task on a remote machine. You will control the task from app.humanlayer.com on a machine or phone.”
- [claimed-docs] “Live multiplayer response drafts — Write prompts together with shared text, named cursors, presence, read-only viewing, and draft recovery.”
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
developerGet notified when an agent completes a task or needs my input
weight 2 · round to HumanLayerFactory integrates with Slack and PagerDuty and supports tiered autonomy that pauses for manual confirmation, implying some mechanism for alerting developers when input is needed, but there's no explicit documentation of a notification feature for task completion or input requests. missing for 10: explicit notification/alert documentation, evidence of completion pings, confirmation of Slack/PagerDuty being used specifically for task-status alerts.
- [claimed-docs] “Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
HumanLayer supports Slack/GitHub/Jira/Linear integrations that push task artifact updates and human-in-the-loop approvals, implying notification when tasks progress or need input, and its core design centers on human oversight of agent work. However, there is no explicit documentation of a dedicated 'task complete' or 'needs input' notification/alert mechanism (e.g., push notification, email, or webhook triggered specifically on completion/approval-required events) beyond generic artifact updates in Slack. missing for 10: explicit completion/needs-input notification docs, evidence of notification latency/reliability, independent/hands-on confirmation of notification delivery.
- [claimed-docs] “Connect Slack to send HumanLayer task artifact updates into the channels where your team already works.”
- [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
- [claimed-docs] “Connect Linear so HumanLayer can create tasks from Linear issues, sync issue status, and link HumanLayer artifacts back to the source issue.”
- [claimed-docs] “Connect Jira Cloud so HumanLayer can create tasks from Jira tickets and keep ticket-driven agent work connected to its source.”
- [community] “"I feel much more comfortable with senior developers approving AI changes to our codebase, then letting loose an autonomous agent with no hu…”
Intent to spec — stories about intent to spec in this arenaIntent to spec
Stories about intent to spec in this arena
Natural language task intake
developerDescribe a feature or bug in plain language and have it automatically turned into a scoped implementation task
weight 3 · round drawnFactory supports delegating tasks described in plain language (via droid CLI, App, or Missions) which get turned into scoped, executable work with diffs to review and merge, and readiness tooling helps scope repo work automatically. However, there's no explicit documented workflow for turning a raw bug/feature description into a formal 'scoped implementation task' spec artifact (e.g., structured ticket/spec generation before execution) distinct from just running the agent. missing for 10: explicit intent-to-spec artifact generation/preview step, independent/hands-on evidence of accurate scoping from vague input, and detail on how ambiguous requests are clarified before execution.
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “delegate scoped tasks to Custom Droids via /droids, and package reusable procedures as Skills with /skills or /create-skill”
- [claimed-docs] “A designer shares a mockup, and the system implements it without handoffs.”
- [claimed-docs] “Run the /readiness-report slash command in a Factory App web or desktop session, or in the Droid CLI, to evaluate a repository's readiness l…”
- [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”
HumanLayer's task model (docs-5) and workflow-phase selection (docs-6, docs-10) show that a task is created and can be routed through 'Oneshot' for small clear changes or heavier RPI/PRD-oriented flows for ambiguous work, and RPI sub-agents (docs-9) imply a research→plan→implement pipeline that turns a description into a scoped plan. However, there is no explicit walkthrough showing a raw plain-language bug/feature description being automatically parsed into a scoped implementation task end-to-end, and integrations (Jira/Linear/GitHub) mostly create tasks from existing tickets rather than free-form language input. Missing for 10: a concrete example or tutorial of plain-language-to-scoped-task conversion, and independent/hands-on confirmation that this pipeline works as described.
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
- [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
- [claimed-docs] “HumanLayer registers these RPI sub-agents for Claude Code sessions”
- [claimed-docs] “Select **Oneshot**, **RPI**, **PRD-Oriented**, or **Freeform**.”
product-managerConvert user feedback submissions into structured tasks with proposed scope
weight 2 · round to HumanLayerFactorynone0/10Factory documents task delegation, Missions planning, and integrations with Jira/Linear/Notion, but nothing in the evidence describes ingesting user feedback submissions and outputting a structured task with proposed scope — this specific PM workflow is not documented.
HumanLayer supports creating tasks from external issue trackers (Jira, Linear, GitHub) which could serve as a proxy for user feedback submissions, and tasks include shared files/scope info, but there's no evidence of a dedicated feature for ingesting raw user feedback (e.g., support tickets, survey responses) and auto-structuring it into a task with a proposed scope specifically tailored for PM workflows. missing for 10: dedicated feedback-ingestion mechanism, explicit 'proposed scope' generation from unstructured feedback text, PM-specific workflow templates, and any hands-on/community validation of this specific use case.
- [claimed-docs] “Connect Jira Cloud so HumanLayer can create tasks from Jira tickets and keep ticket-driven agent work connected to its source.”
- [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
- [claimed-docs] “Connect Linear so HumanLayer can create tasks from Linear issues, sync issue status, and link HumanLayer artifacts back to the source issue.”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
developerAttach a marked-up screenshot or mockup to a task so the agent implements the correct visual change
weight 2 · round to HumanLayerFactorynone0/10Evidence only mentions a designer sharing a mockup conceptually (factory-docs-8), with no documented capability to attach an image/screenshot with markup annotations to a task or session, nor any multimodal image-input feature described.
- [claimed-docs] “A designer shares a mockup, and the system implements it without handoffs.”
Docs confirm images can be pasted into the new task composer as attachments (humanlayer-docs-22), which supports attaching a screenshot to a task, but there is no evidence of markup/annotation tooling or of the agent parsing visual annotations to implement a corresponding UI change. Missing for 10: annotation/markup capability for screenshots, evidence the agent interprets visual markup into a specific implementation, and any hands-on example of this workflow succeeding.
- [claimed-docs] “Image paste in new tasks — Paste images straight into the new task composer as attachments.”
Plan approval
developerReview and approve an agent's implementation plan before any code changes are made
weight 3 · round to HumanLayerFactory's tiered autonomy in droid exec implies some operations require manual confirmation before running, and Missions offer 'plan and execute' orchestration, suggesting a planning phase exists, but there is no explicit documentation of a discrete implementation-plan artifact that a developer reviews and approves before any code changes are made — the described workflow (docs-1) instead centers on reviewing the diff/output after changes. missing for 10: explicit plan-approval UI/step description, evidence of a pre-execution plan artifact, confirmation that no code is touched until plan is approved.
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
- [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
HumanLayer's workflow-phases and RPI sub-agent docs describe planning phases (e.g., 'a change with unclear behavior or code shape needs more review before implementation') and PRD-Oriented/RPI workflows imply a plan stage before code changes, with tasks providing 'one place for comments and review.' However, no evidence explicitly shows a dedicated plan-approval gate/UI step where a developer reviews and approves a plan artifact before implementation begins. missing for 10: explicit documentation of a plan-approval step/UI, first-party example of blocking implementation until plan is approved, independent/hands-on confirmation of this specific gate.
- [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
- [claimed-docs] “HumanLayer registers these RPI sub-agents for Claude Code sessions”
- [claimed-docs] “Select **Oneshot**, **RPI**, **PRD-Oriented**, or **Freeform**.”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
engineering-leadApprove a task's scope and contract before an agent is allowed to modify the repository
weight 2 · round to HumanLayerFactory documents tiered autonomy that limits what operations run without manual confirmation and diff review before merge, which implies some human-gate mechanics, but there's no explicit evidence of an engineering-lead approving a task's defined scope/contract *before* the agent is permitted to touch the repository — reviews are framed as post-hoc diff review rather than pre-execution scope sign-off. Missing for 10: explicit scope/contract definition step, an approval gate that blocks agent execution until lead sign-off, and evidence this is lead-specific rather than generic autonomy tiering.
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
HumanLayer's workflow-phases doc explicitly supports scoping review before implementation (e.g., 'a change with unclear behavior or code shape needs more review before implementation'), and tasks/sessions provide a structured place for comments and review prior to agent execution, plus human-in-the-loop approval is core to the product's value prop per community discussion. However, there's no explicit documented feature for an engineering-lead specifically approving a 'scope and contract' artifact as a gating step before repo modification — it's inferred from general workflow-phase and review mechanics rather than a dedicated scope-approval gate. missing for 10: explicit documentation of a formal scope/contract approval step tied to lead sign-off, evidence of blocking repo writes until such approval, and independent/hands-on confirmation this gate works as intended.
- [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
- [claimed-docs] “Select **Oneshot**, **RPI**, **PRD-Oriented**, or **Freeform**.”
- [community] “"I feel much more comfortable with senior developers approving AI changes to our codebase, then letting loose an autonomous agent with no hu…”
Ticket driven tasking
developerAssign a coding task to an agent directly from an existing issue or ticket
weight 3 · round to HumanLayerFactory documents integrations with Jira and Linear ('Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems'), implying tickets can feed into agent workflows, but there is no explicit documentation of assigning/triggering a droid task directly from an issue or ticket UI. missing for 10: explicit workflow/screenshot showing task creation from a Jira/Linear ticket, confirmation that ticket assignment auto-triggers an agent session, independent corroboration.
- [claimed-docs] “Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems”
Docs explicitly describe connecting GitHub, Jira, and Linear so HumanLayer creates tasks directly from issues/tickets and links artifacts back to the source, directly matching the story of assigning agent work from an existing ticket. Missing for 10: independent/hands-on confirmation that this ticket-to-task flow works reliably in practice, and more detail on the actual assignment UX.
- [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
- [claimed-docs] “Connect Jira Cloud so HumanLayer can create tasks from Jira tickets and keep ticket-driven agent work connected to its source.”
- [claimed-docs] “Connect Linear so HumanLayer can create tasks from Linear issues, sync issue status, and link HumanLayer artifacts back to the source issue.”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
Openness — open source, data portability, and self-hosting storiesOpenness
Open source, data portability, and self-hosting stories
ai-native userDo everything through the API that I can do in the UI
weight 2 · round to FactoryFactory documents a real API (session creation/lifecycle) alongside its CLI and UI, showing some functional overlap, but there's no OpenAPI spec (404s on all candidate URLs) and no evidence that UI-exclusive features like Missions, Software Factory's coverage map, or readiness-report/fix slash commands are exposed via the API. missing for 10: documented API parity for Missions, Software Factory, and readiness commands; a public OpenAPI/schema confirming full surface coverage.
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
- [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”
- [claimed-docs] “Open Software Factory to see your delivery lifecycle as an automation coverage map.”
- [claimed-docs] “Run the /readiness-report slash command in a Factory App web or desktop session, or in the Droid CLI, to evaluate a repository's readiness l…”
- [claimed-docs] “Automatically fix failing criteria with the /readiness-fix slash command in a Factory App web or desktop session, or in the Droid CLI”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.factory.ai/openapi.json, https://docs.factory.ai/swagger.json, https://docs.factory.ai/…”
HumanLayer offers a CLI (`humanlayer automation run`) and remote daemon controls that let you launch and manage sessions outside the UI, but there is no documented public API/OpenAPI spec (both openapi.json and llms.txt probes 404), and UI-only features like live multiplayer drafts, keyboard navigation, and image paste have no CLI/API equivalent documented. missing for 10: a documented REST/GraphQL API or OpenAPI spec, confirmation that all UI actions (draft PRs, multiplayer editing, metrics views) are exposed programmatically, and independent verification of API-UI parity.
- [probe] “official CLI documented at https://docs.humanlayer.com/guide/remote-daemons”
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
- [probe] “PROBE llms.txt: HTTP 404 at https://docs.humanlayer.com/llms.txt”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.humanlayer.com/openapi.json, https://docs.humanlayer.com/swagger.json, https://docs.hum…”
- [claimed-docs] “Live multiplayer response drafts — Write prompts together with shared text, named cursors, presence, read-only viewing, and draft recovery.”
- [claimed-docs] “Keyboard navigation for changed files — Move through the PR changes tree with J/K, N/P, G shortcuts, and Enter.”
ai-native userExport all of my data in open formats and leave
weight 3 · round drawnFactorynone0/10No evidence addresses data export or portability in open formats, nor any account-deletion/data-takeout mechanism; the docs focus on session management, CLI, and integrations, not exporting user data to leave the platform.
ai-native userRead the product's source under an open license
weight 2 · round drawnFactorynone0/10No evidence of an open-source license or publicly available source code for Factory/Droid; all evidence points to closed docs and a proprietary CLI/platform. missing for 10: any license file, GitHub repo, or open-source statement covering the product's source code.
ai-native userSelf-host the core product
weight 3 · round drawnFactorynone0/10Factory is presented as a cloud-hosted platform (Factory App, Droid CLI connecting to hosted services, API sessions) with no evidence of a self-hostable core server or on-prem deployment option anywhere in the docs or probes.
HumanLayernone0/10Evidence shows HumanLayer's daemon/agent execution can run on a user-controlled host (cloud VM, workstation, private network), but the core control plane is explicitly tied to the hosted app.humanlayer.com service — no docs describe self-hosting that core product. missing for 10: no self-hosted control-plane/server option, no on-prem deployment guide, no Docker/Helm chart or license for running the full stack independently.
- [claimed-docs] “This tutorial teaches you how to run one task on a remote machine. You will control the task from app.humanlayer.com on a machine or phone.”
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
Pricing limits — free-tier ceilings, usage caps, and rate limits before you have to payPricing limits
Free-tier ceilings, usage caps, and rate limits before you have to pay
Enterprise licensing
engineering-leadLicense an enterprise deployment with SSO and commercial support for organization-wide rollout
weight 2 · round drawnFactorynone0/10No evidence pack items mention enterprise licensing, SSO, or commercial support offerings; documentation only covers product features like Droid CLI, MCP integration, and readiness reports. Missing for 10: any mention of SSO/SAML, enterprise tier, commercial support SLAs, or org-wide licensing terms.
Model flexibility
engineering-leadBring my own LLM or API key so agents run on the model of my choice
weight 2 · round to HumanLayerFactorynone0/10No evidence pack item mentions bringing your own LLM, custom API keys, or model selection/configuration options; all docs focus on CLI, integrations, and workflow features.
Docs show HumanLayer sessions can use different backends/models — Claude Code, OpenAI Codex, and Claude via Amazon Bedrock instead of the Anthropic API — indicating some flexibility in model/provider choice, which implies bringing your own credentials for these paths. However, there is no explicit doc describing a generic 'bring your own API key' mechanism for arbitrary LLMs or a pricing-tier note tying this to cost savings for engineering leads. Missing for 10: explicit BYO-API-key configuration docs, support for arbitrary/third-party model providers beyond Claude/Codex/Bedrock, and any pricing-related messaging about cost control via own keys.
- [claimed-docs] “This guide shows you how to install, authenticate, select, and check Codex for HumanLayer sessions.”
- [claimed-docs] “HumanLayer sessions can run Claude through Amazon Bedrock instead of the Anthropic API.”
- [claimed-docs] “HumanLayer registers these RPI sub-agents for Claude Code sessions”
Usage quotas
engineering-leadSee and manage plan-based daily task and concurrency limits for agent workflows
weight 2 · round drawnFactorynone0/10No evidence of plan-based daily task limits, concurrency caps, or admin controls for managing such limits; docs cover CLI, sessions, missions, and integrations but nothing on usage/plan limit visibility or management.
HumanLayernone0/10The evidence pack has no mention of plan-based daily task/concurrency limits or any admin controls for managing such limits; only a vague reference to 'Advanced Metrics for all paid plans' which covers usage/cost/productivity viewing, not concurrency or daily task limits management.
- [claimed-docs] “Advanced Metrics for all paid plans — View usage, cost, and productivity metrics, with optional access for every organization member.”
Privacy posture — data-handling and privacy storiesPrivacy posture
Data-handling and privacy stories
ai-native userChoose where my data is stored (region/residency)
weight 2 · round drawnFactorynone0/10No evidence in the pack addresses data residency, regional storage options, or compliance/data-location controls; all evidence covers agent workflows, CLI, and integrations unrelated to data storage location.
ai-native userPrevent my data from being used to train AI models
weight 3 · round drawnFactorynone0/10No evidence in the pack addresses data usage/training opt-out, privacy policy, or data retention controls; the docs focus entirely on product features like CLI, missions, and integrations.
ai-native userControl data retention and deletion
weight 2 · round drawnFactorynone0/10No evidence pack items address data retention policies, deletion controls, or privacy settings for user data; all citations relate to product features like CLI, MCP integrations, and agent workflows rather than privacy posture.
ai-native userOpt out of telemetry and usage tracking
weight 2 · round drawnFactorynone0/10No evidence in the pack mentions telemetry, usage tracking, data collection, or any opt-out/privacy settings for Factory or the Droid CLI.
HumanLayernone0/10No evidence in the pack addresses telemetry, usage tracking, or opt-out controls for HumanLayer; docs cover integrations, workflows, and CLI usage but nothing about privacy/telemetry settings. Missing for 10: any mention of telemetry collection, opt-out mechanism, privacy policy, or data-handling documentation.
Repo integration — stories about repo integration in this arenaRepo integration
Stories about repo integration in this arena
Chat integration
developerTag an agent in a chat thread to discuss and delegate a bug or task
weight 2 · round to FactoryFactory's docs mention connecting Slack as an integration to 'keep development synchronized with team systems,' implying some chat-based interaction, but there is no explicit evidence of an @-mention/tagging mechanism in a chat thread to discuss or delegate a specific bug/task to the agent. Missing for 10: explicit Slack @droid tagging workflow, thread-based task delegation UI, and confirmation that discussion happens inline in chat rather than just triggering external actions.
- [claimed-docs] “Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems”
HumanLayernone0/10HumanLayer's Slack integration only pushes task-artifact updates into channels (docs-4) and other integrations (GitHub, Jira, Linear) create tasks from tickets/issues, not from tagging an agent inside a chat thread. There is no evidence of a chat-native @mention or in-thread delegation workflow for discussing/assigning tasks to an agent.
- [claimed-docs] “Connect Slack to send HumanLayer task artifact updates into the channels where your team already works.”
- [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
- [claimed-docs] “Connect Jira Cloud so HumanLayer can create tasks from Jira tickets and keep ticket-driven agent work connected to its source.”
- [claimed-docs] “Connect Linear so HumanLayer can create tasks from Linear issues, sync issue status, and link HumanLayer artifacts back to the source issue.”
Knowledge context
developerAdd a context file describing my codebase conventions so agents generate more relevant plans and code
weight 3 · round drawnFactorynone0/10The evidence pack does not mention any context file mechanism (e.g., AGENTS.md, .factory config, or similar) for describing codebase conventions to guide agent behavior; it covers CLI usage, integrations, readiness reports, and missions but nothing about persistent repo-convention context files.
HumanLayernone0/10The docs describe workspace-level config files (workspace.json/workspace.local.json) for team/machine settings and multi-repo setup, but there is no evidence of a dedicated context file for describing codebase conventions to improve agent-generated plans/code.
- [claimed-docs] “Use `.humanlayer/workspace.json` for shared repository or team configuration. Use `.humanlayer/workspace.local.json` for optional user or ma…”
- [claimed-docs] “Configure a multi-repository workspace with this repository, ../api, and ../web. Make ../web the primary repository. Ask me before you choos…”
Project management integration
product-managerConnect issue trackers like Jira, Linear, ClickUp, or Monday.com so agents can manage tickets directly
weight 3 · round to HumanLayerDocs confirm Jira and Linear integration explicitly (factory-docs-4), but ClickUp and Monday.com are not mentioned anywhere in the evidence, and there's no detail on ticket management workflows (creating/updating tickets) beyond 'connecting' systems to sync development. Missing for 10: ClickUp integration, Monday.com integration, concrete ticket-management/CRUD workflows via these connectors, independent corroboration.
- [claimed-docs] “Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems”
HumanLayer documents native Jira Cloud and Linear integrations that create tasks from tickets and sync/link status back to source issues, directly matching the story for those trackers. However, there is no evidence of ClickUp or Monday.com integrations, so the story is only partially delivered. Missing for 10: ClickUp integration docs, Monday.com integration docs, independent/hands-on verification of ticket sync working in practice.
- [claimed-docs] “Connect Jira Cloud so HumanLayer can create tasks from Jira tickets and keep ticket-driven agent work connected to its source.”
- [claimed-docs] “Connect Linear so HumanLayer can create tasks from Linear issues, sync issue status, and link HumanLayer artifacts back to the source issue.”
Version control integration
developerConnect a GitHub repository so an agent can access the code and open pull requests against it
weight 3 · round to HumanLayerDocs indicate Factory works within Git workflows (delegate tasks, review diffs, merge from App/terminal) and can run in CI/CD via droid exec, implying repo access and PR-opening capability, but there is no explicit documentation describing connecting/authorizing a GitHub repository or an explicit PR-creation feature. Missing for 10: explicit GitHub repo connection/auth flow documentation, explicit 'open pull request' feature description, and independent/hands-on confirmation of PR creation.
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
Docs describe connecting GitHub to create tasks from issues and link artifacts back (humanlayer-docs-16), plus agent sessions can access code via configured workspaces/repos (humanlayer-docs-8, humanlayer-docs-11) and open draft PRs directly from the session UI (humanlayer-docs-19). missing for 10: independent/hands-on corroboration of the GitHub connection flow and PR-opening working end-to-end beyond first-party docs.
- [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
- [claimed-docs] “Draft PR creation — Ask the session agent to open a draft pull request from the GitHub tab or diff view.”
- [claimed-docs] “Use `.humanlayer/workspace.json` for shared repository or team configuration. Use `.humanlayer/workspace.local.json` for optional user or ma…”
- [claimed-docs] “Configure a multi-repository workspace with this repository, ../api, and ../web. Make ../web the primary repository. Ask me before you choos…”
developerGrant an agent access to my repositories with a one-click install, without complex setup
weight 2 · round drawnFactorynone0/10Evidence shows Factory integrates with Git workflows, CLI, and external tools like Jira/Slack/MCP, but there is no mention of a one-click GitHub/repo install or simplified OAuth-based repo authorization flow. Setup appears to require CLI installation and configuration rather than a one-click grant.
HumanLayernone0/10Docs describe GitHub/Jira/Linear integrations for creating tasks from issues, but connecting a repo requires selecting a host, configuring workspace.json/workspace.local.json, and setting up remote daemons or multi-repo workspaces — none of this is framed as a one-click, no-setup install. No evidence pack item claims or demonstrates a one-click repo access flow.
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [claimed-docs] “Use `.humanlayer/workspace.json` for shared repository or team configuration. Use `.humanlayer/workspace.local.json` for optional user or ma…”
- [claimed-docs] “Configure a multi-repository workspace with this repository, ../api, and ../web. Make ../web the primary repository. Ask me before you choos…”
- [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
Review quality gates — quality gates on changes — review flow, required checks, merge protectionReview quality gates
Quality gates on changes — review flow, required checks, merge protection
Ci remediation
engineering-leadHave failed CI workflows automatically diagnosed and fixed with a proposed pull request
weight 3 · round to HumanLayerFactory's droid exec is explicitly designed to run in CI/CD pipelines as a one-shot task with tiered autonomy, and Droid CLI integrates with Git workflows, suggesting the building blocks exist for automating CI fixes. However, there is no direct evidence of a dedicated feature that detects a failed CI workflow, diagnoses the failure, and automatically opens a proposed pull request end-to-end. missing for 10: explicit CI-failure detection/trigger integration, automatic diagnosis-to-PR workflow documentation, and any hands-on/independent proof of this specific use case.
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
HumanLayer supports running automation sessions from CI (`humanlayer automation run`) and can open draft PRs from a session's diff/GitHub tab, and GitHub integration links tasks to issues—so the building blocks for a CI-triggered fix-and-PR flow exist. However there is no direct evidence of automatic diagnosis of failed CI logs/errors or a documented end-to-end 'CI failure → agent diagnosis → PR' pipeline. Missing for 10: explicit CI-failure-detection/diagnosis workflow docs, example of a failing pipeline auto-triggering a session, and confirmation the resulting PR addresses the CI failure specifically.
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Draft PR creation — Ask the session agent to open a draft pull request from the GitHub tab or diff view.”
- [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
developerTrigger an agent from CI/CD pipelines to fix a broken build or failing test
weight 2 · round to FactoryFactory explicitly documents `droid exec` as a one-shot CLI command designed for CI/CD pipelines, shell scripts, and batch processing, with tiered autonomy controls for unattended operation — directly enabling triggering an agent from CI to fix builds/tests. Missing for 10: a concrete worked example of a CI pipeline invoking droid exec on a failing test/build, and independent/hands-on corroboration beyond vendor docs.
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
- [probe] “official CLI documented at https://docs.factory.ai/droid-cli/quickstart”
HumanLayer docs explicitly describe `humanlayer automation run` for running a Cloud-visible coding session 'from any automation environment — a CI job, a cron machine, or a script on a server,' plus launch tokens for non-interactive command execution, directly supporting CI/CD-triggered agent runs. However, there is no explicit example or integration guide showing a CI failure (broken build/failing test) triggering the agent to diagnose and fix it, nor independent/hands-on confirmation of this workflow. missing for 10: a concrete CI/CD pipeline example tied to build/test failures, evidence of automatic failure detection triggering the agent, and independent verification of this automation flow working in practice.
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
Diff review
developerConfigure an agent to automatically open a pull request when its task completes
weight 2 · round to HumanLayerDocs show git workflow integration (droid-cli git workflow, delegate task then review diff and merge, droid exec for CI/CD pipelines) which implies PR-related automation, but there is no explicit documentation of a feature to automatically open a pull request when a task completes. Missing for 10: explicit config/flag for auto-PR creation, first-party example of a droid opening a PR on completion, independent/hands-on confirmation.
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
Docs confirm HumanLayer can open a draft pull request from a session (humanlayer-docs-19) and integrates with GitHub for issue-linked tasks (humanlayer-docs-16), but the evidence describes PR creation as a manual 'ask the session agent' action from the UI, not an automatic trigger fired upon task completion. Missing for 10: explicit configuration option/workflow setting for automatic PR creation on task completion, and any evidence of it happening without a manual prompt.
- [claimed-docs] “Draft PR creation — Ask the session agent to open a draft pull request from the GitHub tab or diff view.”
- [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
developerReview a diff of an agent's changes and approve it before it becomes a pull request
weight 3 · round to HumanLayerDocs explicitly describe the delegate-review-merge workflow ('Delegate a task, review the diff, and merge from the App or your terminal') and tiered autonomy controls that gate what runs without confirmation, supporting a review-before-PR gate. However, there is no detailed documentation of the diff review UI itself, approval mechanics, or independent/hands-on confirmation of this exact flow. Missing for 10: dedicated diff-review UI documentation, explicit approval-step mechanics, and independent verification of the review-before-PR gate.
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
Release notes explicitly reference a diff view and 'draft PR creation' workflow (docs-19, docs-21), and tasks include 'One place for comments and review' (docs-5), indicating a review-before-PR mechanism. However, there's no detailed documentation of an explicit approve/reject gate tied specifically to diff review prior to PR creation, and community discussion raises concerns about automation bias in approvals (comm-3) without disputing the core capability. Missing for 10: explicit step-by-step approval workflow docs, independent hands-on verification of the diff-review-then-approve flow, and clarity on how rejection/edits are handled before PR creation.
- [claimed-docs] “Draft PR creation — Ask the session agent to open a draft pull request from the GitHub tab or diff view.”
- [claimed-docs] “Keyboard navigation for changed files — Move through the PR changes tree with J/K, N/P, G shortcuts, and Enter.”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
- [community] “"I feel much more comfortable with senior developers approving AI changes to our codebase, then letting loose an autonomous agent with no hu…”
- [community] “User raised concern about automation bias: once an agent proves reliable, humans may rubber-stamp approvals without careful review, letting …”
Pr review automation
ai-native userHave incoming issues automatically triaged with severity suggested and routed to the right owner
weight 2 · round drawnFactorynone0/10Factory documents integrations with issue trackers like Jira, Linear, and PagerDuty (factory-docs-4) and generic custom droid/skill automation (factory-docs-5), but there is no evidence of an automated triage workflow that suggests severity or routes issues to an owner. missing for 10: any documented triage feature, severity classification logic, or owner-routing capability.
- [claimed-docs] “Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems”
- [claimed-docs] “delegate scoped tasks to Custom Droids via /droids, and package reusable procedures as Skills with /skills or /create-skill”
HumanLayernone0/10HumanLayer connects issue trackers (Jira, GitHub, Linear) to create tasks from tickets, but there is no evidence of automatic triage, severity classification, or routing to an owner — integrations only create/link tasks, not assess or assign severity/ownership.
- [claimed-docs] “Connect Jira Cloud so HumanLayer can create tasks from Jira tickets and keep ticket-driven agent work connected to its source.”
- [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
- [claimed-docs] “Connect Linear so HumanLayer can create tasks from Linear issues, sync issue status, and link HumanLayer artifacts back to the source issue.”
engineering-leadHave every pull request automatically reviewed with AI-generated inline comments
weight 3 · round drawnFactorynone0/10Evidence shows Factory's droid CLI/exec can review diffs, be triggered in CI/CD, and connect to Git workflows, but nothing in the pack describes automatic PR review with AI-generated inline comments posted to pull requests. Missing for 10: any documentation of automated PR-triggered review, inline comment generation on PRs, or GitHub/GitLab PR integration specifics.
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
HumanLayernone0/10HumanLayer's evidence covers task/session management, human-in-the-loop approvals, draft PR creation, and a PR diff-viewing UI, but there is no mention of an automated review process that generates inline review comments on every pull request. Missing for 10: no evidence of automatic PR review triggers, no mention of AI-generated inline comments, no review-quality-gate CI integration for PRs.
- [claimed-docs] “Draft PR creation — Ask the session agent to open a draft pull request from the GitHub tab or diff view.”
- [claimed-docs] “Keyboard navigation for changed files — Move through the PR changes tree with J/K, N/P, G shortcuts, and Enter.”
- [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
Readiness checks
engineering-leadRun a readiness report that evaluates how ready my repository is for autonomous agents
weight 2 · round to FactoryFactory explicitly documents a `/readiness-report` slash command (in App or Droid CLI) that evaluates a repository's agent-readiness level, plus a companion `/readiness-fix` command to auto-remediate failing criteria, directly matching the engineering-lead's story. Missing for 10: independent/hands-on validation of report contents and no sample report output shown.
- [claimed-docs] “Run the /readiness-report slash command in a Factory App web or desktop session, or in the Droid CLI, to evaluate a repository's readiness l…”
- [claimed-docs] “Automatically fix failing criteria with the /readiness-fix slash command in a Factory App web or desktop session, or in the Droid CLI”
- [claimed-docs] “A designer shares a mockup, and the system implements it without handoffs.”
Security remediation
engineering-leadHave security alerts automatically validated and remediated with an opened pull request
weight 2 · round drawnFactorynone0/10Evidence covers task delegation, CLI automation, readiness reports/fixes, and CI/CD execution, but nothing describes ingesting security alerts (e.g., from a scanner like Dependabot/Snyk) and automatically validating and remediating them via an opened PR. No security-alert-specific workflow or integration is documented.
HumanLayernone0/10HumanLayer's docs show generic task creation from GitHub/Jira/Linear issues and draft PR creation from agent sessions, but there is no evidence of any security-alert-specific validation or automated vulnerability remediation workflow (no CVE, dependency-alert, or security-scanner integration mentioned).
Scale parallelism — running many jobs at once — concurrency, fleets, queueingScale parallelism
Running many jobs at once — concurrency, fleets, queueing
Concurrent execution
engineering-leadRun many agent tasks concurrently to scale delivery throughput
weight 3 · round to FactoryFactory provides several building blocks for scaling agent work: droid exec is designed for one-shot, CI/CD, and batch processing use cases, the Sessions API lets you create and manage multiple Droid sessions programmatically, and Missions support orchestrating large multi-feature projects with structured coordination. Together these imply the ability to run many concurrent tasks, but no evidence explicitly states a documented concurrency limit, dashboard for tracking many simultaneous droids, or independent case study proving throughput scaling. Missing for 10: explicit concurrency/parallelism guarantees or limits, a multi-task monitoring UI description, and independent/hands-on validation of running many tasks simultaneously.
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
- [claimed-docs] “Open Software Factory to see your delivery lifecycle as an automation coverage map.”
Docs describe an architecture (tasks/sessions, multi-repository workspaces, remote daemons on cloud VMs, and a CLI 'automation run' for CI/cron/scripts) that could support running many agent tasks in parallel, and 'Advanced Metrics' track usage/productivity across an org. However, tutorials and guides are framed around running one task/session at a time, and there is no explicit documentation or evidence of concurrent multi-task orchestration, throughput dashboards, or scaling guarantees for many simultaneous agents. Missing for 10: explicit multi-session concurrency docs, evidence of parallel task orchestration at scale, and independent confirmation of throughput gains.
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [claimed-docs] “Configure a multi-repository workspace with this repository, ../api, and ../web. Make ../web the primary repository. Ask me before you choos…”
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Advanced Metrics for all paid plans — View usage, cost, and productivity metrics, with optional access for every organization member.”
engineering-leadCreate agent sessions on behalf of other users in my organization
weight 2 · round drawnFactorynone0/10The API reference (factory-docs-13) shows session creation/management exists, but nothing in the evidence indicates an org-admin or lead can create/manage sessions on behalf of other specific users within an organization. missing for 10: evidence of org-level user impersonation, admin controls for delegating sessions to teammates, or role-based session ownership across users.
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
HumanLayernone0/10The evidence describes tasks, sessions, integrations, and org-wide metrics access but never mentions an engineering-lead or admin creating/assigning agent sessions on behalf of another named user in the organization. No account-delegation, impersonation, or 'assign session to teammate' capability is documented.
Deployment flexibility
developerUse a managed cloud offering to run agents without operating my own backend infrastructure
weight 2 · round to FactoryFactory's docs describe a hosted 'Factory App' (web/desktop) where users delegate tasks and review diffs without local setup, plus an API for creating and managing Droid sessions server-side, indicating a managed backend rather than self-hosted infra. Missing for 10: independent/hands-on confirmation of the cloud service's scalability and infrastructure details beyond first-party docs.
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “Run the /readiness-report slash command in a Factory App web or desktop session, or in the Droid CLI, to evaluate a repository's readiness l…”
- [claimed-docs] “Automatically fix failing criteria with the /readiness-fix slash command in a Factory App web or desktop session, or in the Droid CLI”
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
HumanLayer offers a cloud control plane (app.humanlayer.com, automation sessions marked 'Cloud-visible', advanced metrics for paid plans) that lets users monitor and manage agent runs remotely, suggesting a managed service layer. However, docs explicitly state the actual agent execution still runs on a host that the user must select and operate—'a cloud VM, workstation, or private-network machine'—meaning the developer still needs to provision/manage backend compute for the daemon rather than HumanLayer fully hosting execution. Missing for 10: evidence of a fully HumanLayer-operated compute backend (no user-managed VM/daemon required), and independent confirmation of a true zero-ops managed runtime.
- [claimed-docs] “This tutorial teaches you how to run one task on a remote machine. You will control the task from app.humanlayer.com on a machine or phone.”
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
- [claimed-docs] “Advanced Metrics for all paid plans — View usage, cost, and productivity metrics, with optional access for every organization member.”
engineering-leadSelf-host agent infrastructure locally, in containers, or on my own VMs
weight 2 · round to HumanLayerFactorynone0/10No evidence describes self-hosting Factory's agent infrastructure locally, in containers, or on customer-owned VMs; all evidence points to Factory's own cloud-hosted App, CLI, and API rather than a deployable/self-hosted backend.
Docs describe running the remote daemon on a cloud VM, workstation, or private-network machine that you control (humanlayer-docs-7), plus automation sessions from CI/cron/server environments (humanlayer-docs-12) and launch tokens for bootstrap scripts (humanlayer-docs-13), showing self-hosted deployment flexibility. However there is no explicit mention of container/Docker deployment or an official container image, and no independent verification of self-hosted setups at scale. Missing for 10: explicit container/Docker packaging docs, independent hands-on confirmation of self-hosted deployments.
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [claimed-docs] “This tutorial teaches you how to run one task on a remote machine. You will control the task from app.humanlayer.com on a machine or phone.”
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
Headless automation
developerRun an agent headlessly inside CI/CD pipelines and shell scripts
weight 2 · round to FactoryFactory explicitly documents droid exec as a one-shot headless command designed for CI/CD pipelines, shell scripts, and batch processing, with tiered autonomy controls for unattended operation. Missing for 10: independent/hands-on confirmation of real-world CI pipeline usage beyond first-party docs.
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
Docs explicitly describe `humanlayer automation run` for running Cloud-visible coding sessions from CI jobs, cron machines, or scripts, plus launch tokens for non-interactive/non-PTY execution suited to headless environments. This directly matches running an agent headlessly in CI/CD and shell scripts. Missing for 10: independent/hands-on verification of CI usage and concrete pipeline examples (e.g. GitHub Actions config).
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
Not comparable on these axes
ai-native userGet AI-generated insights and suggestions from my data inside the product
weight 2 · not comparableFactoryn/aFactory is an agentic coding platform focused on delegating software development tasks (code diffs, PRs, readiness reports), not a data analytics or BI product that surfaces AI-generated insights/suggestions from a user's own data. This story is a category error for this product type.
HumanLayer's agents do generate task artifacts, draft PRs, and comments derived from a user's codebase/tickets, and 'Advanced Metrics' surfaces usage/cost/productivity data, which loosely resembles data-derived insight. However there is no evidence of dedicated AI-generated analytical insights or proactive suggestions distinct from executing assigned coding tasks. Missing for 10: explicit insight/analytics generation from user data, proactive suggestion features beyond task execution, and any first-party or community evidence of an 'insights' capability.
- [claimed-docs] “Advanced Metrics for all paid plans — View usage, cost, and productivity metrics, with optional access for every organization member.”
- [claimed-docs] “Draft PR creation — Ask the session agent to open a draft pull request from the GitHub tab or diff view.”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
developerQuery generated documentation for any public or private repository
weight 1 · not comparableFactorynone0/10The evidence pack covers Factory's CLI, agent-readiness reports, missions, and integrations, but nothing describes generating or querying documentation for a repository's codebase (e.g., an auto-generated repo wiki/docs feature). This is a plausible capability for a repo-integrated dev tool, so the axis applies, but no supporting evidence exists.
engineering-leadAutomatically fix failing agent-readiness criteria in my repository
weight 1 · not comparableFactory explicitly documents a /readiness-fix slash command that automatically fixes failing agent-readiness criteria, usable from the Factory App or Droid CLI, complementing the /readiness-report diagnostic command. This directly matches the story's ask, though only first-party docs support it with no independent/hands-on corroboration. Missing for 10: independent or hands-on verification that /readiness-fix reliably resolves criteria, and detail on which criteria types it can/cannot auto-fix.
- [claimed-docs] “Run the /readiness-report slash command in a Factory App web or desktop session, or in the Droid CLI, to evaluate a repository's readiness l…”
- [claimed-docs] “Automatically fix failing criteria with the /readiness-fix slash command in a Factory App web or desktop session, or in the Droid CLI”
HumanLayern/aHumanLayer is a human-in-the-loop agent orchestration/approval tool for running coding tasks, not a repository readiness/compliance scanner with auto-remediation of 'agent-readiness criteria'. This axis is a category error for this product type — no evidence pack content relates to detecting or auto-fixing repo readiness criteria.