YYLO vs HumanLayer

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round drawn
   YYLO—none0/10

   Direct probes show no llms.txt (404), no docs.md machine-readable variant (404), and no OpenAPI spec (404), meaning there is no agent-oriented docs endpoint to point an agent at; the only docs are standard human-facing HTML pages.

   • [probe] “PROBE llms.txt: HTTP 404 at https://yylo.dev/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://yylo.dev/docs/yylo.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://yylo.dev/openapi.json, https://yylo.dev/swagger.json, https://yylo.dev/api/openapi.json, htt…”

   proof ↗⚑ flag

   HumanLayer—none0/10

   A direct probe for llms.txt returned 404, and no evidence pack item mentions agent-oriented documentation formats or an alternative machine-readable docs endpoint.

   • [probe] “PROBE llms.txt: HTTP 404 at https://docs.humanlayer.com/llms.txt”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to HumanLayer
   YYLO✓fullprobed7/10

   YYLO is a CLI orchestrator with scriptable commands (init, task start, ledger, loop, parallel-runner, run-until-completion) that produce structured watch receipts with exit codes and JSON state, and it's installable via npm as a public package with documented bins (yylo, yy, ypl) suitable for CI invocation. missing for 10: no explicit CI pipeline example (e.g., GitHub Actions config), no independent hands-on report of running it headlessly in CI, and llms.txt/openapi probes returned 404 suggesting thinner machine-readable integration docs.

   • [claimed-docs] “npm install -g @yylo/cli@0.2.2 yylo init --task "Describe one verifiable outcome" --subagent claude”
   • [claimed-docs] “`yy loop` repeats arbitrary shell commands sequentially.”
   • [claimed-docs] “Bounded concurrent fan-out for independent kanban tasks, data items, or complete commands—with structured evidence for every item.”
   • [claimed-docs] “Repeat bounded YYLO iterations until no open kanban work remains.”
   • [github] “A successful run prints the installed YYLO version, initializes `.juno_task/`, then emits a watch receipt with `"state":"COMPLETED"`, `"exit…”
   • [github] “YYLO is a command-line orchestrator for coding agents, repeatable workflows, and receipt-backed repository changes. It is for developers who…”
   • [probe] “PROBE runtime (recorded 2026-09-14): `npm view @yylo/cli name version bin` — the official CLI is live on the public npm registry (@yylo/cli …”

   proof ↗⚑ flag

   HumanLayer✓fullprobed8/10

   Docs explicitly describe running HumanLayer headlessly via `humanlayer automation run` from CI jobs, cron machines, or scripts, plus launch tokens for non-interactive/non-PTY environments, and remote daemon hosts on cloud VMs or servers, directly matching the CI/automation story. Missing for 10: independent/hands-on confirmation of CI usage and more detail on auth/config specifics for automated pipelines.

   • [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
   • [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
   • [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
   • [probe] “official CLI documented at https://docs.humanlayer.com/guide/remote-daemons”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round drawn
   YYLO—none0/10

   YYLO's docs describe orchestrating coding agents, ledgers, workflows, and an 'open-standard path' for skills, but no evidence anywhere mentions MCP servers or a mechanism to plug in MCP tools for the agents it orchestrates.

   ⚑ flag

   HumanLayer—none0/10

   No evidence anywhere in the pack mentions MCP servers or the ability to plug external tool servers into HumanLayer; integrations mentioned (Jira, Slack, GitHub, Linear) are task-source connectors, not MCP tool servers. Missing for 10: any mention of MCP protocol support, MCP server configuration, or tool-plugin mechanism.

   ⚑ flag

4. ai-native userConnect an agent via an official MCP server

   weight 3 · round drawn
   YYLO—none0/10

   YYLO is a CLI orchestrator for coding agents/workflows, and as such platform-type product it could plausibly ship an official MCP server for other agents to connect to, but no evidence pack item mentions MCP at all (only 'open-standard' skills installation, ledger, workflow-runner, etc.). No official MCP server documentation, endpoint, or announcement exists in the evidence.

   ⚑ flag

   HumanLayer—none0/10

   HumanLayer is a platform/control-plane for running and overseeing coding-agent sessions (Claude Code, Codex) rather than itself being an agent, so an official MCP server is a fair, applicable axis. The evidence pack documents many integrations (Jira, Slack, GitHub, Linear), a CLI, and remote daemons, but no MCP server offering is mentioned anywhere, and API/OpenAPI probes returned 404s. missing for 10: any first-party MCP server documentation, endpoint, or 'mcp serve' style capability.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.humanlayer.com/openapi.json, https://docs.humanlayer.com/swagger.json, https://docs.hum…”
   • [probe] “PROBE llms.txt: HTTP 404 at https://docs.humanlayer.com/llms.txt”
   • [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”

   proof ↗⚑ flag

5. ai-native userUse an official CLI

   weight 2 · round to YYLO
   YYLO✓fullprobed8/10

   YYLO ships an official CLI (@yylo/cli) with documented commands (init, ledger, loop, doctor, task start, merge land) and is verifiably published on npm plus MIT-licensed on GitHub, matching an AI-native agentic workflow tool. missing for 10: independent third-party usage reports/reviews beyond vendor docs and registry probes, and some llms.txt/docs-md/openapi endpoints 404 suggesting incomplete machine-readable doc surface.

   • [claimed-docs] “npm install -g @yylo/cli@0.2.2 yylo init --task "Describe one verifiable outcome" --subagent claude”
   • [claimed-docs] “Start hydrates an exact-base task worktree. Preflight is read-only, finish queues the committed candidate”
   • [claimed-docs] “yy ledger create "Validate recovery" --status todo --tags auth yy ledger ready --sort asc yy ledger get TASK_ID”
   • [github] “YYLO is a command-line orchestrator for coding agents, repeatable workflows, and receipt-backed repository changes. It is for developers who…”
   • [github] “`task start` freezes the protected target SHA, creates a dedicated branch/worktree, and completes configured dependency hydration before rep…”
   • [probe] “PROBE runtime (recorded 2026-09-14): `npm view @yylo/cli name version bin` — the official CLI is live on the public npm registry (@yylo/cli …”
   • [probe] “PROBE runtime (recorded 2026-09-14): `curl -s https://raw.githubusercontent.com/yylo-dev/yylo/HEAD/LICENSE | head -3` returns 'MIT License /…”

   proof ↗⚑ flag

   HumanLayer✓fullprobed7/10

   HumanLayer documents an official CLI (e.g. `humanlayer automation run`, launch tokens, remote-daemon control) used for CI, cron, and scripted agentic workflows, confirmed by a dedicated CLI doc page. missing for 10: no independent/hands-on verification of the CLI, no full command reference, and no evidence of broader CLI feature parity with the app.

   • [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
   • [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
   • [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
   • [probe] “official CLI documented at https://docs.humanlayer.com/guide/remote-daemons”

   proof ↗⚑ flag

6. ai-native userDrive the product through a documented public API

   weight 3 · round to YYLO
   YYLO~partialprobed5/10

   YYLO exposes a well-documented CLI (`yylo`, `yy`, `ypl`) with structured JSON output, a ledger/task API, YAML workflow contracts, and templating for chaining steps, which an AI-native user could script against — supported by first-party docs and a live public npm package. However, there is no true public HTTP/OpenAPI-style API: explicit probes for llms.txt, docs.md, and OpenAPI specs all returned 404, meaning the only 'API' is the CLI surface, not a documented network API a remote agent could call directly. Missing for 10: an OpenAPI/REST API spec, an llms.txt or machine-readable API manifest, and evidence of remote/programmatic (non-CLI) invocation.

   • [claimed-docs] “npm install -g @yylo/cli@0.2.2 yylo init --task "Describe one verifiable outcome" --subagent claude”
   • [claimed-docs] “`yy loop` repeats arbitrary shell commands sequentially.”
   • [claimed-docs] “Save the same contract as YAML for a reusable workflow.”
   • [claimed-docs] “yy ledger create "Validate recovery" --status todo --tags auth yy ledger ready --sort asc yy ledger get TASK_ID”
   • [probe] “PROBE llms.txt: HTTP 404 at https://yylo.dev/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://yylo.dev/docs/yylo.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://yylo.dev/openapi.json, https://yylo.dev/swagger.json, https://yylo.dev/api/openapi.json, htt…”
   • [probe] “PROBE runtime (recorded 2026-09-14): `npm view @yylo/cli name version bin` — the official CLI is live on the public npm registry (@yylo/cli …”

   proof ↗⚑ flag

   HumanLayer~partialprobed4/10

   HumanLayer documents a CLI (`humanlayer automation run`, launch tokens, remote daemons) that lets automation environments drive sessions programmatically, which is a form of documented programmatic control, but there is no evidence of a documented public REST/OpenAPI API — probes explicitly found openapi.json/swagger.json/llms.txt all 404. missing for 10: a documented HTTP/OpenAPI public API spec, SDK/client library docs, and independent confirmation of API usage.

   • [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
   • [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
   • [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.humanlayer.com/openapi.json, https://docs.humanlayer.com/swagger.json, https://docs.hum…”
   • [probe] “PROBE llms.txt: HTTP 404 at https://docs.humanlayer.com/llms.txt”
   • [probe] “official CLI documented at https://docs.humanlayer.com/guide/remote-daemons”

   proof ↗⚑ flag

7. ai-native userBuild against official SDKs

   weight 2 · round drawn
   YYLO—none0/10

   YYLO ships an official CLI (@yylo/cli) and open MIT-licensed repo, but there is no evidence of an SDK (client library/API) to build against — OpenAPI/spec probes and llms.txt/docs-md endpoints all return 404, and no SDK docs are mentioned anywhere in the pack.

   • [probe] “PROBE llms.txt: HTTP 404 at https://yylo.dev/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://yylo.dev/docs/yylo.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://yylo.dev/openapi.json, https://yylo.dev/swagger.json, https://yylo.dev/api/openapi.json, htt…”
   • [probe] “PROBE runtime (recorded 2026-09-14): `npm view @yylo/cli name version bin` — the official CLI is live on the public npm registry (@yylo/cli …”
   • [probe] “PROBE runtime (recorded 2026-09-14): `curl -s https://raw.githubusercontent.com/yylo-dev/yylo/HEAD/LICENSE | head -3` returns 'MIT License /…”

   proof ↗⚑ flag

   HumanLayer—none0/10

   The evidence pack covers HumanLayer's CLI, workspace config, and third-party integrations (Jira, Slack, GitHub, Linear) but contains no mention of an official SDK (Python, TypeScript, etc.) for building against HumanLayer programmatically. Probes for openapi.json and llms.txt both returned 404, further suggesting no discoverable API/SDK surface.

   • [probe] “PROBE llms.txt: HTTP 404 at https://docs.humanlayer.com/llms.txt”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.humanlayer.com/openapi.json, https://docs.humanlayer.com/swagger.json, https://docs.hum…”
   • [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
   • [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”

   proof ↗⚑ flag

8. ai-native userSubscribe to events via webhooks

   weight 2 · round drawn
   YYLO—none0/10

   No evidence of any webhook subscription mechanism; YYLO documents Slack/GitHub integrations pulling into kanban but nothing about outbound event webhooks, and API/openapi probes returned 404s.

   • [claimed-docs] “Bring Slack messages or GitHub issues into kanban and return completed responses to their source thread.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://yylo.dev/openapi.json, https://yylo.dev/swagger.json, https://yylo.dev/api/openapi.json, htt…”

   proof ↗⚑ flag

   HumanLayer—none0/10

   No evidence pack item mentions webhooks or event subscription mechanisms; integrations described (Slack, Jira, Linear, GitHub) are inbound task-creation connectors, not outbound webhook events, and API/OpenAPI probes returned 404s.

   • [probe] “PROBE llms.txt: HTTP 404 at https://docs.humanlayer.com/llms.txt”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.humanlayer.com/openapi.json, https://docs.humanlayer.com/swagger.json, https://docs.hum…”

   proof ↗⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round to HumanLayer
   YYLO—none0/10

   YYLO is an orchestration/CLI tool for running coding agents, workflows, and task ledgers; it launches subagents to perform tasks but nothing in the evidence describes it generating analytic insights or suggestions from a user's own data. Merge/validation explicitly avoids invoking models (yylo-gh-6), and no docs mention dashboards, analytics, or AI-generated insight surfacing.

   • [github] “Tests and semantic reviews are explicit project checks outside merge. Merge launches no models, chooses no reviewers, schedules no suites, a…”
   • [claimed-docs] “npm install -g @yylo/cli@0.2.2 yylo init --task "Describe one verifiable outcome" --subagent claude”
   • [claimed-docs] “yy ledger create "Validate recovery" --status todo --tags auth yy ledger ready --sort asc yy ledger get TASK_ID”

   proof ↗⚑ flag

   HumanLayer~partialclaimed4/10

   HumanLayer's agents do generate task artifacts, draft PRs, and comments derived from a user's codebase/tickets, and 'Advanced Metrics' surfaces usage/cost/productivity data, which loosely resembles data-derived insight. However there is no evidence of dedicated AI-generated analytical insights or proactive suggestions distinct from executing assigned coding tasks. Missing for 10: explicit insight/analytics generation from user data, proactive suggestion features beyond task execution, and any first-party or community evidence of an 'insights' capability.

   • [claimed-docs] “Advanced Metrics for all paid plans — View usage, cost, and productivity metrics, with optional access for every organization member.”
   • [claimed-docs] “Draft PR creation — Ask the session agent to open a draft pull request from the GitHub tab or diff view.”
   • [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”

   proof ↗⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to HumanLayer
   YYLO~partialclaimed7/10

   Docs describe multiple mechanisms for autonomous, repeated execution — `yy loop` for repeating shell commands, run-until-completion scripts that iterate until kanban work is done, a workflow-runner for chained multi-step automations, a bounded parallel-runner for concurrent fan-out, and integrations that pull Slack/GitHub work into a kanban queue and post results back. This supports background-style autonomous task execution, and sessions can be resumed via `yy continue SESSION_ID` rather than reconstructed from a terminal. However, there's no evidence of a persistent scheduler/cron-like trigger or a hosted background service — everything appears CLI/session-driven, and there is no independent or hands-on confirmation that these loops truly run unattended over long periods. Missing for 10: evidence of scheduled/triggered automations independent of an active session, and third-party corroboration of long-running unattended execution.

   • [claimed-docs] “`yy loop` repeats arbitrary shell commands sequentially.”
   • [claimed-docs] “Bounded concurrent fan-out for independent kanban tasks, data items, or complete commands—with structured evidence for every item.”
   • [claimed-docs] “Continue a captured session with `yy continue SESSION_ID`; do not reconstruct work from tmux scrollback.”
   • [claimed-docs] “Choose Workflow Runner when a step consumes `{{ steps.<id>.response }}`, a generated file, or a session from an earlier step.”
   • [claimed-docs] “Repeat bounded YYLO iterations until no open kanban work remains.”
   • [claimed-docs] “Bring Slack messages or GitHub issues into kanban and return completed responses to their source thread.”
   • [github] “YYLO is a command-line orchestrator for coding agents, repeatable workflows, and receipt-backed repository changes. It is for developers who…”

   proof ↗⚑ flag

   HumanLayer✓fullclaimed7/10

   Docs explicitly describe `humanlayer automation run` for running Cloud-visible coding sessions from CI jobs, cron machines, or scripts, plus remote daemons and launch tokens for non-interactive/background execution contexts, directly supporting autonomous background automations. Missing for 10: independent/hands-on verification of long-running background automations, native scheduling UI, and clarity on how human-approval gates interact with continuous autonomous runs.

   • [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
   • [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
   • [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
   • [claimed-docs] “Use `.humanlayer/workspace.json` for shared repository or team configuration. Use `.humanlayer/workspace.local.json` for optional user or ma…”

   proof ↗⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round to HumanLayer
   YYLO~partialclaimed4/10

   YYLO's CLI lets a user delegate a task to an AI agent via `--subagent claude` and manage it through kanban/ledger workflows, so task delegation to an AI is documented, but the AI capability is an external subagent (e.g., Claude) invoked by the orchestrator rather than an assistant built into YYLO itself. missing for 10: evidence of an assistant embedded in the product (not requiring an external model/agent), and any first-party assistant UI/API rather than orchestration of third-party agents.

   • [claimed-docs] “npm install -g @yylo/cli@0.2.2 yylo init --task "Describe one verifiable outcome" --subagent claude”
   • [github] “YYLO is a command-line orchestrator for coding agents, repeatable workflows, and receipt-backed repository changes. It is for developers who…”
   • [claimed-docs] “yy ledger create "Validate recovery" --status todo --tags auth yy ledger ready --sort asc yy ledger get TASK_ID”

   proof ↗⚑ flag

   HumanLayer✓fullclaimed7/10

   HumanLayer's core product model is task delegation to AI coding agents (Claude Code, Codex, Bedrock-backed Claude) via 'sessions', with workflow modes (Oneshot, RPI, PRD-Oriented, Freeform) and automation runs from CI/cron, directly matching 'delegate tasks to a built-in AI assistant'. missing for 10: independent/hands-on verification of the delegation experience beyond vendor docs, and clarity on how autonomous vs supervised the assistant is in practice.

   • [claimed-docs] “This tutorial teaches you how to create and run one small task in HumanLayer. You will use the macOS app and Claude Code.”
   • [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
   • [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
   • [claimed-docs] “HumanLayer registers these RPI sub-agents for Claude Code sessions”
   • [claimed-docs] “Select **Oneshot**, **RPI**, **PRD-Oriented**, or **Freeform**.”
   • [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
   • [claimed-docs] “This guide shows you how to install, authenticate, select, and check Codex for HumanLayer sessions.”

   proof ↗⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round to HumanLayer
   YYLO—none0/10

   YYLO's interface is a structured CLI (yylo/yy commands with explicit flags like --task, --subagent) rather than a natural-language command interface; the --task string is passed through to a subagent, not parsed as an operator instruction to YYLO itself. No evidence shows a chat-like or NL command surface for driving YYLO's own operations (init, start, finish, ledger, merge, etc.).

   • [claimed-docs] “npm install -g @yylo/cli@0.2.2 yylo init --task "Describe one verifiable outcome" --subagent claude”
   • [claimed-docs] “Start hydrates an exact-base task worktree. Preflight is read-only, finish queues the committed candidate”
   • [github] “YYLO is a command-line orchestrator for coding agents, repeatable workflows, and receipt-backed repository changes. It is for developers who…”
   • [github] “`task start` freezes the protected target SHA, creates a dedicated branch/worktree, and completes configured dependency hydration before rep…”

   proof ↗⚑ flag

   HumanLayer✓fullclaimed7/10

   HumanLayer's core interaction model is giving natural-language instructions to agent sessions (Claude Code, Codex) to create tasks, configure workspaces, and choose workflow phases, as shown by the example NL workspace-config prompt and workflow-selection docs. Missing for 10: independent/hands-on corroboration of NL command robustness and no evidence of a broader NL command surface beyond task/workflow setup.

   • [claimed-docs] “Configure a multi-repository workspace with this repository, ../api, and ../web. Make ../web the primary repository. Ask me before you choos…”
   • [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
   • [claimed-docs] “Select **Oneshot**, **RPI**, **PRD-Oriented**, or **Freeform**.”
   • [claimed-docs] “This tutorial teaches you how to create and run one small task in HumanLayer. You will use the macOS app and Claude Code.”
   • [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   YYLO—none0/10

   YYLO is a CLI orchestrator for coding agents, not an API product; there is no evidence of an interactive API reference, and explicit probes for openapi.json/swagger.json and llms.txt all return 404, indicating no such reference exists.

   • [probe] “PROBE llms.txt: HTTP 404 at https://yylo.dev/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://yylo.dev/docs/yylo.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://yylo.dev/openapi.json, https://yylo.dev/swagger.json, https://yylo.dev/api/openapi.json, htt…”

   proof ↗⚑ flag

   HumanLayer—none0/10

   No evidence of an interactive API reference or runnable examples; probes explicitly show no OpenAPI/swagger spec and no llms.txt found, and docs are guide/tutorial style rather than an API reference sandbox.

   • [probe] “PROBE llms.txt: HTTP 404 at https://docs.humanlayer.com/llms.txt”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.humanlayer.com/openapi.json, https://docs.humanlayer.com/swagger.json, https://docs.hum…”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   YYLO—none0/10

   Explicit probes for llms.txt, docs-md, and OpenAPI/swagger endpoints all returned 404, and no evidence shows any downloadable machine-readable API spec; YYLO is a CLI orchestrator without a documented API surface to spec.

   • [probe] “PROBE llms.txt: HTTP 404 at https://yylo.dev/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://yylo.dev/docs/yylo.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://yylo.dev/openapi.json, https://yylo.dev/swagger.json, https://yylo.dev/api/openapi.json, htt…”

   proof ↗⚑ flag

   HumanLayer—none0/10

   A direct probe for OpenAPI/swagger specs at all standard locations returned 404s, and no evidence pack item shows a downloadable machine-readable API spec being offered.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.humanlayer.com/openapi.json, https://docs.humanlayer.com/swagger.json, https://docs.hum…”
   • [probe] “PROBE llms.txt: HTTP 404 at https://docs.humanlayer.com/llms.txt”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round to YYLO
   YYLO✓fullclaimed7/10

   YYLO's task worktrees isolate work from the protected target SHA, preflight checks are documented as read-only, and merges compose changes into a private detached candidate rather than touching the live branch directly, while the benchmark tool explicitly runs 'isolated attempts' with 'recover safely' semantics—together these describe a sandboxed testing flow that avoids touching the protected/production state. Missing for 10: explicit 'production data' terminology or a dedicated staging/prod environment concept, and independent (non-vendor) confirmation that isolation holds up in practice.

   • [claimed-docs] “Start hydrates an exact-base task worktree. Preflight is read-only, finish queues the committed candidate”
   • [github] “`task start` freezes the protected target SHA, creates a dedicated branch/worktree, and completes configured dependency hydration before rep…”
   • [github] “`doctor workspace` is intentionally nonzero when it finds an actionable topology problem; it never fetches or changes the workspace.”
   • [github] “`merge land` selects one immutable task source, composes in a private detached candidate, and uses Git expected-old ref protection. A moved …”
   • [claimed-docs] “Plan immutable experiments, execute isolated attempts, retain evaluator provenance, recover safely, and produce bounded reports.”

   proof ↗⚑ flag

   HumanLayer—none0/10

   HumanLayer's docs describe remote daemons, workspaces, and automation sessions, but there is no mention of a sandbox environment, staging/test data isolation, or any mechanism to separate test runs from production data. missing for 10: explicit sandbox/staging environment documentation, data isolation guarantees, evidence of test-vs-production separation.

   ⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   YYLO—none0/10

   No evidence of any versioned API contract or documented deprecation policy; the product is a CLI orchestrator with a version like 0.2.1rc6, but no API versioning scheme or deprecation guarantees are mentioned, and openapi/llms.txt probes returned 404.

   • [probe] “PROBE openapi: all candidate paths 404 (https://yylo.dev/openapi.json, https://yylo.dev/swagger.json, https://yylo.dev/api/openapi.json, htt…”
   • [probe] “PROBE llms.txt: HTTP 404 at https://yylo.dev/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://yylo.dev/docs/yylo.md”
   • [claimed-docs] “The 0.2.1rc6 channel adds ID-first general Records and typed task, wiki, workflow, and artifact profiles.”

   proof ↗⚑ flag

   HumanLayer—none0/10

   No evidence of API versioning scheme or a documented deprecation policy; probes for openapi.json/llms.txt returned 404s and no API reference or changelog covering versioning/deprecation is present.

   • [probe] “PROBE llms.txt: HTTP 404 at https://docs.humanlayer.com/llms.txt”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.humanlayer.com/openapi.json, https://docs.humanlayer.com/swagger.json, https://docs.hum…”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to YYLO
   YYLO✓fullprobed8/10

   YYLO's parallel-runner explicitly supports 'bounded concurrent fan-out for independent kanban tasks, data items, or complete commands—with structured evidence for every item,' and run-until-completion repeats iterations until all kanban work is done, directly enabling bulk operations across many items with automated evidence capture. This is corroborated by a live public CLI (npm registry, MIT-licensed source), though there's no independent hands-on report of large-scale fan-out in practice. Missing for 10: independent/third-party validation of bulk-scale runs and concrete concurrency limits or throughput numbers.

   • [claimed-docs] “Bounded concurrent fan-out for independent kanban tasks, data items, or complete commands—with structured evidence for every item.”
   • [claimed-docs] “Repeat bounded YYLO iterations until no open kanban work remains.”
   • [claimed-docs] “Continue a captured session with `yy continue SESSION_ID`; do not reconstruct work from tmux scrollback.”
   • [probe] “PROBE runtime (recorded 2026-09-14): `npm view @yylo/cli name version bin` — the official CLI is live on the public npm registry (@yylo/cli …”
   • [probe] “PROBE runtime (recorded 2026-09-14): `curl -s https://raw.githubusercontent.com/yylo-dev/yylo/HEAD/LICENSE | head -3` returns 'MIT License /…”

   proof ↗⚑ flag

   HumanLayer—none0/10

   HumanLayer's documentation consistently frames work as single tasks/sessions ('one small task', 'one task on a remote machine', 'one Cloud-visible coding session') with per-task review and approval workflows; there is no mention of batch/bulk operations spanning many items at once. missing for 10: any documented bulk-action API/CLI flag, batch approval mechanism, or multi-item automation workflow.

   • [claimed-docs] “This tutorial teaches you how to create and run one small task in HumanLayer. You will use the macOS app and Claude Code.”
   • [claimed-docs] “This tutorial teaches you how to run one task on a remote machine. You will control the task from app.humanlayer.com on a machine or phone.”
   • [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
   • [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to HumanLayer
   YYLO~partialclaimed5/10

   YYLO's integrations pull Slack messages and GitHub issues into its kanban system and return responses to source threads, which is a form of event-triggered automation, and run-until-completion/workflow-runner allow chained/looping actions. However there's no evidence of a general user-defined rule engine (conditions + custom triggers) — the event handling is limited to specific hardcoded integrations rather than an open rule-definition system. Missing for 10: a documented rules/conditions DSL, support for arbitrary custom event sources/triggers, and evidence of user-authored trigger logic beyond the built-in Slack/GitHub integrations.

   • [claimed-docs] “Bring Slack messages or GitHub issues into kanban and return completed responses to their source thread.”
   • [claimed-docs] “Repeat bounded YYLO iterations until no open kanban work remains.”
   • [claimed-docs] “Choose Workflow Runner when a step consumes `{{ steps.<id>.response }}`, a generated file, or a session from an earlier step.”

   proof ↗⚑ flag

   HumanLayer~partialclaimed6/10

   HumanLayer integrations (GitHub, Jira, Linear) create tasks automatically from external events like new issues/tickets, and `humanlayer automation run` lets sessions be triggered from CI jobs, cron, or scripts — both are forms of event-driven automation. However, there's no evidence of a general-purpose rules/conditions engine letting users define arbitrary trigger-condition-action logic; the automation is limited to fixed integration hooks and script-based invocation. Missing for 10: a documented rule-definition interface (conditions, filters, custom triggers) beyond fixed platform integrations, and independent confirmation these event-triggers work reliably in practice.

   • [claimed-docs] “Connect Jira Cloud so HumanLayer can create tasks from Jira tickets and keep ticket-driven agent work connected to its source.”
   • [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
   • [claimed-docs] “Connect Linear so HumanLayer can create tasks from Linear issues, sync issue status, and link HumanLayer artifacts back to the source issue.”
   • [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
   • [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round to YYLO
   YYLO~partialclaimed5/10

   YYLO supports repeatable workflows (YAML-saved contracts, `yy loop` shell repetition, run-until-completion looping, workflow-runner chaining) which cover automation-depth, but there is no evidence of a scheduler, cron-like trigger, or time-based recurring job execution — workflows appear to be manually invoked or run-to-completion rather than scheduled on a recurring basis. missing for 10: cron/schedule syntax or trigger mechanism, evidence of unattended recurring execution (e.g., daily/hourly runs), and independent confirmation of scheduling in production use.

   • [claimed-docs] “`yy loop` repeats arbitrary shell commands sequentially.”
   • [claimed-docs] “Save the same contract as YAML for a reusable workflow.”
   • [claimed-docs] “Choose Workflow Runner when a step consumes `{{ steps.<id>.response }}`, a generated file, or a session from an earlier step.”
   • [claimed-docs] “Repeat bounded YYLO iterations until no open kanban work remains.”
   • [github] “YYLO is a command-line orchestrator for coding agents, repeatable workflows, and receipt-backed repository changes. It is for developers who…”

   proof ↗⚑ flag

   HumanLayer~partialclaimed4/10

   HumanLayer's `automation run` CLI lets you trigger a Cloud-visible coding session from a cron machine or CI job, implying recurring runs are possible via external schedulers, but there is no documented native scheduling/recurrence feature (no cron syntax, interval config, or job queue) inside HumanLayer itself. missing for 10: built-in recurring scheduler, interval/cron configuration options, evidence of persistent recurring workflow management.

   • [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
   • [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”

   proof ↗⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round drawn
   YYLO~partialclaimed5/10

   YYLO ledger provides task versioning-like constructs (immutable task source, worktree branching, candidate composition, expected-old ref protection for merges) and workflow YAML contracts can be saved/reused, giving some review/rollback semantics via git-based merge protections. However there is no explicit 'automation version history', diff/review UI, or a documented rollback command for automations themselves beyond git branch/worktree mechanics. missing for 10: dedicated automation versioning/history view, explicit rollback command for a run/automation (not just git ref protection), review workflow UI, and independent/hands-on confirmation of these rollback flows.

   • [github] “`task start` freezes the protected target SHA, creates a dedicated branch/worktree, and completes configured dependency hydration before rep…”
   • [github] “`merge land` selects one immutable task source, composes in a private detached candidate, and uses Git expected-old ref protection. A moved …”
   • [claimed-docs] “Save the same contract as YAML for a reusable workflow.”
   • [claimed-docs] “The 0.2.1rc6 channel adds ID-first general Records and typed task, wiki, workflow, and artifact profiles.”

   proof ↗⚑ flag

   HumanLayer~partialclaimed5/10

   HumanLayer's task model provides review (comments, PR draft creation) and history that persists across sessions, giving some review/audit capability, but there is no explicit documentation of versioning workflow definitions or rolling back an automation to a prior version. missing for 10: explicit automation versioning/history diffing, a documented rollback mechanism, and independent confirmation these review features extend to full automation lifecycle management.

   • [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
   • [claimed-docs] “Draft PR creation — Ask the session agent to open a draft pull request from the GitHub tab or diff view.”
   • [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”

   proof ↗⚑ flag

End to end feature delivery

1. ai-native userHave an agent automatically generate and run tests to validate its own code changes before proposing them

   weight 2 · round drawn
   YYLO—none0/10

   The evidence explicitly states that tests and semantic reviews are 'explicit project checks outside merge' and that YYLO's merge step 'launches no models, chooses no reviewers, schedules no suites, and maintains no validation cache' (yylo-gh-6). While `yy loop` can repeat arbitrary shell commands, there is no evidence of an agent autonomously generating tests or validating its own changes before proposing them.

   • [github] “Tests and semantic reviews are explicit project checks outside merge. Merge launches no models, chooses no reviewers, schedules no suites, a…”
   • [claimed-docs] “`yy loop` repeats arbitrary shell commands sequentially.”

   proof ↗⚑ flag

   HumanLayer—none0/10

   No evidence in the pack describes HumanLayer agents automatically writing or executing tests to self-validate code changes; docs cover task/session management, integrations, and workflow phases but not test generation/execution as a validation step.

   ⚑ flag

2. developerHave an agent autonomously diagnose and fix a reported bug

   weight 3 · round to HumanLayer
   YYLO~partialclaimed5/10

   YYLO orchestrates coding agents through task worktrees, kanban-driven work items, and iteration loops (run-until-completion, subagent tasking) that could be pointed at a bug-fix task, and it supports ingesting GitHub issues into kanban as a trigger. However there is no evidence of autonomous bug diagnosis (root-causing, log/trace analysis) as a distinct capability — the docs describe generic task/workflow orchestration and merge/validation boundaries rather than an end-to-end 'diagnose then fix' agent behavior. missing for 10: explicit diagnosis/root-cause capability, an end-to-end bug-fix case study or hands-on validation, evidence the agent itself (vs. the orchestrator) performs debugging reasoning.

   • [claimed-docs] “npm install -g @yylo/cli@0.2.2 yylo init --task "Describe one verifiable outcome" --subagent claude”
   • [claimed-docs] “Repeat bounded YYLO iterations until no open kanban work remains.”
   • [claimed-docs] “Bring Slack messages or GitHub issues into kanban and return completed responses to their source thread.”
   • [github] “YYLO is a command-line orchestrator for coding agents, repeatable workflows, and receipt-backed repository changes. It is for developers who…”
   • [github] “`task start` freezes the protected target SHA, creates a dedicated branch/worktree, and completes configured dependency hydration before rep…”

   proof ↗⚑ flag

   HumanLayer~partialcommunity6/10

   HumanLayer supports creating tasks directly from GitHub/Jira/Linear issues and then running an agent session (Claude Code/Codex) against the linked repo, with an 'Oneshot' workflow phase designed for small, clear changes — a plausible bug-fix pipeline. However, the product's core design is human-in-the-loop with approval gates rather than fully autonomous action, and there's no end-to-end documented example of an agent independently diagnosing a bug from a ticket and shipping a fix without human review. Missing for 10: a concrete autonomous bug-diagnosis-and-fix walkthrough, and clarity on how much human approval is required mid-flow.

   • [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
   • [claimed-docs] “Connect Linear so HumanLayer can create tasks from Linear issues, sync issue status, and link HumanLayer artifacts back to the source issue.”
   • [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
   • [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
   • [community] “"I feel much more comfortable with senior developers approving AI changes to our codebase, then letting loose an autonomous agent with no hu…”

   proof ↗⚑ flag

3. product-managerGo from a mockup or design to a working implementation without an engineering handoff

   weight 2 · round drawn
   YYLO—none0/10

   YYLO's evidence describes a CLI orchestrator for coding agents (task/worktree/merge/ledger workflows) aimed at developers and project operators, with no mention of ingesting mockups/designs or enabling a non-technical PM to go from a design to working code without engineering involvement.

   • [github] “YYLO is a command-line orchestrator for coding agents, repeatable workflows, and receipt-backed repository changes. It is for developers who…”
   • [claimed-docs] “npm install -g @yylo/cli@0.2.2 yylo init --task "Describe one verifiable outcome" --subagent claude”
   • [github] “`task start` freezes the protected target SHA, creates a dedicated branch/worktree, and completes configured dependency hydration before rep…”
   • [github] “`merge land` selects one immutable task source, composes in a private detached candidate, and uses Git expected-old ref protection. A moved …”

   proof ↗⚑ flag

   HumanLayer—none0/10

   HumanLayer's evidence is entirely about developer-facing workflows: running coding sessions, connecting Jira/Linear/GitHub/Slack, managing remote daemons, and CLI automation for engineers overseeing coding agents. Nothing in the pack shows a mockup/design import capability, a no-code interface, or any path for a non-engineer product manager to turn a design into a working implementation without engineering involvement — in fact the workflow language (RPI, PRD-oriented, Oneshot) and workspace configs assume an engineering operator. Missing for evidence of delivering this story: mockup/design ingestion, PM-oriented no-code UI, and any case study of a non-engineer shipping code end-to-end.

   • [claimed-docs] “This tutorial teaches you how to create and run one small task in HumanLayer. You will use the macOS app and Claude Code.”
   • [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
   • [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
   • [claimed-docs] “Select **Oneshot**, **RPI**, **PRD-Oriented**, or **Freeform**.”

   proof ↗⚑ flag

4. developerHave an agent implement a requested feature end-to-end, including writing tests

   weight 3 · round drawn
   YYLO~partialclaimed5/10

   YYLO documents an end-to-end task loop (yylo init --task ... --subagent claude, worktree creation, commit-and-queue candidate flow) showing an agent can implement a described feature autonomously, but the evidence explicitly states 'tests and semantic reviews are explicit project checks outside merge' — meaning YYLO's own agent loop does not itself guarantee test-writing as part of implementation, only that separate check scripts exist for validation. missing for 10: explicit evidence the invoked subagent writes/adds tests as part of a task, and any example showing test-authoring within the init/finish workflow.

   • [claimed-docs] “npm install -g @yylo/cli@0.2.2 yylo init --task "Describe one verifiable outcome" --subagent claude”
   • [claimed-docs] “Start hydrates an exact-base task worktree. Preflight is read-only, finish queues the committed candidate”
   • [github] “YYLO is a command-line orchestrator for coding agents, repeatable workflows, and receipt-backed repository changes. It is for developers who…”
   • [github] “`task start` freezes the protected target SHA, creates a dedicated branch/worktree, and completes configured dependency hydration before rep…”
   • [github] “Tests and semantic reviews are explicit project checks outside merge. Merge launches no models, chooses no reviewers, schedules no suites, a…”

   proof ↗⚑ flag

   HumanLayer~partialclaimed5/10

   HumanLayer clearly supports end-to-end autonomous coding sessions (Oneshot, RPI, PRD-Oriented workflows) that implement tasks using agents like Claude Code and Codex, including structured phases and sub-agents for research/plan/implement, which implies substantial feature work can be delegated (humanlayer-docs-6, humanlayer-docs-9, humanlayer-docs-10, humanlayer-docs-14). However, no evidence explicitly confirms the agent writes or runs tests as part of the workflow, and no hands-on account demonstrates a full feature-plus-tests delivery. Missing for 10: explicit documentation or example showing test generation/execution as part of the implementation flow, and independent verification of end-to-end feature completion including tests.

   • [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
   • [claimed-docs] “HumanLayer registers these RPI sub-agents for Claude Code sessions”
   • [claimed-docs] “Select **Oneshot**, **RPI**, **PRD-Oriented**, or **Freeform**.”
   • [claimed-docs] “This guide shows you how to install, authenticate, select, and check Codex for HumanLayer sessions.”
   • [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”

   proof ↗⚑ flag

Environment setup

1. developerHave an agent automatically clone the repo, install dependencies, and configure its own working environment

   weight 2 · round to YYLO
   YYLO~partialprobed7/10

   Docs and GitHub README describe `task start`/`yy init` hydrating a dedicated worktree from a protected base SHA and completing 'configured dependency hydration' before reporting WORKING, which covers automated environment setup and dependency install; the CLI is confirmed live on npm and the repo public. Missing for 10: explicit description of cloning an arbitrary remote repo (vs. hydrating a pre-defined workspace), and independent/hands-on confirmation that dependency install works end-to-end.

   • [github] “`task start` freezes the protected target SHA, creates a dedicated branch/worktree, and completes configured dependency hydration before rep…”
   • [claimed-docs] “npm install -g @yylo/cli@0.2.2 yylo init --task "Describe one verifiable outcome" --subagent claude”
   • [claimed-docs] “Start hydrates an exact-base task worktree. Preflight is read-only, finish queues the committed candidate”
   • [probe] “PROBE runtime (recorded 2026-09-14): `npm view @yylo/cli name version bin` — the official CLI is live on the public npm registry (@yylo/cli …”
   • [github] “A successful run prints the installed YYLO version, initializes `.juno_task/`, then emits a watch receipt with `"state":"COMPLETED"`, `"exit…”

   proof ↗⚑ flag

   HumanLayer~partialclaimed5/10

   Docs describe workspace configuration (workspace.json, multi-repo setups) and remote hosts that must have access to code/tools/credentials, and one example prompt asks the agent about 'setup commands or local files to copy,' implying some environment configuration ability. However, there is no explicit description of the agent autonomously cloning a repo or installing dependencies end-to-end without human setup of the host/workspace first. Missing for 10: explicit documentation of automatic repo cloning, dependency installation steps, and end-to-end environment bootstrap without prior manual host/workspace configuration.

   • [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
   • [claimed-docs] “Use `.humanlayer/workspace.json` for shared repository or team configuration. Use `.humanlayer/workspace.local.json` for optional user or ma…”
   • [claimed-docs] “Configure a multi-repository workspace with this repository, ../api, and ../web. Make ../web the primary repository. Ask me before you choos…”
   • [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”

   proof ↗⚑ flag

Interactive takeover

1. developerTake over an in-progress agent task in my editor, terminal, or browser to finish or redirect the work

   weight 2 · round to HumanLayer
   YYLO~partialclaimed4/10

   YYLO offers terminal-based session continuation (`yy continue SESSION_ID`) that lets a developer pick back up captured agent work rather than reconstructing it from scrollback, and task start/finish flows expose worktree/branch state that could be inspected or redirected from the CLI. However, there is no evidence of any editor integration or browser UI for taking over tasks — YYLO is documented purely as a CLI/orchestrator tool. Missing for 10: editor plugin/extension support, browser-based task takeover UI, and explicit interactive 'redirect mid-task' semantics beyond resuming a session.

   • [claimed-docs] “Continue a captured session with `yy continue SESSION_ID`; do not reconstruct work from tmux scrollback.”
   • [github] “`task start` freezes the protected target SHA, creates a dedicated branch/worktree, and completes configured dependency hydration before rep…”
   • [github] “YYLO is a command-line orchestrator for coding agents, repeatable workflows, and receipt-backed repository changes. It is for developers who…”
   • [claimed-docs] “Start hydrates an exact-base task worktree. Preflight is read-only, finish queues the committed candidate”

   proof ↗⚑ flag

   HumanLayer✓fullclaimed7/10

   Docs describe tasks with session/history continuity 'across agents and workstations', remote control from app.humanlayer.com on any machine or phone, and CLI-driven remote daemons for terminal/server contexts, all pointing to genuine hand-off of in-progress work between editor (Claude Code), terminal (remote daemon/CLI), and browser (web app). Live multiplayer draft recovery further shows shared/continuable session state. missing for 10: independent/hands-on confirmation of a live takeover mid-task, and explicit description of an in-editor (IDE plugin) takeover UI rather than just CLI/app/web.

   • [claimed-docs] “This tutorial teaches you how to create and run one small task in HumanLayer. You will use the macOS app and Claude Code.”
   • [claimed-docs] “This tutorial teaches you how to run one task on a remote machine. You will control the task from app.humanlayer.com on a machine or phone.”
   • [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
   • [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
   • [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
   • [claimed-docs] “Live multiplayer response drafts — Write prompts together with shared text, named cursors, presence, read-only viewing, and draft recovery.”

   proof ↗⚑ flag

2. developerSend follow-up instructions to an active agent session to steer its work without restarting

   weight 2 · round to HumanLayer
   YYLO—none0/10

   YYLO's docs describe task lifecycle (init/start/preflight/finish) and resuming a captured session via `yy continue SESSION_ID`, but there is no evidence of sending new instructions to an already-running agent session to redirect its work mid-flight; `continue` appears to resume/reattach rather than inject steering input. missing for 10: any documented mechanism for mid-session instruction injection or steering, evidence that an active agent process accepts new prompts without restart.

   • [claimed-docs] “Continue a captured session with `yy continue SESSION_ID`; do not reconstruct work from tmux scrollback.”
   • [claimed-docs] “Start hydrates an exact-base task worktree. Preflight is read-only, finish queues the committed candidate”
   • [github] “`task start` freezes the protected target SHA, creates a dedicated branch/worktree, and completes configured dependency hydration before rep…”

   proof ↗⚑ flag

   HumanLayer~partialclaimed4/10

   HumanLayer's task/session model implies ongoing interaction with agents across a task's lifetime (docs-5) and release notes mention live prompt collaboration during sessions (docs-18), suggesting some capacity to interact with an active session, but there is no explicit documentation of sending mid-session follow-up instructions to steer a running agent without restarting it. missing for 10: explicit docs on injecting new instructions into a live/running session, confirmation the agent incorporates such input without restart, and independent/hands-on verification of this steering behavior.

   • [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
   • [claimed-docs] “Live multiplayer response drafts — Write prompts together with shared text, named cursors, presence, read-only viewing, and draft recovery.”

   proof ↗⚑ flag

Sandbox execution

1. developerHave an agent safely execute code and install dependencies inside an isolated sandbox

   weight 3 · round drawn
   YYLO—none0/10

   YYLO's docs describe git-worktree/branch isolation for tasks and merge safety, but none of the evidence mentions a sandboxed execution environment (container/VM) for running agent code or installing dependencies safely. Worktree isolation protects git state, not runtime/process isolation.

   • [github] “`task start` freezes the protected target SHA, creates a dedicated branch/worktree, and completes configured dependency hydration before rep…”
   • [claimed-docs] “Start hydrates an exact-base task worktree. Preflight is read-only, finish queues the committed candidate”
   • [github] “YYLO is a command-line orchestrator for coding agents, repeatable workflows, and receipt-backed repository changes. It is for developers who…”

   proof ↗⚑ flag

   HumanLayer—none0/10

   HumanLayer docs describe running sessions on remote hosts (cloud VM, workstation, private-network machine) and automation environments, but there is no mention of an isolated/sandboxed execution environment for running code or installing dependencies safely — the host selection is about access/credentials, not isolation guarantees. Missing for full/partial: any explicit sandbox, container, or isolation mechanism; no evidence of dependency-install safety controls.

   • [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
   • [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
   • [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”

   proof ↗⚑ flag

Approval controls

1. developerConfigure an agent to auto-approve all its actions instead of confirming each one

   weight 2 · round to HumanLayer
   YYLO—none0/10

   The evidence describes YYLO's orchestration architecture (task worktrees, ledger, merge protections) but nothing addresses a configurable auto-approve/no-confirmation mode for agent actions; in fact merge land explicitly requires checks and human-oversight-style gating rather than blanket auto-approval. Missing for 10: any documented auto-approve flag/setting, evidence of confirmation prompts being bypassable, or explicit human-oversight configuration options.

   • [github] “`task start` freezes the protected target SHA, creates a dedicated branch/worktree, and completes configured dependency hydration before rep…”
   • [github] “`merge land` selects one immutable task source, composes in a private detached candidate, and uses Git expected-old ref protection. A moved …”
   • [github] “Tests and semantic reviews are explicit project checks outside merge. Merge launches no models, chooses no reviewers, schedules no suites, a…”

   proof ↗⚑ flag

   HumanLayer~partialclaimed4/10

   HumanLayer's core premise is human-in-the-loop approval, and docs mention workflow phases like 'Oneshot' for low-risk changes and automation sessions (humanlayer automation run) that run non-interactively without confirmation, implying some auto-approve capability exists, but no explicit documentation of a configurable 'auto-approve all actions' toggle or setting is shown. missing for 10: explicit config/flag to disable per-action confirmation entirely, documentation confirming automation sessions skip all human review rather than just running unattended, and independent confirmation this works as intended.

   • [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
   • [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
   • [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”

   proof ↗⚑ flag

2. product-managerApprove key agent decisions from my phone while agents continue working

   weight 1 · round to HumanLayer
   YYLO—none0/10

   YYLO is documented purely as a CLI/terminal orchestrator (yylo/yy commands, ledger, worktrees, merge gating) with no mention of any mobile app, phone notification, or remote-approval interface for product managers. The axis is plausible for an agent-orchestration tool but no evidence supports it.

   • [github] “YYLO is a command-line orchestrator for coding agents, repeatable workflows, and receipt-backed repository changes. It is for developers who…”
   • [claimed-docs] “npm install -g @yylo/cli@0.2.2 yylo init --task "Describe one verifiable outcome" --subagent claude”
   • [claimed-docs] “yy ledger create "Validate recovery" --status todo --tags auth yy ledger ready --sort asc yy ledger get TASK_ID”
   • [github] “`task start` freezes the protected target SHA, creates a dedicated branch/worktree, and completes configured dependency hydration before rep…”
   • [github] “`merge land` selects one immutable task source, composes in a private detached candidate, and uses Git expected-old ref protection. A moved …”

   proof ↗⚑ flag

   HumanLayer✓fullcommunity8/10

   Docs explicitly describe controlling and continuing a running agent task from a phone via app.humanlayer.com, with tasks providing a shared review/comment history across devices, directly matching the phone-approval-while-agent-continues story; community sentiment corroborates the human-approval-of-agent-actions use case. Missing for 10: a hands-on/independent account specifically confirming the mobile approval UI in practice, and explicit documentation of an 'approve/deny decision' action (vs. general task control) on mobile.

   • [claimed-docs] “This tutorial teaches you how to run one task on a remote machine. You will control the task from app.humanlayer.com on a machine or phone.”
   • [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
   • [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
   • [community] “"I feel much more comfortable with senior developers approving AI changes to our codebase, then letting loose an autonomous agent with no hu…”

   proof ↗⚑ flag

3. engineering-leadSet tiered autonomy levels controlling what an agent can do without manual confirmation

   weight 3 · round to HumanLayer
   YYLO—none0/10

   Evidence shows preflight/read-only checks, merge protections, and validation boundaries, but there is no mention of configurable tiered autonomy levels or a settings mechanism letting an engineering-lead define graduated confirmation thresholds for agent actions. missing for 10: explicit autonomy-level configuration, tiered permission settings, evidence of user-controlled confirmation thresholds.

   ⚑ flag

   HumanLayer~partialclaimed5/10

   HumanLayer's workflow-phases doc shows tiered approaches (Oneshot for low-risk changes vs. more review for unclear/risky changes) and skills-workflows lets users select Oneshot/RPI/PRD-Oriented/Freeform modes, which map to different levels of autonomy vs. oversight. However, there's no explicit documentation of a formal 'autonomy level' setting per agent/task with configurable confirmation thresholds, and no independent evidence confirming this tiered control works as an oversight mechanism in practice. missing for 10: explicit named autonomy-tier configuration (e.g., low/medium/high) tied to confirmation gating, evidence of engineering-lead-level policy controls across a team, and independent/hands-on validation that these workflow phases actually reduce unnecessary confirmations without sacrificing safety.

   • [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
   • [claimed-docs] “Select **Oneshot**, **RPI**, **PRD-Oriented**, or **Freeform**.”
   • [claimed-docs] “HumanLayer registers these RPI sub-agents for Claude Code sessions”

   proof ↗⚑ flag

Model control

1. ai-native userHave each task prompt automatically routed to the most suitable underlying model

   weight 2 · round drawn
   YYLO—none0/10

   Evidence shows YYLO lets users manually specify a subagent/model (e.g. `--subagent claude`) but nothing describes automatic routing of a task prompt to the 'most suitable' underlying model based on task characteristics. missing for 10: any evidence of automatic model-selection logic, routing criteria, or multi-model comparison/selection mechanism.

   • [claimed-docs] “npm install -g @yylo/cli@0.2.2 yylo init --task "Describe one verifiable outcome" --subagent claude”

   proof ↗⚑ flag

   HumanLayer—none0/10

   Evidence shows HumanLayer lets users manually select or configure which model/backend to use (Claude via Bedrock, Codex, RPI sub-agents) but there is no evidence of automatic routing of a task prompt to the 'most suitable' model based on task characteristics.

   • [claimed-docs] “HumanLayer registers these RPI sub-agents for Claude Code sessions”
   • [claimed-docs] “This guide shows you how to install, authenticate, select, and check Codex for HumanLayer sessions.”
   • [claimed-docs] “HumanLayer sessions can run Claude through Amazon Bedrock instead of the Anthropic API.”

   proof ↗⚑ flag

2. engineering-leadSwitch away from automatic model selection to a specific model of my choice

   weight 1 · round to HumanLayer
   YYLO~partialclaimed5/10

   The CLI init command shows a --subagent flag (e.g. 'claude') letting a user specify which model/agent to use instead of relying on defaults, implying manual model selection is possible; however there is no documentation of an explicit 'automatic model selection' mode being overridden, no list of supported models, and no engineering-lead-oriented control/settings UI shown. missing for 10: explicit documentation of an automatic/default model-selection mode, a full list of selectable models, and confirmation that this override is persistent/configurable at a project or team level.

   • [claimed-docs] “npm install -g @yylo/cli@0.2.2 yylo init --task "Describe one verifiable outcome" --subagent claude”

   proof ↗⚑ flag

   HumanLayer~partialclaimed6/10

   Docs show explicit model/backend selection — choosing Codex ("install, authenticate, select, and check Codex") or running Claude via Amazon Bedrock instead of the Anthropic API, plus subagent model registration — indicating an engineering lead can pick a specific model rather than a default. However, there is no explicit documentation of an 'automatic' default-selection mode being toggled off, so the framing of 'switching away from automatic' isn't directly evidenced. Missing for 10: explicit mention of an automatic/default model-selection setting and a documented UI/CLI flag to override it, independent confirmation of model-switching behavior.

   • [claimed-docs] “This guide shows you how to install, authenticate, select, and check Codex for HumanLayer sessions.”
   • [claimed-docs] “HumanLayer sessions can run Claude through Amazon Bedrock instead of the Anthropic API.”
   • [claimed-docs] “HumanLayer registers these RPI sub-agents for Claude Code sessions”

   proof ↗⚑ flag

Visibility monitoring

1. developerWatch what a running agent is doing in real time, including its current status

   weight 3 · round to HumanLayer
   YYLO~partialclaimed4/10

   The GitHub docs mention a successful run ending in a 'watch receipt' with a state field (e.g. COMPLETED, exit_code, log_bytes), implying some status-tracking mechanism exists, and 'task start' reports a WORKING state before completion. However there is no dedicated 'watch' command, live dashboard, or streaming log evidence showing real-time observation of an in-progress agent's actions. missing for 10: explicit real-time streaming/monitoring command or UI, documentation of live status polling during execution, independent confirmation of watch behavior.

   • [github] “A successful run prints the installed YYLO version, initializes `.juno_task/`, then emits a watch receipt with `"state":"COMPLETED"`, `"exit…”
   • [github] “`task start` freezes the protected target SHA, creates a dedicated branch/worktree, and completes configured dependency hydration before rep…”
   • [claimed-docs] “Continue a captured session with `yy continue SESSION_ID`; do not reconstruct work from tmux scrollback.”

   proof ↗⚑ flag

   HumanLayer~partialclaimed6/10

   HumanLayer's docs describe remote-daemon control from app.humanlayer.com (including from a phone), live multiplayer session viewing with presence/read-only viewing, and task/session history — all suggesting real-time visibility into agent activity. However, there's no explicit documentation of a dedicated live status/log stream or dashboard showing granular agent state (e.g., current tool call, progress bar) beyond session/task views. missing for 10: explicit real-time status/log streaming documentation, independent hands-on confirmation of live monitoring UX.

   • [claimed-docs] “This tutorial teaches you how to run one task on a remote machine. You will control the task from app.humanlayer.com on a machine or phone.”
   • [claimed-docs] “Live multiplayer response drafts — Write prompts together with shared text, named cursors, presence, read-only viewing, and draft recovery.”
   • [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
   • [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”

   proof ↗⚑ flag

2. developerGet notified when an agent completes a task or needs my input

   weight 2 · round to HumanLayer
   YYLO~partialclaimed5/10

   YYLO emits watch receipts with a COMPLETED state/exit_code after a run finishes, and its Slack/GitHub integrations can return completed responses to the originating thread, giving developers a way to learn a task finished. However there's no evidence of a proactive notification for when an agent 'needs input' mid-task, nor any push/alert channel beyond polling receipts or the source-thread reply. Missing for 10: explicit 'needs input' alerting, a dedicated notification/webhook system, and independent confirmation the integration loop works end-to-end.

   • [github] “A successful run prints the installed YYLO version, initializes `.juno_task/`, then emits a watch receipt with `"state":"COMPLETED"`, `"exit…”
   • [claimed-docs] “Bring Slack messages or GitHub issues into kanban and return completed responses to their source thread.”
   • [github] “Tests and semantic reviews are explicit project checks outside merge. Merge launches no models, chooses no reviewers, schedules no suites, a…”

   proof ↗⚑ flag

   HumanLayer~partialcommunity6/10

   HumanLayer supports Slack/GitHub/Jira/Linear integrations that push task artifact updates and human-in-the-loop approvals, implying notification when tasks progress or need input, and its core design centers on human oversight of agent work. However, there is no explicit documentation of a dedicated 'task complete' or 'needs input' notification/alert mechanism (e.g., push notification, email, or webhook triggered specifically on completion/approval-required events) beyond generic artifact updates in Slack. missing for 10: explicit completion/needs-input notification docs, evidence of notification latency/reliability, independent/hands-on confirmation of notification delivery.

   • [claimed-docs] “Connect Slack to send HumanLayer task artifact updates into the channels where your team already works.”
   • [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
   • [claimed-docs] “Connect Linear so HumanLayer can create tasks from Linear issues, sync issue status, and link HumanLayer artifacts back to the source issue.”
   • [claimed-docs] “Connect Jira Cloud so HumanLayer can create tasks from Jira tickets and keep ticket-driven agent work connected to its source.”
   • [community] “"I feel much more comfortable with senior developers approving AI changes to our codebase, then letting loose an autonomous agent with no hu…”

   proof ↗⚑ flag

Natural language task intake

1. developerDescribe a feature or bug in plain language and have it automatically turned into a scoped implementation task

   weight 3 · round drawn
   YYLO~partialclaimed6/10

   YYLO's CLI lets a developer pass a plain-language task string (e.g. `yylo init --task "Describe one verifiable outcome"` or `yy ledger create "Validate recovery"`) which is then hydrated into a dedicated branch/worktree and handed to a subagent (yylo-docs-1, yylo-docs-2, yylo-gh-4, yylo-docs-11). This covers the 'turned into a scoped implementation task' half of the story, but there's no evidence of NLP-based scoping/decomposition logic — the description appears passed through largely as-is rather than analyzed/refined into a structured spec. Missing for 10: evidence of automatic task decomposition or requirement extraction from free-text input, and independent/hands-on confirmation that vague feature/bug descriptions produce well-scoped tasks.

   • [claimed-docs] “npm install -g @yylo/cli@0.2.2 yylo init --task "Describe one verifiable outcome" --subagent claude”
   • [claimed-docs] “Start hydrates an exact-base task worktree. Preflight is read-only, finish queues the committed candidate”
   • [github] “`task start` freezes the protected target SHA, creates a dedicated branch/worktree, and completes configured dependency hydration before rep…”
   • [claimed-docs] “yy ledger create "Validate recovery" --status todo --tags auth yy ledger ready --sort asc yy ledger get TASK_ID”

   proof ↗⚑ flag

   HumanLayer~partialclaimed6/10

   HumanLayer's task model (docs-5) and workflow-phase selection (docs-6, docs-10) show that a task is created and can be routed through 'Oneshot' for small clear changes or heavier RPI/PRD-oriented flows for ambiguous work, and RPI sub-agents (docs-9) imply a research→plan→implement pipeline that turns a description into a scoped plan. However, there is no explicit walkthrough showing a raw plain-language bug/feature description being automatically parsed into a scoped implementation task end-to-end, and integrations (Jira/Linear/GitHub) mostly create tasks from existing tickets rather than free-form language input. Missing for 10: a concrete example or tutorial of plain-language-to-scoped-task conversion, and independent/hands-on confirmation that this pipeline works as described.

   • [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
   • [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
   • [claimed-docs] “HumanLayer registers these RPI sub-agents for Claude Code sessions”
   • [claimed-docs] “Select **Oneshot**, **RPI**, **PRD-Oriented**, or **Freeform**.”

   proof ↗⚑ flag

2. product-managerConvert user feedback submissions into structured tasks with proposed scope

   weight 2 · round drawn
   YYLO~partialprobed4/10

   YYLO can ingest external feedback (Slack messages, GitHub issues) directly into its kanban/ledger as structured tasks (yylo-docs-10), and a dedicated `feedback-yylo` CLI binary is confirmed live on npm (yylo-probe-rt-1), suggesting a feedback-to-task pathway exists. However, there is no documented mechanism for generating a 'proposed scope' alongside the task — no scope estimation, sizing, or planning artifact is described anywhere in the docs or GitHub evidence. Missing for 10: explicit scope-proposal output tied to feedback conversion, documentation of what the feedback-yylo binary actually produces, and any PM-facing evidence of structured scoping fields.

   • [claimed-docs] “Bring Slack messages or GitHub issues into kanban and return completed responses to their source thread.”
   • [probe] “PROBE runtime (recorded 2026-09-14): `npm view @yylo/cli name version bin` — the official CLI is live on the public npm registry (@yylo/cli …”
   • [claimed-docs] “yy ledger create "Validate recovery" --status todo --tags auth yy ledger ready --sort asc yy ledger get TASK_ID”

   proof ↗⚑ flag

   HumanLayer~partialclaimed4/10

   HumanLayer supports creating tasks from external issue trackers (Jira, Linear, GitHub) which could serve as a proxy for user feedback submissions, and tasks include shared files/scope info, but there's no evidence of a dedicated feature for ingesting raw user feedback (e.g., support tickets, survey responses) and auto-structuring it into a task with a proposed scope specifically tailored for PM workflows. missing for 10: dedicated feedback-ingestion mechanism, explicit 'proposed scope' generation from unstructured feedback text, PM-specific workflow templates, and any hands-on/community validation of this specific use case.

   • [claimed-docs] “Connect Jira Cloud so HumanLayer can create tasks from Jira tickets and keep ticket-driven agent work connected to its source.”
   • [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
   • [claimed-docs] “Connect Linear so HumanLayer can create tasks from Linear issues, sync issue status, and link HumanLayer artifacts back to the source issue.”
   • [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”

   proof ↗⚑ flag

Plan approval

1. developerReview and approve an agent's implementation plan before any code changes are made

   weight 3 · round to HumanLayer
   YYLO—none0/10

   Evidence describes YYLO's task lifecycle (init, start, preflight read-only, finish queuing a candidate, merge land) but nothing indicates the agent produces an implementation plan that a developer reviews and approves before any code is written — preflight/checks occur on already-produced work, not a pre-code plan gate.

   • [claimed-docs] “Start hydrates an exact-base task worktree. Preflight is read-only, finish queues the committed candidate”
   • [github] “`task start` freezes the protected target SHA, creates a dedicated branch/worktree, and completes configured dependency hydration before rep…”
   • [github] “`merge land` selects one immutable task source, composes in a private detached candidate, and uses Git expected-old ref protection. A moved …”
   • [github] “Tests and semantic reviews are explicit project checks outside merge. Merge launches no models, chooses no reviewers, schedules no suites, a…”

   proof ↗⚑ flag

   HumanLayer~partialclaimed6/10

   HumanLayer's workflow-phases and RPI sub-agent docs describe planning phases (e.g., 'a change with unclear behavior or code shape needs more review before implementation') and PRD-Oriented/RPI workflows imply a plan stage before code changes, with tasks providing 'one place for comments and review.' However, no evidence explicitly shows a dedicated plan-approval gate/UI step where a developer reviews and approves a plan artifact before implementation begins. missing for 10: explicit documentation of a plan-approval step/UI, first-party example of blocking implementation until plan is approved, independent/hands-on confirmation of this specific gate.

   • [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
   • [claimed-docs] “HumanLayer registers these RPI sub-agents for Claude Code sessions”
   • [claimed-docs] “Select **Oneshot**, **RPI**, **PRD-Oriented**, or **Freeform**.”
   • [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”

   proof ↗⚑ flag

2. engineering-leadApprove a task's scope and contract before an agent is allowed to modify the repository

   weight 2 · round to HumanLayer
   YYLO~partialclaimed4/10

   YYLO documents a task/contract concept (YAML contracts, ledger tasks) and isolates agent work in dedicated branches/worktrees with a read-only preflight before any commit is queued (yylo-docs-2, yylo-docs-4, yylo-gh-4), which implies some spec-first gating. However there is no documented human approval/sign-off step where an engineering-lead explicitly reviews and approves scope/contract before the agent is permitted to start modifying the repo—task start appears automatic once invoked. Missing for 10: explicit lead-approval gate/workflow, evidence of a review UI or command requiring human sign-off, and confirmation that agent modification is blocked pending that approval.

   • [claimed-docs] “Start hydrates an exact-base task worktree. Preflight is read-only, finish queues the committed candidate”
   • [claimed-docs] “Save the same contract as YAML for a reusable workflow.”
   • [github] “`task start` freezes the protected target SHA, creates a dedicated branch/worktree, and completes configured dependency hydration before rep…”
   • [github] “`merge land` selects one immutable task source, composes in a private detached candidate, and uses Git expected-old ref protection. A moved …”

   proof ↗⚑ flag

   HumanLayer~partialcommunity6/10

   HumanLayer's workflow-phases doc explicitly supports scoping review before implementation (e.g., 'a change with unclear behavior or code shape needs more review before implementation'), and tasks/sessions provide a structured place for comments and review prior to agent execution, plus human-in-the-loop approval is core to the product's value prop per community discussion. However, there's no explicit documented feature for an engineering-lead specifically approving a 'scope and contract' artifact as a gating step before repo modification — it's inferred from general workflow-phase and review mechanics rather than a dedicated scope-approval gate. missing for 10: explicit documentation of a formal scope/contract approval step tied to lead sign-off, evidence of blocking repo writes until such approval, and independent/hands-on confirmation this gate works as intended.

   • [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
   • [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
   • [claimed-docs] “Select **Oneshot**, **RPI**, **PRD-Oriented**, or **Freeform**.”
   • [community] “"I feel much more comfortable with senior developers approving AI changes to our codebase, then letting loose an autonomous agent with no hu…”

   proof ↗⚑ flag

Ticket driven tasking

1. developerAssign a coding task to an agent directly from an existing issue or ticket

   weight 3 · round to HumanLayer
   YYLO~partialclaimed6/10

   YYLO integrations pull GitHub issues into its kanban ledger and return completed responses to the source thread, and ledger tasks can then be started with a subagent via task start or yylo init, supporting the flow of turning an issue into an agent task. Missing for 10: a single documented command that directly converts one specific issue into an agent task in one step, and hands-on confirmation the GitHub-issue import works end-to-end.

   • [claimed-docs] “Bring Slack messages or GitHub issues into kanban and return completed responses to their source thread.”
   • [claimed-docs] “npm install -g @yylo/cli@0.2.2 yylo init --task "Describe one verifiable outcome" --subagent claude”
   • [claimed-docs] “yy ledger create "Validate recovery" --status todo --tags auth yy ledger ready --sort asc yy ledger get TASK_ID”
   • [github] “`task start` freezes the protected target SHA, creates a dedicated branch/worktree, and completes configured dependency hydration before rep…”

   proof ↗⚑ flag

   HumanLayer✓fullclaimed8/10

   Docs explicitly describe connecting GitHub, Jira, and Linear so HumanLayer creates tasks directly from issues/tickets and links artifacts back to the source, directly matching the story of assigning agent work from an existing ticket. Missing for 10: independent/hands-on confirmation that this ticket-to-task flow works reliably in practice, and more detail on the actual assignment UX.

   • [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
   • [claimed-docs] “Connect Jira Cloud so HumanLayer can create tasks from Jira tickets and keep ticket-driven agent work connected to its source.”
   • [claimed-docs] “Connect Linear so HumanLayer can create tasks from Linear issues, sync issue status, and link HumanLayer artifacts back to the source issue.”
   • [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”

   proof ↗⚑ flag

1. ai-native userExport all of my data in open formats and leave

   weight 3 · round to YYLO
   YYLO~partialprobed4/10

   Workflow contracts can be saved as YAML (yylo-docs-4) and the CLI/ledger source is fully open-source under MIT (yylo-probe-rt-2), suggesting no vendor lock-in, but there is no documented single 'export all data' command covering ledger records, task history, sessions, and artifacts in open formats. Missing for 10: an explicit data-export command/feature, documentation of export formats for ledger/kanban/session data, and confirmation that all state (not just workflow YAML) is portable.

   • [claimed-docs] “Save the same contract as YAML for a reusable workflow.”
   • [claimed-docs] “The 0.2.1rc6 channel adds ID-first general Records and typed task, wiki, workflow, and artifact profiles.”
   • [probe] “PROBE runtime (recorded 2026-09-14): `curl -s https://raw.githubusercontent.com/yylo-dev/yylo/HEAD/LICENSE | head -3` returns 'MIT License /…”
   • [github] “YYLO is a command-line orchestrator for coding agents, repeatable workflows, and receipt-backed repository changes. It is for developers who…”

   proof ↗⚑ flag

   HumanLayer—none0/10

   No evidence of data export functionality or open-format data portability; docs focus on tasks, workflows, and integrations, with no mention of exporting user data or account deletion/leaving.

   ⚑ flag

2. ai-native userRead the product's source under an open license

   weight 2 · round to YYLO
   YYLO✓fullprobed8/10

   The GitHub repo is public and licensed under MIT, confirmed by a runtime probe reading the LICENSE file directly, and the repo (yylo-dev/yylo) is documented as the CLI orchestrator's source. missing for 10: no independent third-party audit or community commentary confirming completeness of the published source beyond the license file check.

   • [probe] “PROBE runtime (recorded 2026-09-14): `curl -s https://raw.githubusercontent.com/yylo-dev/yylo/HEAD/LICENSE | head -3` returns 'MIT License /…”
   • [github] “YYLO is a command-line orchestrator for coding agents, repeatable workflows, and receipt-backed repository changes. It is for developers who…”

   proof ↗⚑ flag

   HumanLayer—none0/10

   No evidence pack item mentions an open-source license or public source repository for HumanLayer; all evidence is docs, community discussion, or probes about APIs/CLI. Absence of evidence for this applicable axis yields none.

   ⚑ flag

3. ai-native userSelf-host the core product

   weight 3 · round to YYLO
   YYLO✓fullprobed8/10

   YYLO is an open-source, MIT-licensed CLI orchestrator (installable via npm, source on GitHub) that runs locally against a user's own repo/agents, so self-hosting the core product is inherently satisfied — confirmed by the public npm registry listing and the MIT LICENSE in the public repo. missing for 10: no dedicated self-hosting/deployment guide (e.g. server install, Docker, or persistent-service setup instructions) and no independent third-party report of someone self-hosting it in production.

   • [probe] “PROBE runtime (recorded 2026-09-14): `npm view @yylo/cli name version bin` — the official CLI is live on the public npm registry (@yylo/cli …”
   • [probe] “PROBE runtime (recorded 2026-09-14): `curl -s https://raw.githubusercontent.com/yylo-dev/yylo/HEAD/LICENSE | head -3` returns 'MIT License /…”
   • [github] “YYLO is a command-line orchestrator for coding agents, repeatable workflows, and receipt-backed repository changes. It is for developers who…”
   • [claimed-docs] “npm install -g @yylo/cli@0.2.2 yylo init --task "Describe one verifiable outcome" --subagent claude”

   proof ↗⚑ flag

   HumanLayer—none0/10

   Evidence shows HumanLayer's daemon/agent execution can run on a user-controlled host (cloud VM, workstation, private network), but the core control plane is explicitly tied to the hosted app.humanlayer.com service — no docs describe self-hosting that core product. missing for 10: no self-hosted control-plane/server option, no on-prem deployment guide, no Docker/Helm chart or license for running the full stack independently.

   • [claimed-docs] “This tutorial teaches you how to run one task on a remote machine. You will control the task from app.humanlayer.com on a machine or phone.”
   • [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
   • [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”

   proof ↗⚑ flag

Enterprise licensing

1. engineering-leadLicense an enterprise deployment with SSO and commercial support for organization-wide rollout

   weight 2 · round drawn
   YYLO—none0/10

   YYLO's evidence pack shows only an open-source CLI orchestrator (MIT-licensed, npm package) with no mention of enterprise licensing tiers, SSO integration, or commercial support offerings; there is no pricing/plans page or enterprise sales material in evidence. missing for 10: enterprise/SSO licensing tier, commercial support plans, organization-wide deployment documentation.

   • [github] “YYLO is a command-line orchestrator for coding agents, repeatable workflows, and receipt-backed repository changes. It is for developers who…”
   • [probe] “PROBE runtime (recorded 2026-09-14): `npm view @yylo/cli name version bin` — the official CLI is live on the public npm registry (@yylo/cli …”
   • [probe] “PROBE runtime (recorded 2026-09-14): `curl -s https://raw.githubusercontent.com/yylo-dev/yylo/HEAD/LICENSE | head -3` returns 'MIT License /…”

   proof ↗⚑ flag

   HumanLayer—none0/10

   No evidence pack item mentions SSO, enterprise licensing, org-wide rollout, or commercial support tiers; docs cover tutorials, integrations, and workflow features only, and community discussion focuses on pricing/skepticism unrelated to enterprise licensing terms.

   ⚑ flag

Model flexibility

1. engineering-leadBring my own LLM or API key so agents run on the model of my choice

   weight 2 · round to HumanLayer
   YYLO—none0/10

   Evidence shows a `--subagent claude` CLI flag implying some agent selection, but there is no documentation of configuring API keys, choosing alternate LLM providers, or any pricing/billing control for engineering leads. Missing for 10: explicit BYO-API-key setup, multi-provider/model configuration docs, and any pricing-limits guidance.

   • [claimed-docs] “npm install -g @yylo/cli@0.2.2 yylo init --task "Describe one verifiable outcome" --subagent claude”

   proof ↗⚑ flag

   HumanLayer~partialclaimed5/10

   Docs show HumanLayer sessions can use different backends/models — Claude Code, OpenAI Codex, and Claude via Amazon Bedrock instead of the Anthropic API — indicating some flexibility in model/provider choice, which implies bringing your own credentials for these paths. However, there is no explicit doc describing a generic 'bring your own API key' mechanism for arbitrary LLMs or a pricing-tier note tying this to cost savings for engineering leads. Missing for 10: explicit BYO-API-key configuration docs, support for arbitrary/third-party model providers beyond Claude/Codex/Bedrock, and any pricing-related messaging about cost control via own keys.

   • [claimed-docs] “This guide shows you how to install, authenticate, select, and check Codex for HumanLayer sessions.”
   • [claimed-docs] “HumanLayer sessions can run Claude through Amazon Bedrock instead of the Anthropic API.”
   • [claimed-docs] “HumanLayer registers these RPI sub-agents for Claude Code sessions”

   proof ↗⚑ flag

1. ai-native userControl data retention and deletion

   weight 2 · round drawn
   YYLO—none0/10

   YYLO's evidence covers task orchestration, worktrees, ledgers, and merges, but nothing addresses data retention policies, storage duration, or user-controlled deletion of tasks/sessions/records. No docs mention exporting, purging, or deleting stored data or session logs.

   ⚑ flag

   HumanLayer—none0/10

   No evidence pack items mention data retention policies, deletion controls, or privacy settings for HumanLayer; the docs focus on task/session workflows, integrations, and remote daemons rather than data governance.

   ⚑ flag

2. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   YYLO—none0/10

   No evidence pack item mentions telemetry, usage tracking, analytics collection, or an opt-out mechanism for YYLO; the docs and GitHub description focus entirely on task orchestration features.

   ⚑ flag

   HumanLayer—none0/10

   No evidence in the pack addresses telemetry, usage tracking, or opt-out controls for HumanLayer; docs cover integrations, workflows, and CLI usage but nothing about privacy/telemetry settings. Missing for 10: any mention of telemetry collection, opt-out mechanism, privacy policy, or data-handling documentation.

   ⚑ flag

Knowledge context

1. developerAdd a context file describing my codebase conventions so agents generate more relevant plans and code

   weight 3 · round drawn
   YYLO—none0/10

   The evidence pack covers task orchestration, kanban ledgers, merge protections, and workflow runners, but nowhere describes a context/conventions file that agents read to generate more relevant plans or code. No mention of AGENTS.md, project instructions, or codebase-convention configuration exists in the docs or GitHub items provided.

   ⚑ flag

   HumanLayer—none0/10

   The docs describe workspace-level config files (workspace.json/workspace.local.json) for team/machine settings and multi-repo setup, but there is no evidence of a dedicated context file for describing codebase conventions to improve agent-generated plans/code.

   • [claimed-docs] “Use `.humanlayer/workspace.json` for shared repository or team configuration. Use `.humanlayer/workspace.local.json` for optional user or ma…”
   • [claimed-docs] “Configure a multi-repository workspace with this repository, ../api, and ../web. Make ../web the primary repository. Ask me before you choos…”

   proof ↗⚑ flag

Project management integration

1. product-managerConnect issue trackers like Jira, Linear, ClickUp, or Monday.com so agents can manage tickets directly

   weight 3 · round to HumanLayer
   YYLO—none0/10

   The integrations doc only mentions bringing Slack messages or GitHub issues into kanban (yylo-docs-10); there is no mention of Jira, Linear, ClickUp, or Monday.com anywhere in the evidence pack. Missing for 10: any documented connector or API integration for Jira, Linear, ClickUp, or Monday.com.

   • [claimed-docs] “Bring Slack messages or GitHub issues into kanban and return completed responses to their source thread.”

   proof ↗⚑ flag

   HumanLayer~partialclaimed6/10

   HumanLayer documents native Jira Cloud and Linear integrations that create tasks from tickets and sync/link status back to source issues, directly matching the story for those trackers. However, there is no evidence of ClickUp or Monday.com integrations, so the story is only partially delivered. Missing for 10: ClickUp integration docs, Monday.com integration docs, independent/hands-on verification of ticket sync working in practice.

   • [claimed-docs] “Connect Jira Cloud so HumanLayer can create tasks from Jira tickets and keep ticket-driven agent work connected to its source.”
   • [claimed-docs] “Connect Linear so HumanLayer can create tasks from Linear issues, sync issue status, and link HumanLayer artifacts back to the source issue.”

   proof ↗⚑ flag

Version control integration

1. developerConnect a GitHub repository so an agent can access the code and open pull requests against it

   weight 3 · round to HumanLayer
   YYLO—none0/10

   Evidence shows YYLO operates on local git worktrees/branches and has an internal 'merge land' step, and can pull GitHub issues into its kanban, but there is no evidence of connecting a GitHub repository as a remote and having the agent open pull requests against it — the merge feature explicitly stays local/internal with no GitHub PR API integration mentioned.

   • [github] “`task start` freezes the protected target SHA, creates a dedicated branch/worktree, and completes configured dependency hydration before rep…”
   • [github] “`merge land` selects one immutable task source, composes in a private detached candidate, and uses Git expected-old ref protection. A moved …”
   • [github] “Tests and semantic reviews are explicit project checks outside merge. Merge launches no models, chooses no reviewers, schedules no suites, a…”
   • [claimed-docs] “Bring Slack messages or GitHub issues into kanban and return completed responses to their source thread.”

   proof ↗⚑ flag

   HumanLayer✓fullclaimed8/10

   Docs describe connecting GitHub to create tasks from issues and link artifacts back (humanlayer-docs-16), plus agent sessions can access code via configured workspaces/repos (humanlayer-docs-8, humanlayer-docs-11) and open draft PRs directly from the session UI (humanlayer-docs-19). missing for 10: independent/hands-on corroboration of the GitHub connection flow and PR-opening working end-to-end beyond first-party docs.

   • [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
   • [claimed-docs] “Draft PR creation — Ask the session agent to open a draft pull request from the GitHub tab or diff view.”
   • [claimed-docs] “Use `.humanlayer/workspace.json` for shared repository or team configuration. Use `.humanlayer/workspace.local.json` for optional user or ma…”
   • [claimed-docs] “Configure a multi-repository workspace with this repository, ../api, and ../web. Make ../web the primary repository. Ask me before you choos…”

   proof ↗⚑ flag

2. developerGrant an agent access to my repositories with a one-click install, without complex setup

   weight 2 · round drawn
   YYLO—none0/10

   YYLO is documented as a CLI orchestrator requiring `npm install -g @yylo/cli` plus explicit `yylo init`/`task start` workflows that freeze SHAs, create worktrees, and hydrate dependencies — this is CLI-based setup, not a one-click repo-access grant. No evidence describes a GitHub App-style one-click install or OAuth flow for repo access. Missing for 10: any one-click install/authorization mechanism, evidence of simplified non-CLI onboarding, or a hosted install button.

   • [claimed-docs] “npm install -g @yylo/cli@0.2.2 yylo init --task "Describe one verifiable outcome" --subagent claude”
   • [github] “`task start` freezes the protected target SHA, creates a dedicated branch/worktree, and completes configured dependency hydration before rep…”
   • [probe] “PROBE runtime (recorded 2026-09-14): `npm view @yylo/cli name version bin` — the official CLI is live on the public npm registry (@yylo/cli …”

   proof ↗⚑ flag

   HumanLayer—none0/10

   Docs describe GitHub/Jira/Linear integrations for creating tasks from issues, but connecting a repo requires selecting a host, configuring workspace.json/workspace.local.json, and setting up remote daemons or multi-repo workspaces — none of this is framed as a one-click, no-setup install. No evidence pack item claims or demonstrates a one-click repo access flow.

   • [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
   • [claimed-docs] “Use `.humanlayer/workspace.json` for shared repository or team configuration. Use `.humanlayer/workspace.local.json` for optional user or ma…”
   • [claimed-docs] “Configure a multi-repository workspace with this repository, ../api, and ../web. Make ../web the primary repository. Ask me before you choos…”
   • [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”

   proof ↗⚑ flag

Ci remediation

1. engineering-leadHave failed CI workflows automatically diagnosed and fixed with a proposed pull request

   weight 3 · round to HumanLayer
   YYLO—none0/10

   YYLO is a CLI orchestrator for coding agents/workflows with kanban, ledger, and merge tooling, but nothing in the evidence pack mentions CI workflow failure detection, diagnosis, or auto-generating a fix PR from a failing CI run. Merge/land explicitly excludes running tests or validation ('Tests and semantic reviews are explicit project checks outside merge'), which is the opposite of an integrated CI-fix loop.

   • [github] “Tests and semantic reviews are explicit project checks outside merge. Merge launches no models, chooses no reviewers, schedules no suites, a…”
   • [github] “`merge land` selects one immutable task source, composes in a private detached candidate, and uses Git expected-old ref protection. A moved …”
   • [github] “YYLO is a command-line orchestrator for coding agents, repeatable workflows, and receipt-backed repository changes. It is for developers who…”

   proof ↗⚑ flag

   HumanLayer~partialclaimed5/10

   HumanLayer supports running automation sessions from CI (`humanlayer automation run`) and can open draft PRs from a session's diff/GitHub tab, and GitHub integration links tasks to issues—so the building blocks for a CI-triggered fix-and-PR flow exist. However there is no direct evidence of automatic diagnosis of failed CI logs/errors or a documented end-to-end 'CI failure → agent diagnosis → PR' pipeline. Missing for 10: explicit CI-failure-detection/diagnosis workflow docs, example of a failing pipeline auto-triggering a session, and confirmation the resulting PR addresses the CI failure specifically.

   • [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
   • [claimed-docs] “Draft PR creation — Ask the session agent to open a draft pull request from the GitHub tab or diff view.”
   • [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”

   proof ↗⚑ flag

2. developerTrigger an agent from CI/CD pipelines to fix a broken build or failing test

   weight 2 · round to HumanLayer
   YYLO—none0/10

   YYLO is documented as a CLI orchestrator with kanban tasks, workflow/parallel runners, and Slack/GitHub-issue integrations, but nothing in the evidence describes triggering it from a CI/CD pipeline or having it react to a failing build/test. Since it's a scriptable CLI, this axis plausibly applies, but there is no documented CI hook, GitHub Actions example, or build-failure-triggered workflow.

   • [github] “YYLO is a command-line orchestrator for coding agents, repeatable workflows, and receipt-backed repository changes. It is for developers who…”
   • [claimed-docs] “Repeat bounded YYLO iterations until no open kanban work remains.”
   • [claimed-docs] “Bring Slack messages or GitHub issues into kanban and return completed responses to their source thread.”

   proof ↗⚑ flag

   HumanLayer~partialclaimed6/10

   HumanLayer docs explicitly describe `humanlayer automation run` for running a Cloud-visible coding session 'from any automation environment — a CI job, a cron machine, or a script on a server,' plus launch tokens for non-interactive command execution, directly supporting CI/CD-triggered agent runs. However, there is no explicit example or integration guide showing a CI failure (broken build/failing test) triggering the agent to diagnose and fix it, nor independent/hands-on confirmation of this workflow. missing for 10: a concrete CI/CD pipeline example tied to build/test failures, evidence of automatic failure detection triggering the agent, and independent verification of this automation flow working in practice.

   • [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
   • [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
   • [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”

   proof ↗⚑ flag

Diff review

1. developerConfigure an agent to automatically open a pull request when its task completes

   weight 2 · round to HumanLayer
   YYLO—none0/10

   YYLO's docs describe task branches, worktrees, and an internal 'merge land' step that composes a candidate directly, explicitly stating merge 'launches no models, chooses no reviewers'—there is no mention anywhere of opening a GitHub pull request on task completion, only importing issues into kanban and returning responses to source threads. No CLI flag, workflow step, or integration doc references PR creation.

   • [github] “`merge land` selects one immutable task source, composes in a private detached candidate, and uses Git expected-old ref protection. A moved …”
   • [github] “Tests and semantic reviews are explicit project checks outside merge. Merge launches no models, chooses no reviewers, schedules no suites, a…”
   • [claimed-docs] “Bring Slack messages or GitHub issues into kanban and return completed responses to their source thread.”
   • [claimed-docs] “Start hydrates an exact-base task worktree. Preflight is read-only, finish queues the committed candidate”

   proof ↗⚑ flag

   HumanLayer~partialclaimed4/10

   Docs confirm HumanLayer can open a draft pull request from a session (humanlayer-docs-19) and integrates with GitHub for issue-linked tasks (humanlayer-docs-16), but the evidence describes PR creation as a manual 'ask the session agent' action from the UI, not an automatic trigger fired upon task completion. Missing for 10: explicit configuration option/workflow setting for automatic PR creation on task completion, and any evidence of it happening without a manual prompt.

   • [claimed-docs] “Draft PR creation — Ask the session agent to open a draft pull request from the GitHub tab or diff view.”
   • [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”

   proof ↗⚑ flag

2. developerReview a diff of an agent's changes and approve it before it becomes a pull request

   weight 3 · round to HumanLayer
   YYLO—none0/10

   Docs describe worktrees, candidate branches, and a 'merge land' step, but nowhere is there evidence of a diff-review UI or an explicit developer approval gate before a pull request is opened; in fact merge is described as launching 'no models, no reviewers' and reviews are called 'explicit project checks outside merge', with no PR-creation flow documented at all.

   • [github] “`task start` freezes the protected target SHA, creates a dedicated branch/worktree, and completes configured dependency hydration before rep…”
   • [github] “`merge land` selects one immutable task source, composes in a private detached candidate, and uses Git expected-old ref protection. A moved …”
   • [github] “Tests and semantic reviews are explicit project checks outside merge. Merge launches no models, chooses no reviewers, schedules no suites, a…”
   • [claimed-docs] “Start hydrates an exact-base task worktree. Preflight is read-only, finish queues the committed candidate”

   proof ↗⚑ flag

   HumanLayer~partialcommunity7/10

   Release notes explicitly reference a diff view and 'draft PR creation' workflow (docs-19, docs-21), and tasks include 'One place for comments and review' (docs-5), indicating a review-before-PR mechanism. However, there's no detailed documentation of an explicit approve/reject gate tied specifically to diff review prior to PR creation, and community discussion raises concerns about automation bias in approvals (comm-3) without disputing the core capability. Missing for 10: explicit step-by-step approval workflow docs, independent hands-on verification of the diff-review-then-approve flow, and clarity on how rejection/edits are handled before PR creation.

   • [claimed-docs] “Draft PR creation — Ask the session agent to open a draft pull request from the GitHub tab or diff view.”
   • [claimed-docs] “Keyboard navigation for changed files — Move through the PR changes tree with J/K, N/P, G shortcuts, and Enter.”
   • [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
   • [community] “"I feel much more comfortable with senior developers approving AI changes to our codebase, then letting loose an autonomous agent with no hu…”
   • [community] “User raised concern about automation bias: once an agent proves reliable, humans may rubber-stamp approvals without careful review, letting …”

   proof ↗⚑ flag

Pr review automation

1. ai-native userHave incoming issues automatically triaged with severity suggested and routed to the right owner

   weight 2 · round drawn
   YYLO—none0/10

   YYLO's integrations feature only pulls GitHub issues/Slack messages into a kanban board and returns responses to the source thread (yylo-docs-10); there is no evidence of automated severity classification or owner-based routing logic anywhere in the pack.

   • [claimed-docs] “Bring Slack messages or GitHub issues into kanban and return completed responses to their source thread.”

   proof ↗⚑ flag

   HumanLayer—none0/10

   HumanLayer connects issue trackers (Jira, GitHub, Linear) to create tasks from tickets, but there is no evidence of automatic triage, severity classification, or routing to an owner — integrations only create/link tasks, not assess or assign severity/ownership.

   • [claimed-docs] “Connect Jira Cloud so HumanLayer can create tasks from Jira tickets and keep ticket-driven agent work connected to its source.”
   • [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
   • [claimed-docs] “Connect Linear so HumanLayer can create tasks from Linear issues, sync issue status, and link HumanLayer artifacts back to the source issue.”

   proof ↗⚑ flag

2. engineering-leadHave every pull request automatically reviewed with AI-generated inline comments

   weight 3 · round drawn
   YYLO—none0/10

   YYLO's own docs describe it as a CLI orchestrator for coding-agent tasks, workflows, and receipt-backed merges — not a PR-review tool. Evidence explicitly states 'Tests and semantic reviews are explicit project checks outside merge. Merge launches no models, chooses no reviewers' (yylo-gh-6), meaning there is no automated AI-generated inline PR review capability in the product.

   • [github] “Tests and semantic reviews are explicit project checks outside merge. Merge launches no models, chooses no reviewers, schedules no suites, a…”
   • [github] “`merge land` selects one immutable task source, composes in a private detached candidate, and uses Git expected-old ref protection. A moved …”
   • [github] “YYLO is a command-line orchestrator for coding agents, repeatable workflows, and receipt-backed repository changes. It is for developers who…”

   proof ↗⚑ flag

   HumanLayer—none0/10

   HumanLayer's evidence covers task/session management, human-in-the-loop approvals, draft PR creation, and a PR diff-viewing UI, but there is no mention of an automated review process that generates inline review comments on every pull request. Missing for 10: no evidence of automatic PR review triggers, no mention of AI-generated inline comments, no review-quality-gate CI integration for PRs.

   • [claimed-docs] “Draft PR creation — Ask the session agent to open a draft pull request from the GitHub tab or diff view.”
   • [claimed-docs] “Keyboard navigation for changed files — Move through the PR changes tree with J/K, N/P, G shortcuts, and Enter.”
   • [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”

   proof ↗⚑ flag

Readiness checks

1. engineering-leadRun a readiness report that evaluates how ready my repository is for autonomous agents

   weight 2 · round to YYLO
   YYLO~partialclaimed4/10

   YYLO ships a `doctor workspace` check that flags actionable topology problems without mutating the repo, and its GitHub description references 'release-readiness boundaries,' which gesture at repo-readiness diagnostics, but there is no documented dedicated report scoring or evaluating overall repository readiness for autonomous agents. missing for 10: a named readiness-report command/output, criteria for 'agent readiness' beyond topology checks, and any sample report artifact.

   • [github] “`doctor workspace` is intentionally nonzero when it finds an actionable topology problem; it never fetches or changes the workspace.”
   • [github] “YYLO is a command-line orchestrator for coding agents, repeatable workflows, and receipt-backed repository changes. It is for developers who…”

   proof ↗⚑ flag

   HumanLayer—none0/10

   No evidence of any readiness-report or repository-readiness-assessment feature; HumanLayer's docs focus on task/session management, integrations, and human-in-the-loop workflows, not a diagnostic scan of a repo's agent-readiness.

   ⚑ flag

Concurrent execution

1. engineering-leadRun many agent tasks concurrently to scale delivery throughput

   weight 3 · round to YYLO
   YYLO~partialclaimed6/10

   YYLO documents a 'bounded concurrent fan-out' parallel-runner for independent kanban tasks with structured evidence per item, plus run-until-completion looping, which supports running multiple agent tasks concurrently. However, evidence lacks details on concurrency limits, scaling numbers, resource isolation at scale, or independent/hands-on validation of throughput gains. missing for 10: concrete concurrency limits/benchmarks, independent third-party validation of parallel scaling, evidence of large-scale (10s-100s of tasks) usage in production.

   • [claimed-docs] “Bounded concurrent fan-out for independent kanban tasks, data items, or complete commands—with structured evidence for every item.”
   • [claimed-docs] “Repeat bounded YYLO iterations until no open kanban work remains.”
   • [claimed-docs] “Continue a captured session with `yy continue SESSION_ID`; do not reconstruct work from tmux scrollback.”
   • [github] “YYLO is a command-line orchestrator for coding agents, repeatable workflows, and receipt-backed repository changes. It is for developers who…”

   proof ↗⚑ flag

   HumanLayer~partialclaimed5/10

   Docs describe an architecture (tasks/sessions, multi-repository workspaces, remote daemons on cloud VMs, and a CLI 'automation run' for CI/cron/scripts) that could support running many agent tasks in parallel, and 'Advanced Metrics' track usage/productivity across an org. However, tutorials and guides are framed around running one task/session at a time, and there is no explicit documentation or evidence of concurrent multi-task orchestration, throughput dashboards, or scaling guarantees for many simultaneous agents. Missing for 10: explicit multi-session concurrency docs, evidence of parallel task orchestration at scale, and independent confirmation of throughput gains.

   • [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
   • [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
   • [claimed-docs] “Configure a multi-repository workspace with this repository, ../api, and ../web. Make ../web the primary repository. Ask me before you choos…”
   • [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
   • [claimed-docs] “Advanced Metrics for all paid plans — View usage, cost, and productivity metrics, with optional access for every organization member.”

   proof ↗⚑ flag

Deployment flexibility

1. engineering-leadSelf-host agent infrastructure locally, in containers, or on my own VMs

   weight 2 · round to HumanLayer
   YYLO~partialprobed5/10

   YYLO is an open-source, MIT-licensed CLI (npm package + public GitHub repo) that runs locally as an orchestrator, implying it can be run on a developer's own machine, in containers, or VMs since it's just a Node CLI operating on a local git worktree. However, there is no explicit documentation of container/VM deployment, Docker images, self-hosting guides, or infrastructure requirements for running at scale. missing for 10: explicit self-hosting/deployment docs (Docker/container images, VM setup guides), infrastructure/scaling guidance, and confirmation of statelessness or multi-instance operation for parallel agent infrastructure.

   • [github] “YYLO is a command-line orchestrator for coding agents, repeatable workflows, and receipt-backed repository changes. It is for developers who…”
   • [probe] “PROBE runtime (recorded 2026-09-14): `npm view @yylo/cli name version bin` — the official CLI is live on the public npm registry (@yylo/cli …”
   • [probe] “PROBE runtime (recorded 2026-09-14): `curl -s https://raw.githubusercontent.com/yylo-dev/yylo/HEAD/LICENSE | head -3` returns 'MIT License /…”
   • [github] “A successful run prints the installed YYLO version, initializes `.juno_task/`, then emits a watch receipt with `"state":"COMPLETED"`, `"exit…”

   proof ↗⚑ flag

   HumanLayer~partialclaimed7/10

   Docs describe running the remote daemon on a cloud VM, workstation, or private-network machine that you control (humanlayer-docs-7), plus automation sessions from CI/cron/server environments (humanlayer-docs-12) and launch tokens for bootstrap scripts (humanlayer-docs-13), showing self-hosted deployment flexibility. However there is no explicit mention of container/Docker deployment or an official container image, and no independent verification of self-hosted setups at scale. Missing for 10: explicit container/Docker packaging docs, independent hands-on confirmation of self-hosted deployments.

   • [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
   • [claimed-docs] “This tutorial teaches you how to run one task on a remote machine. You will control the task from app.humanlayer.com on a machine or phone.”
   • [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
   • [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”

   proof ↗⚑ flag

Headless automation

1. developerRun an agent headlessly inside CI/CD pipelines and shell scripts

   weight 2 · round to HumanLayer
   YYLO~partialprobed6/10

   YYLO is a CLI-first orchestrator with commands (init, task start, ledger, loop, workflow-runner, parallel-runner) that are inherently scriptable and non-interactive, and its runtime bins (yylo, yy, ypl) are published on npm confirming CLI availability. However, there's no explicit CI/CD documentation, no exit-code/error-handling guarantance for pipeline use beyond doctor workspace, and no CI examples (GitHub Actions, GitLab CI, etc.) or headless/no-TTY confirmation. missing for 10: explicit CI/CD pipeline examples or docs, confirmed non-interactive/headless mode guarantees, exit-code contract documentation for scripting, independent hands-on CI usage reports.

   • [claimed-docs] “npm install -g @yylo/cli@0.2.2 yylo init --task "Describe one verifiable outcome" --subagent claude”
   • [claimed-docs] “`yy loop` repeats arbitrary shell commands sequentially.”
   • [claimed-docs] “Bounded concurrent fan-out for independent kanban tasks, data items, or complete commands—with structured evidence for every item.”
   • [github] “`doctor workspace` is intentionally nonzero when it finds an actionable topology problem; it never fetches or changes the workspace.”
   • [github] “YYLO is a command-line orchestrator for coding agents, repeatable workflows, and receipt-backed repository changes. It is for developers who…”
   • [probe] “PROBE runtime (recorded 2026-09-14): `npm view @yylo/cli name version bin` — the official CLI is live on the public npm registry (@yylo/cli …”

   proof ↗⚑ flag

   HumanLayer✓fullclaimed8/10

   Docs explicitly describe `humanlayer automation run` for running Cloud-visible coding sessions from CI jobs, cron machines, or scripts, plus launch tokens for non-interactive/non-PTY execution suited to headless environments. This directly matches running an agent headlessly in CI/CD and shell scripts. Missing for 10: independent/hands-on verification of CI usage and concrete pipeline examples (e.g. GitHub Actions config).

   • [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
   • [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
   • [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”

   proof ↗⚑ flag