OpenHands vs HumanLayer
open-source · free-tier · usage-based
·free-tier · subscription-per-seat · enterprise-custom
OpenHands wins · 35–18 (17 drawn)
Agenticness — how well agents can access and operate the productAgenticness
How well agents can access and operate the product
Agent access
ai-native userPoint an agent at llms.txt or agent-oriented docs
weight 2 · round to OpenHandsA probe confirms OpenHands hosts a live llms.txt at docs.openhands.dev/llms.txt returning HTTP 200 with an explicit LLM-friendly documentation index, directly satisfying the story. Missing for 10: no independent third-party confirmation that agents actually consume/parse this file successfully in practice.
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.openhands.dev/llms.txt # OpenHands Docs > LLM-friendly index of OpenHands documentation (V1). Lega…”
ai-native userRun the product headlessly / in CI for automation
weight 2 · round drawnOpenHands supports CLI mode with auto-approve for non-interactive runs, API support for automation/scripting, webhook/schedule-triggered automations (Slack, GitHub, Linear), and can run headless in Docker/VMs/servers — all consistent with CI/automation usage. Missing for 10: an explicit first-party CI pipeline example (e.g., GitHub Actions config) or independent hands-on confirmation of headless CI runs.
- [claimed-docs] “Auto-approve all actions (use with caution)”
- [claimed-docs] “API support for automation and scripting”
- [github] “Create automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events”
- [github] “Run agents locally, in Docker, on VMs, or anywhere you can run an agent server backend”
- [probe] “official CLI documented at https://docs.openhands.dev/usage/how-to/cli-mode”
Docs explicitly describe running HumanLayer headlessly via `humanlayer automation run` from CI jobs, cron machines, or scripts, plus launch tokens for non-interactive/non-PTY environments, and remote daemon hosts on cloud VMs or servers, directly matching the CI/automation story. Missing for 10: independent/hands-on confirmation of CI usage and more detail on auth/config specifics for automated pipelines.
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [probe] “official CLI documented at https://docs.humanlayer.com/guide/remote-daemons”
ai-native userPlug MCP servers into this product so it can use their tools
weight 3 · round drawnOpenHandsnone0/10OpenHands is an AI coding agent, so the axis of consuming MCP servers as a client is applicable (unlike serving MCP, which would be na for an agent), but no evidence in the pack mentions MCP integration, configuration, or tool-plugging capability at all.
HumanLayernone0/10No evidence anywhere in the pack mentions MCP servers or the ability to plug external tool servers into HumanLayer; integrations mentioned (Jira, Slack, GitHub, Linear) are task-source connectors, not MCP tool servers. Missing for 10: any mention of MCP protocol support, MCP server configuration, or tool-plugin mechanism.
ai-native userUse an official CLI
weight 2 · round to OpenHandsOpenHands documents an official CLI mode with natural language task execution, conversation resumption, and auto-approve controls, confirmed by dedicated docs pages and a probe verifying the CLI documentation page exists. Missing for 10: independent/hands-on third-party corroboration beyond vendor docs, and more detail on CLI installation/distribution mechanics.
- [claimed-docs] “Type natural language tasks and receive instant feedback”
- [claimed-docs] “Resume previous conversations”
- [claimed-docs] “Auto-approve all actions (use with caution)”
- [probe] “official CLI documented at https://docs.openhands.dev/usage/how-to/cli-mode”
HumanLayer documents an official CLI (e.g. `humanlayer automation run`, launch tokens, remote-daemon control) used for CI, cron, and scripted agentic workflows, confirmed by a dedicated CLI doc page. missing for 10: no independent/hands-on verification of the CLI, no full command reference, and no evidence of broader CLI feature parity with the app.
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [probe] “official CLI documented at https://docs.humanlayer.com/guide/remote-daemons”
ai-native userDrive the product through a documented public API
weight 3 · round to OpenHandsOpenHands exposes a documented OpenAPI spec (openapi.json) plus an llms.txt LLM-friendly docs index, and pricing docs explicitly mention 'API support for automation and scripting,' confirming a public, documented API for programmatic/agentic control. missing for 10: independent third-party corroboration of API usage/reliability and more detailed API reference docs beyond the openapi.json probe.
- [probe] “PROBE openapi: HTTP 200 at https://docs.openhands.dev/openapi.json — contains "openapi" key”
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.openhands.dev/llms.txt # OpenHands Docs > LLM-friendly index of OpenHands documentation (V1). Lega…”
- [claimed-docs] “API support for automation and scripting”
HumanLayer documents a CLI (`humanlayer automation run`, launch tokens, remote daemons) that lets automation environments drive sessions programmatically, which is a form of documented programmatic control, but there is no evidence of a documented public REST/OpenAPI API — probes explicitly found openapi.json/swagger.json/llms.txt all 404. missing for 10: a documented HTTP/OpenAPI public API spec, SDK/client library docs, and independent confirmation of API usage.
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.humanlayer.com/openapi.json, https://docs.humanlayer.com/swagger.json, https://docs.hum…”
- [probe] “PROBE llms.txt: HTTP 404 at https://docs.humanlayer.com/llms.txt”
- [probe] “official CLI documented at https://docs.humanlayer.com/guide/remote-daemons”
ai-native userIssue scoped/least-privilege API credentials for an agent
weight 2 · round to HumanLayerOpenHandsnone0/10Evidence shows OpenHands supports bringing your own LLM keys, SAML/SSO, and general API access, but nothing describes issuing scoped or least-privilege API credentials specifically for an agent's actions/tool access.
The only relevant evidence is a mention of a 'launch token' scoped to a single non-interactive command, which implies some least-privilege token issuance, but there is no documentation of a broader credential/permission model, scopes, or API key management for agents. missing for 10: explicit least-privilege credential scoping model, permission granularity, revocation/rotation mechanisms, and any independent corroboration.
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
ai-native userBuild against official SDKs
weight 2 · round to OpenHandsOpenHands documents an official Software Agent SDK ('a composable Python library for building agents that work with code') plus a public OpenAPI spec and llms.txt index, giving AI-native users concrete official interfaces to build against. Missing for 10: independent/third-party corroboration of SDK usage, and richer SDK-specific docs (examples, API reference depth) beyond the single description.
- [claimed-docs] “The Software Agent SDK is a composable Python library for building agents that work with code.”
- [probe] “PROBE openapi: HTTP 200 at https://docs.openhands.dev/openapi.json — contains "openapi" key”
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.openhands.dev/llms.txt # OpenHands Docs > LLM-friendly index of OpenHands documentation (V1). Lega…”
HumanLayernone0/10The evidence pack covers HumanLayer's CLI, workspace config, and third-party integrations (Jira, Slack, GitHub, Linear) but contains no mention of an official SDK (Python, TypeScript, etc.) for building against HumanLayer programmatically. Probes for openapi.json and llms.txt both returned 404, further suggesting no discoverable API/SDK surface.
- [probe] “PROBE llms.txt: HTTP 404 at https://docs.humanlayer.com/llms.txt”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.humanlayer.com/openapi.json, https://docs.humanlayer.com/swagger.json, https://docs.hum…”
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
ai-native userSubscribe to events via webhooks
weight 2 · round to OpenHandsGitHub README explicitly states OpenHands can 'run on a schedule or in response to webhook events' for automations, indicating webhook-triggered event subscription, but there is no dedicated documentation page detailing webhook setup, payload schema, or event types. missing for 10: dedicated webhook docs/config guide, independent/hands-on confirmation, and detail on which events can be subscribed to.
- [github] “Create automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events”
HumanLayernone0/10No evidence pack item mentions webhooks or event subscription mechanisms; integrations described (Slack, Jira, Linear, GitHub) are inbound task-creation connectors, not outbound webhook events, and API/OpenAPI probes returned 404s.
Agentic features
ai-native userGet AI-generated insights and suggestions from my data inside the product
weight 2 · round to OpenHandsOpenHands ships automation agents that inspect real data (incident logs, PR diffs, workflow failures, security alerts) and generate AI insights/suggestions such as severity assessment, review comments, root-cause analysis, and remediation PRs, directly matching the story's intent for a coding-agent product. Evidence is vendor-documented only, with no independent/hands-on corroboration of the quality of these insights. Missing for 10: independent validation of suggestion quality, a unified 'insights' UI/dashboard, and evidence of insights beyond code/security/ops contexts.
- [claimed-docs] “Watch for new bugs or incidents, gather logs and recent changes, suggest severity, and route the issue to the right owner.”
- [claimed-docs] “Watch for a configurable label on pull requests, inspect full PR context, and post an AI review comment once per label event.”
- [claimed-docs] “Detect failed workflows, inspect logs, identify the likely cause, and open a pull request with a proposed fix.”
- [claimed-docs] “Review security alerts, validate the finding, update affected code, and open a pull request with the remediation.”
- [claimed-docs] “Watch Slack channels for @openhands mentions, open a conversation with the message context, and reply when the agent finishes.”
HumanLayer's agents do generate task artifacts, draft PRs, and comments derived from a user's codebase/tickets, and 'Advanced Metrics' surfaces usage/cost/productivity data, which loosely resembles data-derived insight. However there is no evidence of dedicated AI-generated analytical insights or proactive suggestions distinct from executing assigned coding tasks. Missing for 10: explicit insight/analytics generation from user data, proactive suggestion features beyond task execution, and any first-party or community evidence of an 'insights' capability.
- [claimed-docs] “Advanced Metrics for all paid plans — View usage, cost, and productivity metrics, with optional access for every organization member.”
- [claimed-docs] “Draft PR creation — Ask the session agent to open a draft pull request from the GitHub tab or diff view.”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
ai-native userSet up automations that run autonomously in the background
weight 2 · round to OpenHandsOpenHands supports scheduled/webhook-triggered automations integrating with Slack, GitHub, Linear, etc., and ships prebuilt autonomous workflows (bug triage, PR review, CI failure fixing, security remediation, Slack mention handling) that run without human intervention, plus API support for scripting automations. Missing for 10: independent/hands-on verification of the scheduling/webhook trigger reliability and no detailed docs excerpt on configuring schedules beyond marketing copy.
- [github] “Create automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events”
- [claimed-docs] “Watch for new bugs or incidents, gather logs and recent changes, suggest severity, and route the issue to the right owner.”
- [claimed-docs] “Watch for a configurable label on pull requests, inspect full PR context, and post an AI review comment once per label event.”
- [claimed-docs] “Detect failed workflows, inspect logs, identify the likely cause, and open a pull request with a proposed fix.”
- [claimed-docs] “Review security alerts, validate the finding, update affected code, and open a pull request with the remediation.”
- [claimed-docs] “Watch Slack channels for @openhands mentions, open a conversation with the message context, and reply when the agent finishes.”
- [claimed-docs] “API support for automation and scripting”
Docs explicitly describe `humanlayer automation run` for running Cloud-visible coding sessions from CI jobs, cron machines, or scripts, plus remote daemons and launch tokens for non-interactive/background execution contexts, directly supporting autonomous background automations. Missing for 10: independent/hands-on verification of long-running background automations, native scheduling UI, and clarity on how human-approval gates interact with continuous autonomous runs.
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [claimed-docs] “Use `.humanlayer/workspace.json` for shared repository or team configuration. Use `.humanlayer/workspace.local.json` for optional user or ma…”
ai-native userDelegate tasks to a built-in AI assistant inside the product
weight 3 · round to OpenHandsOpenHands is itself the built-in AI assistant/agent: CLI mode lets users type natural language tasks and get instant feedback, resume conversations, and auto-approve actions, and it can run locally, remote, or in the cloud with any LLM. This directly matches delegating tasks to a built-in assistant within the product. Missing for 10: independent/hands-on user reports validating the delegation experience beyond vendor docs, and more detail on task delegation depth (e.g., multi-step autonomy limits).
- [claimed-docs] “Type natural language tasks and receive instant feedback”
- [claimed-docs] “Resume previous conversations”
- [claimed-docs] “Auto-approve all actions (use with caution)”
- [github] “Switch between local, remote, and cloud agents without losing focus”
- [github] “Use with any LLM”
- [probe] “official CLI documented at https://docs.openhands.dev/usage/how-to/cli-mode”
HumanLayer's core product model is task delegation to AI coding agents (Claude Code, Codex, Bedrock-backed Claude) via 'sessions', with workflow modes (Oneshot, RPI, PRD-Oriented, Freeform) and automation runs from CI/cron, directly matching 'delegate tasks to a built-in AI assistant'. missing for 10: independent/hands-on verification of the delegation experience beyond vendor docs, and clarity on how autonomous vs supervised the assistant is in practice.
- [claimed-docs] “This tutorial teaches you how to create and run one small task in HumanLayer. You will use the macOS app and Claude Code.”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
- [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
- [claimed-docs] “HumanLayer registers these RPI sub-agents for Claude Code sessions”
- [claimed-docs] “Select **Oneshot**, **RPI**, **PRD-Oriented**, or **Freeform**.”
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “This guide shows you how to install, authenticate, select, and check Codex for HumanLayer sessions.”
ai-native userOperate the product with natural-language commands
weight 2 · round to OpenHandsOpenHands' CLI mode explicitly lets users type natural language tasks and get instant feedback, resume conversations, and control approvals, directly matching the story, and this is corroborated by an official documented CLI probe. Missing for 10: independent/hands-on user reports validating the natural-language interaction quality beyond vendor docs.
- [claimed-docs] “Type natural language tasks and receive instant feedback”
- [claimed-docs] “Resume previous conversations”
- [claimed-docs] “Auto-approve all actions (use with caution)”
- [probe] “official CLI documented at https://docs.openhands.dev/usage/how-to/cli-mode”
HumanLayer's core interaction model is giving natural-language instructions to agent sessions (Claude Code, Codex) to create tasks, configure workspaces, and choose workflow phases, as shown by the example NL workspace-config prompt and workflow-selection docs. Missing for 10: independent/hands-on corroboration of NL command robustness and no evidence of a broader NL command surface beyond task/workflow setup.
- [claimed-docs] “Configure a multi-repository workspace with this repository, ../api, and ../web. Make ../web the primary repository. Ask me before you choos…”
- [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
- [claimed-docs] “Select **Oneshot**, **RPI**, **PRD-Oriented**, or **Freeform**.”
- [claimed-docs] “This tutorial teaches you how to create and run one small task in HumanLayer. You will use the macOS app and Claude Code.”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
Api quality
ai-native userExplore an interactive API reference with runnable examples
weight 2 · round to OpenHandsEvidence shows an OpenAPI spec is served (openhands-probe-2) and API support is advertised for automation/scripting (openhands-docs-13), implying some API reference exists, but there is no evidence of an interactive documentation UI (e.g., Swagger/Redoc 'try it' console) or runnable code examples tied to that spec. missing for 10: interactive API explorer UI, runnable/try-it code examples, confirmation the openapi.json is rendered as browsable docs.
- [probe] “PROBE openapi: HTTP 200 at https://docs.openhands.dev/openapi.json — contains "openapi" key”
- [claimed-docs] “API support for automation and scripting”
HumanLayernone0/10No evidence of an interactive API reference or runnable examples; probes explicitly show no OpenAPI/swagger spec and no llms.txt found, and docs are guide/tutorial style rather than an API reference sandbox.
ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)
weight 2 · round to OpenHandsA probe confirms an OpenAPI spec is served at docs.openhands.dev/openapi.json with a valid 'openapi' key, directly satisfying the machine-readable API spec requirement, and API support for automation/scripting is also documented in pricing. Missing for 10: independent third-party corroboration or detailed docs describing spec coverage/versioning.
- [probe] “PROBE openapi: HTTP 200 at https://docs.openhands.dev/openapi.json — contains "openapi" key”
- [claimed-docs] “API support for automation and scripting”
HumanLayernone0/10A direct probe for OpenAPI/swagger specs at all standard locations returned 404s, and no evidence pack item shows a downloadable machine-readable API spec being offered.
ai-native userTest against a sandbox environment without touching production data
weight 1 · round to OpenHandsOpenHands can run agents in Docker/VMs (openhands-gh-5), which implies isolated execution rather than direct production access, but the evidence pack never explicitly describes a sandbox environment for testing safely against non-production data. Missing for 10: explicit documentation of sandbox/isolation guarantees, workspace-vs-production data separation, and any hands-on confirmation that production systems are protected.
- [github] “Run agents locally, in Docker, on VMs, or anywhere you can run an agent server backend”
HumanLayernone0/10HumanLayer's docs describe remote daemons, workspaces, and automation sessions, but there is no mention of a sandbox environment, staging/test data isolation, or any mechanism to separate test runs from production data. missing for 10: explicit sandbox/staging environment documentation, data isolation guarantees, evidence of test-vs-production separation.
ai-native userRely on versioned APIs with a documented deprecation policy
weight 2 · round drawnOpenHandsnone0/10Evidence shows OpenHands exposes an OpenAPI spec and general 'API support for automation and scripting,' and docs reference a 'V1' with legacy V0 pages excluded, but there is no documented API versioning scheme or deprecation policy anywhere in the pack.
- [probe] “PROBE openapi: HTTP 200 at https://docs.openhands.dev/openapi.json — contains "openapi" key”
- [claimed-docs] “API support for automation and scripting”
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.openhands.dev/llms.txt # OpenHands Docs > LLM-friendly index of OpenHands documentation (V1). Lega…”
HumanLayernone0/10No evidence of API versioning scheme or a documented deprecation policy; probes for openapi.json/llms.txt returned 404s and no API reference or changelog covering versioning/deprecation is present.
Automation depth — how much of the product can run unattendedAutomation depth
How much of the product can run unattended
ai-native userPerform bulk operations across many items at once
weight 2 · round to OpenHandsOpenHands supports automations/workflows triggered by events (Slack, GitHub, Linear, webhooks) and API access for scripting, which enables some multi-item automation, but there's no explicit evidence of a bulk-operation feature (e.g., processing a batch list of items/tasks in one command or UI action). missing for 10: explicit bulk/batch operation feature, evidence of processing multiple items in a single invocation, UI/CLI support for batch task lists, independent confirmation of scale.
- [github] “Create automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events”
- [claimed-docs] “API support for automation and scripting”
- [claimed-docs] “Watch for new bugs or incidents, gather logs and recent changes, suggest severity, and route the issue to the right owner.”
- [claimed-docs] “Detect failed workflows, inspect logs, identify the likely cause, and open a pull request with a proposed fix.”
HumanLayernone0/10HumanLayer's documentation consistently frames work as single tasks/sessions ('one small task', 'one task on a remote machine', 'one Cloud-visible coding session') with per-task review and approval workflows; there is no mention of batch/bulk operations spanning many items at once. missing for 10: any documented bulk-action API/CLI flag, batch approval mechanism, or multi-item automation workflow.
- [claimed-docs] “This tutorial teaches you how to create and run one small task in HumanLayer. You will use the macOS app and Claude Code.”
- [claimed-docs] “This tutorial teaches you how to run one task on a remote machine. You will control the task from app.humanlayer.com on a machine or phone.”
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
ai-native userDefine rules that trigger actions automatically on events
weight 3 · round to OpenHandsOpenHands ships documented automation triggers—scheduled runs, webhook events, PR label events, Slack mentions, failed-workflow detection, and security-alert triage—each automatically invoking an agent action, matching the 'rules trigger actions on events' story (openhands-gh-4, openhands-docs-7 to openhands-docs-11). missing for 10: independent/hands-on verification that users can define fully custom rule logic (vs. fixed preset automations), and no evidence of a general-purpose rule-authoring UI or DSL for arbitrary event/action pairing
- [github] “Create automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events”
- [claimed-docs] “Watch for new bugs or incidents, gather logs and recent changes, suggest severity, and route the issue to the right owner.”
- [claimed-docs] “Watch for a configurable label on pull requests, inspect full PR context, and post an AI review comment once per label event.”
- [claimed-docs] “Detect failed workflows, inspect logs, identify the likely cause, and open a pull request with a proposed fix.”
- [claimed-docs] “Review security alerts, validate the finding, update affected code, and open a pull request with the remediation.”
- [claimed-docs] “Watch Slack channels for @openhands mentions, open a conversation with the message context, and reply when the agent finishes.”
HumanLayer integrations (GitHub, Jira, Linear) create tasks automatically from external events like new issues/tickets, and `humanlayer automation run` lets sessions be triggered from CI jobs, cron, or scripts — both are forms of event-driven automation. However, there's no evidence of a general-purpose rules/conditions engine letting users define arbitrary trigger-condition-action logic; the automation is limited to fixed integration hooks and script-based invocation. Missing for 10: a documented rule-definition interface (conditions, filters, custom triggers) beyond fixed platform integrations, and independent confirmation these event-triggers work reliably in practice.
- [claimed-docs] “Connect Jira Cloud so HumanLayer can create tasks from Jira tickets and keep ticket-driven agent work connected to its source.”
- [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
- [claimed-docs] “Connect Linear so HumanLayer can create tasks from Linear issues, sync issue status, and link HumanLayer artifacts back to the source issue.”
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
ai-native userSchedule recurring jobs or workflows
weight 2 · round to OpenHandsOpenHands explicitly supports creating automations/workflows that run on a schedule or in response to webhook events, integrating with Slack, GitHub, Linear, etc., which directly matches recurring job scheduling. Missing for 10: independent/hands-on corroboration of the scheduling UI/config and details on job management (pause/edit/monitor recurring jobs).
- [github] “Create automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events”
- [claimed-docs] “Watch for new bugs or incidents, gather logs and recent changes, suggest severity, and route the issue to the right owner.”
- [claimed-docs] “Watch for a configurable label on pull requests, inspect full PR context, and post an AI review comment once per label event.”
- [claimed-docs] “Detect failed workflows, inspect logs, identify the likely cause, and open a pull request with a proposed fix.”
- [claimed-docs] “Review security alerts, validate the finding, update affected code, and open a pull request with the remediation.”
- [claimed-docs] “Watch Slack channels for @openhands mentions, open a conversation with the message context, and reply when the agent finishes.”
HumanLayer's `automation run` CLI lets you trigger a Cloud-visible coding session from a cron machine or CI job, implying recurring runs are possible via external schedulers, but there is no documented native scheduling/recurrence feature (no cron syntax, interval config, or job queue) inside HumanLayer itself. missing for 10: built-in recurring scheduler, interval/cron configuration options, evidence of persistent recurring workflow management.
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
ai-native userVersion, review, and roll back my automations
weight 1 · round to HumanLayerOpenHandsnone0/10Evidence shows OpenHands can create automations/workflows (Slack, GitHub, Linear integrations, scheduled/webhook triggers) but nothing in the pack describes version history, review, or rollback mechanisms specifically for these automations themselves — no changelog, diff view, or revert feature is documented.
- [github] “Create automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events”
- [claimed-docs] “Watch for new bugs or incidents, gather logs and recent changes, suggest severity, and route the issue to the right owner.”
- [claimed-docs] “Watch for a configurable label on pull requests, inspect full PR context, and post an AI review comment once per label event.”
- [claimed-docs] “Detect failed workflows, inspect logs, identify the likely cause, and open a pull request with a proposed fix.”
- [claimed-docs] “Review security alerts, validate the finding, update affected code, and open a pull request with the remediation.”
- [claimed-docs] “Watch Slack channels for @openhands mentions, open a conversation with the message context, and reply when the agent finishes.”
HumanLayer's task model provides review (comments, PR draft creation) and history that persists across sessions, giving some review/audit capability, but there is no explicit documentation of versioning workflow definitions or rolling back an automation to a prior version. missing for 10: explicit automation versioning/history diffing, a documented rollback mechanism, and independent confirmation these review features extend to full automation lifecycle management.
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
- [claimed-docs] “Draft PR creation — Ask the session agent to open a draft pull request from the GitHub tab or diff view.”
- [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
Autonomous implementation — end-to-end implementation by the agent — multi-file changes, task completionAutonomous implementation
End-to-end implementation by the agent — multi-file changes, task completion
End to end feature delivery
ai-native userHave an agent automatically generate and run tests to validate its own code changes before proposing them
weight 2 · round drawnOpenHandsnone0/10No evidence in the pack describes OpenHands agents autonomously generating or running tests to validate their own code changes before proposing them; the docs/GitHub excerpts cover CLI usage, multi-agent backends, automations, and pricing, but not test-generation/self-validation workflows.
developerHave an agent autonomously diagnose and fix a reported bug
weight 3 · round to OpenHandsOpenHands ships explicit automation for bug diagnosis and fixing: watching for new bugs/incidents and gathering logs (openhands-docs-7), detecting failed workflows/inspecting logs/identifying cause and opening a PR fix (openhands-docs-9), and general natural-language task execution via CLI (openhands-docs-1). This directly matches autonomous bug diagnosis-and-fix, though missing for 10: independent hands-on verification of fix success rate and end-to-end case studies beyond vendor docs.
- [claimed-docs] “Watch for new bugs or incidents, gather logs and recent changes, suggest severity, and route the issue to the right owner.”
- [claimed-docs] “Detect failed workflows, inspect logs, identify the likely cause, and open a pull request with a proposed fix.”
- [claimed-docs] “Type natural language tasks and receive instant feedback”
- [claimed-docs] “The Software Agent SDK is a composable Python library for building agents that work with code.”
HumanLayer supports creating tasks directly from GitHub/Jira/Linear issues and then running an agent session (Claude Code/Codex) against the linked repo, with an 'Oneshot' workflow phase designed for small, clear changes — a plausible bug-fix pipeline. However, the product's core design is human-in-the-loop with approval gates rather than fully autonomous action, and there's no end-to-end documented example of an agent independently diagnosing a bug from a ticket and shipping a fix without human review. Missing for 10: a concrete autonomous bug-diagnosis-and-fix walkthrough, and clarity on how much human approval is required mid-flow.
- [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
- [claimed-docs] “Connect Linear so HumanLayer can create tasks from Linear issues, sync issue status, and link HumanLayer artifacts back to the source issue.”
- [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [community] “"I feel much more comfortable with senior developers approving AI changes to our codebase, then letting loose an autonomous agent with no hu…”
product-managerGo from a mockup or design to a working implementation without an engineering handoff
weight 2 · round drawnOpenHandsnone0/10The evidence shows OpenHands executes natural-language coding tasks and automations, but nothing in the pack addresses ingesting mockups/design files or a PM-oriented, engineer-free workflow from design to implementation. missing for 10: mockup/design ingestion capability, no-code PM-facing workflow evidence, any example of design-to-code handoff elimination.
- [claimed-docs] “Type natural language tasks and receive instant feedback”
- [github] “Create automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events”
HumanLayernone0/10HumanLayer's evidence is entirely about developer-facing workflows: running coding sessions, connecting Jira/Linear/GitHub/Slack, managing remote daemons, and CLI automation for engineers overseeing coding agents. Nothing in the pack shows a mockup/design import capability, a no-code interface, or any path for a non-engineer product manager to turn a design into a working implementation without engineering involvement — in fact the workflow language (RPI, PRD-oriented, Oneshot) and workspace configs assume an engineering operator. Missing for evidence of delivering this story: mockup/design ingestion, PM-oriented no-code UI, and any case study of a non-engineer shipping code end-to-end.
- [claimed-docs] “This tutorial teaches you how to create and run one small task in HumanLayer. You will use the macOS app and Claude Code.”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
- [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
- [claimed-docs] “Select **Oneshot**, **RPI**, **PRD-Oriented**, or **Freeform**.”
developerHave an agent implement a requested feature end-to-end, including writing tests
weight 3 · round drawnOpenHands is positioned as an autonomous coding agent that takes natural language tasks and works across CLI, cloud, and automation workflows (e.g., PR review, bug-fixing, incident response), implying it can implement features end-to-end, but the evidence pack lacks a concrete first-party or hands-on example of the agent writing a feature plus tests. Missing for 10: explicit documentation/demo of full feature-implementation-with-tests workflow, independent benchmark or hands-on verification of test-writing capability, and confirmation of end-to-end PR creation including tests.
- [claimed-docs] “Type natural language tasks and receive instant feedback”
- [claimed-docs] “Auto-approve all actions (use with caution)”
- [github] “Run OpenHands, Claude Code, Codex, Gemini, or any ACP-compatible agent across local, remote, and cloud backends.”
- [claimed-docs] “Detect failed workflows, inspect logs, identify the likely cause, and open a pull request with a proposed fix.”
HumanLayer clearly supports end-to-end autonomous coding sessions (Oneshot, RPI, PRD-Oriented workflows) that implement tasks using agents like Claude Code and Codex, including structured phases and sub-agents for research/plan/implement, which implies substantial feature work can be delegated (humanlayer-docs-6, humanlayer-docs-9, humanlayer-docs-10, humanlayer-docs-14). However, no evidence explicitly confirms the agent writes or runs tests as part of the workflow, and no hands-on account demonstrates a full feature-plus-tests delivery. Missing for 10: explicit documentation or example showing test generation/execution as part of the implementation flow, and independent verification of end-to-end feature completion including tests.
- [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
- [claimed-docs] “HumanLayer registers these RPI sub-agents for Claude Code sessions”
- [claimed-docs] “Select **Oneshot**, **RPI**, **PRD-Oriented**, or **Freeform**.”
- [claimed-docs] “This guide shows you how to install, authenticate, select, and check Codex for HumanLayer sessions.”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
Environment setup
developerHave an agent automatically clone the repo, install dependencies, and configure its own working environment
weight 2 · round to HumanLayerEvidence shows OpenHands runs agents in configurable sandboxed backends (Docker, VMs, local/remote/cloud) and supports natural-language task execution with auto-approve, which implies some autonomous environment setup, but no citation explicitly confirms the agent auto-clones repos or installs dependencies on its own. Missing for 10: explicit documentation of repo cloning, dependency installation, and end-to-end environment bootstrap steps.
- [github] “Run agents locally, in Docker, on VMs, or anywhere you can run an agent server backend”
- [claimed-docs] “Type natural language tasks and receive instant feedback”
- [claimed-docs] “Auto-approve all actions (use with caution)”
Docs describe workspace configuration (workspace.json, multi-repo setups) and remote hosts that must have access to code/tools/credentials, and one example prompt asks the agent about 'setup commands or local files to copy,' implying some environment configuration ability. However, there is no explicit description of the agent autonomously cloning a repo or installing dependencies end-to-end without human setup of the host/workspace first. Missing for 10: explicit documentation of automatic repo cloning, dependency installation steps, and end-to-end environment bootstrap without prior manual host/workspace configuration.
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [claimed-docs] “Use `.humanlayer/workspace.json` for shared repository or team configuration. Use `.humanlayer/workspace.local.json` for optional user or ma…”
- [claimed-docs] “Configure a multi-repository workspace with this repository, ../api, and ../web. Make ../web the primary repository. Ask me before you choos…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
Interactive takeover
developerTake over an in-progress agent task in my editor, terminal, or browser to finish or redirect the work
weight 2 · round to HumanLayerOpenHands supports resuming conversations and switching between local/remote/cloud agents 'without losing focus,' implying some cross-surface continuity, but there's no explicit documentation of a developer taking over a live in-progress task from editor/terminal/browser mid-run with redirect capability. missing for 10: explicit IDE/editor integration for live takeover, concrete UI/UX description of mid-task redirect, and independent hands-on confirmation of seamless handoff across all three surfaces.
- [github] “Switch between local, remote, and cloud agents without losing focus”
- [github] “Run OpenHands, Claude Code, Codex, Gemini, or any ACP-compatible agent across local, remote, and cloud backends.”
- [claimed-docs] “Resume previous conversations”
Docs describe tasks with session/history continuity 'across agents and workstations', remote control from app.humanlayer.com on any machine or phone, and CLI-driven remote daemons for terminal/server contexts, all pointing to genuine hand-off of in-progress work between editor (Claude Code), terminal (remote daemon/CLI), and browser (web app). Live multiplayer draft recovery further shows shared/continuable session state. missing for 10: independent/hands-on confirmation of a live takeover mid-task, and explicit description of an in-editor (IDE plugin) takeover UI rather than just CLI/app/web.
- [claimed-docs] “This tutorial teaches you how to create and run one small task in HumanLayer. You will use the macOS app and Claude Code.”
- [claimed-docs] “This tutorial teaches you how to run one task on a remote machine. You will control the task from app.humanlayer.com on a machine or phone.”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
- [claimed-docs] “Live multiplayer response drafts — Write prompts together with shared text, named cursors, presence, read-only viewing, and draft recovery.”
developerSend follow-up instructions to an active agent session to steer its work without restarting
weight 2 · round to OpenHandsCLI mode docs mention typing natural language tasks with instant feedback and resuming previous conversations, which implies interactive follow-up steering, but there is no explicit evidence describing sending new instructions mid-task to an already-running/active agent session without restarting it. missing for 10: explicit documentation of mid-session steering/interrupt-and-redirect behavior while an agent is actively working, and independent/hands-on confirmation that follow-up messages are incorporated without restarting the session.
- [claimed-docs] “Type natural language tasks and receive instant feedback”
- [claimed-docs] “Resume previous conversations”
- [probe] “official CLI documented at https://docs.openhands.dev/usage/how-to/cli-mode”
HumanLayer's task/session model implies ongoing interaction with agents across a task's lifetime (docs-5) and release notes mention live prompt collaboration during sessions (docs-18), suggesting some capacity to interact with an active session, but there is no explicit documentation of sending mid-session follow-up instructions to steer a running agent without restarting it. missing for 10: explicit docs on injecting new instructions into a live/running session, confirmation the agent incorporates such input without restart, and independent/hands-on verification of this steering behavior.
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
- [claimed-docs] “Live multiplayer response drafts — Write prompts together with shared text, named cursors, presence, read-only viewing, and draft recovery.”
Sandbox execution
developerHave an agent safely execute code and install dependencies inside an isolated sandbox
weight 3 · round to OpenHandsEvidence confirms OpenHands supports running agents in isolated environments (Docker/VMs) and explicitly references 'sandbox infrastructure' as part of its backend, implying code execution is sandboxed. However, there is no detailed documentation on dependency installation within the sandbox, security guarantees, or isolation mechanics beyond these brief mentions. missing for 10: explicit sandbox architecture docs, dependency-installation workflow details, isolation/security guarantees, independent verification of sandbox safety.
- [github] “Run agents locally, in Docker, on VMs, or anywhere you can run an agent server backend”
- [claimed-docs] “OpenHands Cloud is the managed commercial service for running OpenHands without operating your own backend and sandbox infrastructure.”
HumanLayernone0/10HumanLayer docs describe running sessions on remote hosts (cloud VM, workstation, private-network machine) and automation environments, but there is no mention of an isolated/sandboxed execution environment for running code or installing dependencies safely — the host selection is about access/credentials, not isolation guarantees. Missing for full/partial: any explicit sandbox, container, or isolation mechanism; no evidence of dependency-install safety controls.
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
Human oversight — keeping a human in the loop — approvals, checkpoints, interruptsHuman oversight
Keeping a human in the loop — approvals, checkpoints, interrupts
Approval controls
developerConfigure an agent to auto-approve all its actions instead of confirming each one
weight 2 · round to OpenHandsDocs explicitly list an 'Auto-approve all actions (use with caution)' setting for CLI mode, directly matching the story. Missing for 10: independent/hands-on confirmation and details on scope (per-action vs global) or how to configure it beyond CLI mode.
- [claimed-docs] “Auto-approve all actions (use with caution)”
- [probe] “official CLI documented at https://docs.openhands.dev/usage/how-to/cli-mode”
HumanLayer's core premise is human-in-the-loop approval, and docs mention workflow phases like 'Oneshot' for low-risk changes and automation sessions (humanlayer automation run) that run non-interactively without confirmation, implying some auto-approve capability exists, but no explicit documentation of a configurable 'auto-approve all actions' toggle or setting is shown. missing for 10: explicit config/flag to disable per-action confirmation entirely, documentation confirming automation sessions skip all human review rather than just running unattended, and independent confirmation this works as intended.
- [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
product-managerApprove key agent decisions from my phone while agents continue working
weight 1 · round to HumanLayerOpenHands Cloud offers 'hosted cloud access from desktop and mobile' and Slack-based interaction (@mentions, replies) that could let a PM check in remotely, but there is no documented feature for approving specific in-progress agent actions/decisions via a mobile interface while the agent continues autonomously — the only approval-related control mentioned is a blanket 'auto-approve all actions' CLI flag, not selective human-in-the-loop approval. Missing for 10: explicit mobile approval/confirmation UI, human-in-the-loop decision gating documentation, evidence of push notifications or approval prompts reaching a phone.
- [claimed-docs] “Hosted cloud access from desktop and mobile”
- [claimed-docs] “Watch Slack channels for @openhands mentions, open a conversation with the message context, and reply when the agent finishes.”
- [claimed-docs] “Auto-approve all actions (use with caution)”
Docs explicitly describe controlling and continuing a running agent task from a phone via app.humanlayer.com, with tasks providing a shared review/comment history across devices, directly matching the phone-approval-while-agent-continues story; community sentiment corroborates the human-approval-of-agent-actions use case. Missing for 10: a hands-on/independent account specifically confirming the mobile approval UI in practice, and explicit documentation of an 'approve/deny decision' action (vs. general task control) on mobile.
- [claimed-docs] “This tutorial teaches you how to run one task on a remote machine. You will control the task from app.humanlayer.com on a machine or phone.”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [community] “"I feel much more comfortable with senior developers approving AI changes to our codebase, then letting loose an autonomous agent with no hu…”
engineering-leadSet tiered autonomy levels controlling what an agent can do without manual confirmation
weight 3 · round to HumanLayerEvidence shows only a binary confirmation toggle (auto-approve all actions vs manual confirmation) in CLI mode, not a tiered/granular autonomy system with multiple configurable levels for an engineering lead to set. Missing for 10: documented multi-tier permission/autonomy levels, admin controls to enforce team-wide policies, and per-action or per-risk-category confirmation settings.
- [claimed-docs] “Auto-approve all actions (use with caution)”
HumanLayer's workflow-phases doc shows tiered approaches (Oneshot for low-risk changes vs. more review for unclear/risky changes) and skills-workflows lets users select Oneshot/RPI/PRD-Oriented/Freeform modes, which map to different levels of autonomy vs. oversight. However, there's no explicit documentation of a formal 'autonomy level' setting per agent/task with configurable confirmation thresholds, and no independent evidence confirming this tiered control works as an oversight mechanism in practice. missing for 10: explicit named autonomy-tier configuration (e.g., low/medium/high) tied to confirmation gating, evidence of engineering-lead-level policy controls across a team, and independent/hands-on validation that these workflow phases actually reduce unnecessary confirmations without sacrificing safety.
- [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
- [claimed-docs] “Select **Oneshot**, **RPI**, **PRD-Oriented**, or **Freeform**.”
- [claimed-docs] “HumanLayer registers these RPI sub-agents for Claude Code sessions”
Model control
ai-native userHave each task prompt automatically routed to the most suitable underlying model
weight 2 · round drawnOpenHandsnone0/10Evidence shows OpenHands supports using any LLM and switching between agent backends manually, but there is no mention of automatic routing of prompts to the most suitable model based on task characteristics. Missing for 10: any model-routing/selection logic, per-task model suitability heuristics, or documentation of automatic model selection.
- [github] “Use with any LLM”
- [github] “Switch between local, remote, and cloud agents without losing focus”
- [claimed-docs] “OpenHands Open Source and OpenHands Cloud plans offer support to bring your own LLM keys.”
HumanLayernone0/10Evidence shows HumanLayer lets users manually select or configure which model/backend to use (Claude via Bedrock, Codex, RPI sub-agents) but there is no evidence of automatic routing of a task prompt to the 'most suitable' model based on task characteristics.
- [claimed-docs] “HumanLayer registers these RPI sub-agents for Claude Code sessions”
- [claimed-docs] “This guide shows you how to install, authenticate, select, and check Codex for HumanLayer sessions.”
- [claimed-docs] “HumanLayer sessions can run Claude through Amazon Bedrock instead of the Anthropic API.”
engineering-leadSwitch away from automatic model selection to a specific model of my choice
weight 1 · round to HumanLayerEvidence confirms OpenHands can be used with any LLM and lets users bring their own LLM keys, implying model choice is configurable, but there is no explicit documentation of an 'automatic model selection' default or a UI/CLI toggle to override it. Missing for 10: explicit docs on default/automatic model selection behavior, step-by-step instructions for switching models, and independent confirmation of the switch working in practice.
- [github] “Use with any LLM”
- [claimed-docs] “OpenHands Open Source and OpenHands Cloud plans offer support to bring your own LLM keys.”
Docs show explicit model/backend selection — choosing Codex ("install, authenticate, select, and check Codex") or running Claude via Amazon Bedrock instead of the Anthropic API, plus subagent model registration — indicating an engineering lead can pick a specific model rather than a default. However, there is no explicit documentation of an 'automatic' default-selection mode being toggled off, so the framing of 'switching away from automatic' isn't directly evidenced. Missing for 10: explicit mention of an automatic/default model-selection setting and a documented UI/CLI flag to override it, independent confirmation of model-switching behavior.
- [claimed-docs] “This guide shows you how to install, authenticate, select, and check Codex for HumanLayer sessions.”
- [claimed-docs] “HumanLayer sessions can run Claude through Amazon Bedrock instead of the Anthropic API.”
- [claimed-docs] “HumanLayer registers these RPI sub-agents for Claude Code sessions”
Visibility monitoring
developerWatch what a running agent is doing in real time, including its current status
weight 3 · round to HumanLayerOpenHands CLI mode offers instant feedback on tasks and resumable conversations, implying some real-time interaction, but there's no explicit evidence of a live status dashboard, streaming action log, or step-by-step progress view while an agent runs. missing for 10: explicit real-time status/progress UI documentation, evidence of live action streaming or step visibility, independent hands-on confirmation of watching an agent live.
- [claimed-docs] “Type natural language tasks and receive instant feedback”
- [claimed-docs] “Resume previous conversations”
- [probe] “official CLI documented at https://docs.openhands.dev/usage/how-to/cli-mode”
HumanLayer's docs describe remote-daemon control from app.humanlayer.com (including from a phone), live multiplayer session viewing with presence/read-only viewing, and task/session history — all suggesting real-time visibility into agent activity. However, there's no explicit documentation of a dedicated live status/log stream or dashboard showing granular agent state (e.g., current tool call, progress bar) beyond session/task views. missing for 10: explicit real-time status/log streaming documentation, independent hands-on confirmation of live monitoring UX.
- [claimed-docs] “This tutorial teaches you how to run one task on a remote machine. You will control the task from app.humanlayer.com on a machine or phone.”
- [claimed-docs] “Live multiplayer response drafts — Write prompts together with shared text, named cursors, presence, read-only viewing, and draft recovery.”
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
developerGet notified when an agent completes a task or needs my input
weight 2 · round drawnOpenHands supports Slack integration where it 'replies when the agent finishes' and workflow automations tied to webhook events (Slack, GitHub, Linear), which implies notification-like behavior on task completion; CLI mode also gives instant feedback in interactive sessions. However, there's no explicit evidence of a dedicated notification system for 'needs my input' events or push/desktop alerts outside Slack context. missing for 10: explicit documentation of notifications when agent needs human input/approval, native mobile/desktop push alerts, and independent/hands-on confirmation of notification reliability.
- [claimed-docs] “Watch Slack channels for @openhands mentions, open a conversation with the message context, and reply when the agent finishes.”
- [github] “Create automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events”
- [claimed-docs] “Type natural language tasks and receive instant feedback”
HumanLayer supports Slack/GitHub/Jira/Linear integrations that push task artifact updates and human-in-the-loop approvals, implying notification when tasks progress or need input, and its core design centers on human oversight of agent work. However, there is no explicit documentation of a dedicated 'task complete' or 'needs input' notification/alert mechanism (e.g., push notification, email, or webhook triggered specifically on completion/approval-required events) beyond generic artifact updates in Slack. missing for 10: explicit completion/needs-input notification docs, evidence of notification latency/reliability, independent/hands-on confirmation of notification delivery.
- [claimed-docs] “Connect Slack to send HumanLayer task artifact updates into the channels where your team already works.”
- [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
- [claimed-docs] “Connect Linear so HumanLayer can create tasks from Linear issues, sync issue status, and link HumanLayer artifacts back to the source issue.”
- [claimed-docs] “Connect Jira Cloud so HumanLayer can create tasks from Jira tickets and keep ticket-driven agent work connected to its source.”
- [community] “"I feel much more comfortable with senior developers approving AI changes to our codebase, then letting loose an autonomous agent with no hu…”
Intent to spec — stories about intent to spec in this arenaIntent to spec
Stories about intent to spec in this arena
Natural language task intake
developerDescribe a feature or bug in plain language and have it automatically turned into a scoped implementation task
weight 3 · round to HumanLayerOpenHands lets users type natural-language tasks directly in the CLI and receive feedback, which is the core mechanism for turning plain-language input into agent-executed work, and automation triggers (Slack mentions, PR labels, failed workflows) show it converting informal signals into concrete PRs/tasks. However, there is no evidence of an explicit 'spec' or scoped task artifact (e.g., a generated plan, ticket, or requirements doc) being produced before implementation — missing for 10: explicit task-scoping/spec generation step, evidence of a structured plan artifact, and independent confirmation that vague bug/feature descriptions reliably become well-scoped tasks rather than direct code edits.
- [claimed-docs] “Type natural language tasks and receive instant feedback”
- [claimed-docs] “Watch for new bugs or incidents, gather logs and recent changes, suggest severity, and route the issue to the right owner.”
- [claimed-docs] “Watch for a configurable label on pull requests, inspect full PR context, and post an AI review comment once per label event.”
- [claimed-docs] “Detect failed workflows, inspect logs, identify the likely cause, and open a pull request with a proposed fix.”
- [claimed-docs] “Watch Slack channels for @openhands mentions, open a conversation with the message context, and reply when the agent finishes.”
HumanLayer's task model (docs-5) and workflow-phase selection (docs-6, docs-10) show that a task is created and can be routed through 'Oneshot' for small clear changes or heavier RPI/PRD-oriented flows for ambiguous work, and RPI sub-agents (docs-9) imply a research→plan→implement pipeline that turns a description into a scoped plan. However, there is no explicit walkthrough showing a raw plain-language bug/feature description being automatically parsed into a scoped implementation task end-to-end, and integrations (Jira/Linear/GitHub) mostly create tasks from existing tickets rather than free-form language input. Missing for 10: a concrete example or tutorial of plain-language-to-scoped-task conversion, and independent/hands-on confirmation that this pipeline works as described.
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
- [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
- [claimed-docs] “HumanLayer registers these RPI sub-agents for Claude Code sessions”
- [claimed-docs] “Select **Oneshot**, **RPI**, **PRD-Oriented**, or **Freeform**.”
product-managerConvert user feedback submissions into structured tasks with proposed scope
weight 2 · round to HumanLayerOpenHandsnone0/10OpenHands integrates with Linear/GitHub/Slack for automations and can process natural-language tasks, so the general axis of turning input into work items is plausible, but there is no evidence of a feature that ingests user feedback and outputs a structured task with proposed scope for PM workflows.
HumanLayer supports creating tasks from external issue trackers (Jira, Linear, GitHub) which could serve as a proxy for user feedback submissions, and tasks include shared files/scope info, but there's no evidence of a dedicated feature for ingesting raw user feedback (e.g., support tickets, survey responses) and auto-structuring it into a task with a proposed scope specifically tailored for PM workflows. missing for 10: dedicated feedback-ingestion mechanism, explicit 'proposed scope' generation from unstructured feedback text, PM-specific workflow templates, and any hands-on/community validation of this specific use case.
- [claimed-docs] “Connect Jira Cloud so HumanLayer can create tasks from Jira tickets and keep ticket-driven agent work connected to its source.”
- [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
- [claimed-docs] “Connect Linear so HumanLayer can create tasks from Linear issues, sync issue status, and link HumanLayer artifacts back to the source issue.”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
developerAttach a marked-up screenshot or mockup to a task so the agent implements the correct visual change
weight 2 · round to HumanLayerOpenHandsnone0/10No evidence anywhere in the pack that OpenHands supports attaching images, screenshots, or mockups to a task, or that the agent can interpret visual markup to drive implementation; documentation focuses on text-based CLI tasks, integrations, and automations.
Docs confirm images can be pasted into the new task composer as attachments (humanlayer-docs-22), which supports attaching a screenshot to a task, but there is no evidence of markup/annotation tooling or of the agent parsing visual annotations to implement a corresponding UI change. Missing for 10: annotation/markup capability for screenshots, evidence the agent interprets visual markup into a specific implementation, and any hands-on example of this workflow succeeding.
- [claimed-docs] “Image paste in new tasks — Paste images straight into the new task composer as attachments.”
Plan approval
developerReview and approve an agent's implementation plan before any code changes are made
weight 3 · round to HumanLayerOpenHandsnone0/10The evidence pack mentions auto-approve controls and CLI usage but contains no mention of a plan-review/approval step before code changes are made; no planning-phase or approval-gate feature is documented. missing for 10: any documentation of a plan-generation step, an approval/confirmation gate prior to code edits, or user testimony confirming such a workflow exists.
HumanLayer's workflow-phases and RPI sub-agent docs describe planning phases (e.g., 'a change with unclear behavior or code shape needs more review before implementation') and PRD-Oriented/RPI workflows imply a plan stage before code changes, with tasks providing 'one place for comments and review.' However, no evidence explicitly shows a dedicated plan-approval gate/UI step where a developer reviews and approves a plan artifact before implementation begins. missing for 10: explicit documentation of a plan-approval step/UI, first-party example of blocking implementation until plan is approved, independent/hands-on confirmation of this specific gate.
- [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
- [claimed-docs] “HumanLayer registers these RPI sub-agents for Claude Code sessions”
- [claimed-docs] “Select **Oneshot**, **RPI**, **PRD-Oriented**, or **Freeform**.”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
engineering-leadApprove a task's scope and contract before an agent is allowed to modify the repository
weight 2 · round to HumanLayerOpenHandsnone0/10No evidence of a scope/contract approval workflow gating agent repository modifications; only auto-approve settings and general automation features are documented, not a pre-execution scope-approval gate for engineering leads.
- [claimed-docs] “Auto-approve all actions (use with caution)”
HumanLayer's workflow-phases doc explicitly supports scoping review before implementation (e.g., 'a change with unclear behavior or code shape needs more review before implementation'), and tasks/sessions provide a structured place for comments and review prior to agent execution, plus human-in-the-loop approval is core to the product's value prop per community discussion. However, there's no explicit documented feature for an engineering-lead specifically approving a 'scope and contract' artifact as a gating step before repo modification — it's inferred from general workflow-phase and review mechanics rather than a dedicated scope-approval gate. missing for 10: explicit documentation of a formal scope/contract approval step tied to lead sign-off, evidence of blocking repo writes until such approval, and independent/hands-on confirmation this gate works as intended.
- [claimed-docs] “Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape ne…”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
- [claimed-docs] “Select **Oneshot**, **RPI**, **PRD-Oriented**, or **Freeform**.”
- [community] “"I feel much more comfortable with senior developers approving AI changes to our codebase, then letting loose an autonomous agent with no hu…”
Ticket driven tasking
developerAssign a coding task to an agent directly from an existing issue or ticket
weight 3 · round to HumanLayerOpenHands supports automations that integrate with GitHub and Linear and can respond to webhook events, plus label-triggered PR review and bug-watching automations, implying issue/ticket-triggered agent runs are possible. However there's no explicit documentation of a workflow where a developer directly assigns a specific issue/ticket to an agent (e.g. via an 'assign to OpenHands' button or issue-comment trigger) as opposed to general automation setup. Missing for 10: explicit documentation of issue-to-agent assignment UX (e.g., GitHub issue comment/label triggering agent to pick up that specific ticket), independent/hands-on confirmation of this workflow, and ticketing system coverage beyond GitHub/Linear mentions.
- [github] “Create automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events”
- [claimed-docs] “Watch for a configurable label on pull requests, inspect full PR context, and post an AI review comment once per label event.”
- [claimed-docs] “Watch for new bugs or incidents, gather logs and recent changes, suggest severity, and route the issue to the right owner.”
Docs explicitly describe connecting GitHub, Jira, and Linear so HumanLayer creates tasks directly from issues/tickets and links artifacts back to the source, directly matching the story of assigning agent work from an existing ticket. Missing for 10: independent/hands-on confirmation that this ticket-to-task flow works reliably in practice, and more detail on the actual assignment UX.
- [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
- [claimed-docs] “Connect Jira Cloud so HumanLayer can create tasks from Jira tickets and keep ticket-driven agent work connected to its source.”
- [claimed-docs] “Connect Linear so HumanLayer can create tasks from Linear issues, sync issue status, and link HumanLayer artifacts back to the source issue.”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
Openness — open source, data portability, and self-hosting storiesOpenness
Open source, data portability, and self-hosting stories
ai-native userDo everything through the API that I can do in the UI
weight 2 · round to OpenHandsOpenHands exposes a documented OpenAPI spec and dedicated 'API support for automation and scripting' tier, plus CLI mode with resume/auto-approve that mirrors UI conversation flows, suggesting broad API parity. However, no evidence explicitly confirms that every UI feature (e.g., cloud dashboard views, integrations config, enterprise SSO management) is fully API-accessible. Missing for 10: explicit parity documentation enumerating UI vs API feature coverage, and independent/hands-on confirmation that API can replicate all UI workflows.
- [probe] “PROBE openapi: HTTP 200 at https://docs.openhands.dev/openapi.json — contains "openapi" key”
- [claimed-docs] “API support for automation and scripting”
- [claimed-docs] “Type natural language tasks and receive instant feedback”
- [claimed-docs] “Resume previous conversations”
- [claimed-docs] “Auto-approve all actions (use with caution)”
- [probe] “official CLI documented at https://docs.openhands.dev/usage/how-to/cli-mode”
HumanLayer offers a CLI (`humanlayer automation run`) and remote daemon controls that let you launch and manage sessions outside the UI, but there is no documented public API/OpenAPI spec (both openapi.json and llms.txt probes 404), and UI-only features like live multiplayer drafts, keyboard navigation, and image paste have no CLI/API equivalent documented. missing for 10: a documented REST/GraphQL API or OpenAPI spec, confirmation that all UI actions (draft PRs, multiplayer editing, metrics views) are exposed programmatically, and independent verification of API-UI parity.
- [probe] “official CLI documented at https://docs.humanlayer.com/guide/remote-daemons”
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
- [probe] “PROBE llms.txt: HTTP 404 at https://docs.humanlayer.com/llms.txt”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.humanlayer.com/openapi.json, https://docs.humanlayer.com/swagger.json, https://docs.hum…”
- [claimed-docs] “Live multiplayer response drafts — Write prompts together with shared text, named cursors, presence, read-only viewing, and draft recovery.”
- [claimed-docs] “Keyboard navigation for changed files — Move through the PR changes tree with J/K, N/P, G shortcuts, and Enter.”
ai-native userExport all of my data in open formats and leave
weight 3 · round drawnOpenHandsnone0/10No evidence in the pack describes an explicit data export feature or open-format data portability; only conversation resume and self-hosting options are mentioned, not a way to export/leave with all user data. missing for 10: explicit export/download feature, documented open data formats, statement on data portability upon leaving the platform.
ai-native userRead the product's source under an open license
weight 2 · round to OpenHandsThe GitHub repository (openhands-gh-1..5) confirms the source code is publicly hosted and readable, and openhands-docs-15 explicitly references an 'OpenHands Open Source' plan/tier, implying the core project is open-licensed. However, no evidence pack item names the specific license (e.g., MIT/Apache) or points to a LICENSE file, so full open-license confirmation is unverified. Missing for 10: explicit license name/file citation, independent confirmation of license terms.
- [github] “Switch between local, remote, and cloud agents without losing focus”
- [claimed-docs] “OpenHands Open Source and OpenHands Cloud plans offer support to bring your own LLM keys.”
ai-native userSelf-host the core product
weight 3 · round to OpenHandsEvidence shows OpenHands can be run locally/self-hosted (Docker, VMs, or any agent server backend) as opposed to only using the managed Cloud/Enterprise offerings, and it is open-source with an SDK for building on it. Missing for 10: independent hands-on confirmation of a full self-hosted setup (e.g., a third-party report of running the entire stack without cloud dependencies) and detailed self-hosting infra requirements/documentation.
- [github] “Run agents locally, in Docker, on VMs, or anywhere you can run an agent server backend”
- [claimed-docs] “OpenHands Cloud is the managed commercial service for running OpenHands without operating your own backend and sandbox infrastructure.”
- [claimed-docs] “The Software Agent SDK is a composable Python library for building agents that work with code.”
- [claimed-docs] “OpenHands Open Source and OpenHands Cloud plans offer support to bring your own LLM keys.”
HumanLayernone0/10Evidence shows HumanLayer's daemon/agent execution can run on a user-controlled host (cloud VM, workstation, private network), but the core control plane is explicitly tied to the hosted app.humanlayer.com service — no docs describe self-hosting that core product. missing for 10: no self-hosted control-plane/server option, no on-prem deployment guide, no Docker/Helm chart or license for running the full stack independently.
- [claimed-docs] “This tutorial teaches you how to run one task on a remote machine. You will control the task from app.humanlayer.com on a machine or phone.”
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
Pricing limits — free-tier ceilings, usage caps, and rate limits before you have to payPricing limits
Free-tier ceilings, usage caps, and rate limits before you have to pay
Enterprise licensing
engineering-leadLicense an enterprise deployment with SSO and commercial support for organization-wide rollout
weight 2 · round to OpenHandsOpenHands explicitly markets an Enterprise tier with licensed self-hosting/managed deployment and commercial support (openhands-docs-5), and the pricing page lists Enterprise SAML/SSO (openhands-docs-14). However, there is no detail on contract terms, SLA specifics, or independent confirmation of enterprise rollouts. Missing for 10: concrete SLA/support-tier documentation, case studies or third-party validation of enterprise SSO rollout, and clarity on org-wide admin/governance controls.
- [claimed-docs] “OpenHands Enterprise provides commercial capabilities and support for organizations that need licensed self-hosting or managed deployment op…”
- [claimed-docs] “Enterprise SAML / SSO”
- [claimed-docs] “OpenHands Cloud is the managed commercial service for running OpenHands without operating your own backend and sandbox infrastructure.”
Model flexibility
engineering-leadBring my own LLM or API key so agents run on the model of my choice
weight 2 · round to OpenHandsGitHub docs explicitly state OpenHands can be used with any LLM, and pricing docs confirm both Open Source and Cloud plans support bringing your own LLM keys, directly matching the story. Missing for 10: independent/hands-on verification of BYO-key setup and any model-specific limitations or edge cases.
- [github] “Use with any LLM”
- [claimed-docs] “OpenHands Open Source and OpenHands Cloud plans offer support to bring your own LLM keys.”
Docs show HumanLayer sessions can use different backends/models — Claude Code, OpenAI Codex, and Claude via Amazon Bedrock instead of the Anthropic API — indicating some flexibility in model/provider choice, which implies bringing your own credentials for these paths. However, there is no explicit doc describing a generic 'bring your own API key' mechanism for arbitrary LLMs or a pricing-tier note tying this to cost savings for engineering leads. Missing for 10: explicit BYO-API-key configuration docs, support for arbitrary/third-party model providers beyond Claude/Codex/Bedrock, and any pricing-related messaging about cost control via own keys.
- [claimed-docs] “This guide shows you how to install, authenticate, select, and check Codex for HumanLayer sessions.”
- [claimed-docs] “HumanLayer sessions can run Claude through Amazon Bedrock instead of the Anthropic API.”
- [claimed-docs] “HumanLayer registers these RPI sub-agents for Claude Code sessions”
Usage quotas
engineering-leadSee and manage plan-based daily task and concurrency limits for agent workflows
weight 2 · round drawnOpenHandsnone0/10Evidence shows pricing page mentions plan features like SSO, API access, and BYO LLM keys, but nothing about daily task limits, concurrency limits, or any management dashboard for such limits.
- [claimed-docs] “Hosted cloud access from desktop and mobile”
- [claimed-docs] “API support for automation and scripting”
- [claimed-docs] “Enterprise SAML / SSO”
- [claimed-docs] “OpenHands Open Source and OpenHands Cloud plans offer support to bring your own LLM keys.”
HumanLayernone0/10The evidence pack has no mention of plan-based daily task/concurrency limits or any admin controls for managing such limits; only a vague reference to 'Advanced Metrics for all paid plans' which covers usage/cost/productivity viewing, not concurrency or daily task limits management.
- [claimed-docs] “Advanced Metrics for all paid plans — View usage, cost, and productivity metrics, with optional access for every organization member.”
Privacy posture — data-handling and privacy storiesPrivacy posture
Data-handling and privacy stories
ai-native userChoose where my data is stored (region/residency)
weight 2 · round drawnOpenHandsnone0/10No evidence of data residency/region selection controls anywhere in the pack; only self-hosting options (local/Docker/VM) are mentioned, which is a workaround, not a documented region-choice feature for the managed/cloud offering.
ai-native userPrevent my data from being used to train AI models
weight 3 · round drawnOpenHandsnone0/10No evidence in the pack addresses data usage for AI model training, opt-out policies, or any privacy commitments regarding training data; the pack only covers CLI usage, deployment options, and integrations.
ai-native userControl data retention and deletion
weight 2 · round drawnOpenHandsnone0/10No evidence pack items describe data retention policies, deletion controls, or user-facing settings for managing stored conversation/data lifecycle; while OpenHands is open-source and self-hostable (implying some inherent control), no explicit retention/deletion feature or documentation is cited.
ai-native userOpt out of telemetry and usage tracking
weight 2 · round drawnOpenHandsnone0/10No evidence in the pack mentions telemetry, usage tracking, analytics, or opt-out settings; the pack covers CLI features, agent orchestration, and pricing tiers only.
HumanLayernone0/10No evidence in the pack addresses telemetry, usage tracking, or opt-out controls for HumanLayer; docs cover integrations, workflows, and CLI usage but nothing about privacy/telemetry settings. Missing for 10: any mention of telemetry collection, opt-out mechanism, privacy policy, or data-handling documentation.
Repo integration — stories about repo integration in this arenaRepo integration
Stories about repo integration in this arena
Chat integration
developerTag an agent in a chat thread to discuss and delegate a bug or task
weight 2 · round to OpenHandsOpenHands documents a Slack integration where the agent watches for @openhands mentions, opens a conversation with the message context, and replies when done (openhands-docs-11), which matches tagging an agent in a chat thread to delegate a task. However this is limited to Slack specifically (not other chat platforms), appears tied to the Cloud/automation feature set rather than the core product, and lacks independent/hands-on corroboration. Missing for 10: support for other chat platforms (e.g., Teams, Discord), independent verification of the Slack flow working in practice, and detail on how delegated context/threading is preserved during the exchange.
- [claimed-docs] “Watch Slack channels for @openhands mentions, open a conversation with the message context, and reply when the agent finishes.”
- [github] “Create automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events”
HumanLayernone0/10HumanLayer's Slack integration only pushes task-artifact updates into channels (docs-4) and other integrations (GitHub, Jira, Linear) create tasks from tickets/issues, not from tagging an agent inside a chat thread. There is no evidence of a chat-native @mention or in-thread delegation workflow for discussing/assigning tasks to an agent.
- [claimed-docs] “Connect Slack to send HumanLayer task artifact updates into the channels where your team already works.”
- [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
- [claimed-docs] “Connect Jira Cloud so HumanLayer can create tasks from Jira tickets and keep ticket-driven agent work connected to its source.”
- [claimed-docs] “Connect Linear so HumanLayer can create tasks from Linear issues, sync issue status, and link HumanLayer artifacts back to the source issue.”
Knowledge context
developerAdd a context file describing my codebase conventions so agents generate more relevant plans and code
weight 3 · round drawnOpenHandsnone0/10No evidence in this pack mentions a repo-level context/convention file (e.g., microagents, custom instructions, or similar) that developers can add to guide agent behavior; the pack only covers CLI usage, deployment, integrations, and pricing.
HumanLayernone0/10The docs describe workspace-level config files (workspace.json/workspace.local.json) for team/machine settings and multi-repo setup, but there is no evidence of a dedicated context file for describing codebase conventions to improve agent-generated plans/code.
- [claimed-docs] “Use `.humanlayer/workspace.json` for shared repository or team configuration. Use `.humanlayer/workspace.local.json` for optional user or ma…”
- [claimed-docs] “Configure a multi-repository workspace with this repository, ../api, and ../web. Make ../web the primary repository. Ask me before you choos…”
Project management integration
product-managerConnect issue trackers like Jira, Linear, ClickUp, or Monday.com so agents can manage tickets directly
weight 3 · round to HumanLayerOpenHands documents automation workflows that integrate with Linear (and Slack/GitHub) via webhooks/schedules, showing some issue-tracker connectivity, but Jira, ClickUp, and Monday.com are never mentioned anywhere in the evidence pack — only vague 'and more' language covers them. missing for 10: explicit Jira/ClickUp/Monday.com integrations, docs on ticket management workflows beyond Linear, evidence of two-way ticket manipulation (create/update/close) rather than just webhook triggers.
- [github] “Create automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events”
HumanLayer documents native Jira Cloud and Linear integrations that create tasks from tickets and sync/link status back to source issues, directly matching the story for those trackers. However, there is no evidence of ClickUp or Monday.com integrations, so the story is only partially delivered. Missing for 10: ClickUp integration docs, Monday.com integration docs, independent/hands-on verification of ticket sync working in practice.
- [claimed-docs] “Connect Jira Cloud so HumanLayer can create tasks from Jira tickets and keep ticket-driven agent work connected to its source.”
- [claimed-docs] “Connect Linear so HumanLayer can create tasks from Linear issues, sync issue status, and link HumanLayer artifacts back to the source issue.”
Version control integration
developerConnect a GitHub repository so an agent can access the code and open pull requests against it
weight 3 · round to HumanLayerGitHub is explicitly listed as an integration target for OpenHands automations, and multiple documented workflows show the agent opening pull requests (fixing failed CI, remediating security alerts, responding to PR-review label events), which implies repo access and PR creation. However, there is no first-party documentation of the actual repo-connection/auth flow (e.g., installing a GitHub App, granting repo scopes) or hands-on confirmation that this works end-to-end. Missing for 10: explicit repo-connection setup docs, evidence of PR creation permissions/scopes, and independent verification of successful PRs opened.
- [github] “Create automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events”
- [claimed-docs] “Detect failed workflows, inspect logs, identify the likely cause, and open a pull request with a proposed fix.”
- [claimed-docs] “Review security alerts, validate the finding, update affected code, and open a pull request with the remediation.”
- [claimed-docs] “Watch for a configurable label on pull requests, inspect full PR context, and post an AI review comment once per label event.”
Docs describe connecting GitHub to create tasks from issues and link artifacts back (humanlayer-docs-16), plus agent sessions can access code via configured workspaces/repos (humanlayer-docs-8, humanlayer-docs-11) and open draft PRs directly from the session UI (humanlayer-docs-19). missing for 10: independent/hands-on corroboration of the GitHub connection flow and PR-opening working end-to-end beyond first-party docs.
- [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
- [claimed-docs] “Draft PR creation — Ask the session agent to open a draft pull request from the GitHub tab or diff view.”
- [claimed-docs] “Use `.humanlayer/workspace.json` for shared repository or team configuration. Use `.humanlayer/workspace.local.json` for optional user or ma…”
- [claimed-docs] “Configure a multi-repository workspace with this repository, ../api, and ../web. Make ../web the primary repository. Ask me before you choos…”
developerGrant an agent access to my repositories with a one-click install, without complex setup
weight 2 · round to OpenHandsOpenHands Cloud offers hosted access and integrations (Slack, GitHub, Linear, webhooks) suggesting some repo connection flow, but there is no concrete evidence of a one-click repo install/auth flow — no screenshots, GitHub App install steps, or onboarding walkthrough. missing for 10: documented one-click GitHub/GitLab App install flow, evidence of minimal setup steps, independent confirmation of ease of onboarding.
- [github] “Create automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events”
- [claimed-docs] “OpenHands Cloud is the managed commercial service for running OpenHands without operating your own backend and sandbox infrastructure.”
- [claimed-docs] “Hosted cloud access from desktop and mobile”
HumanLayernone0/10Docs describe GitHub/Jira/Linear integrations for creating tasks from issues, but connecting a repo requires selecting a host, configuring workspace.json/workspace.local.json, and setting up remote daemons or multi-repo workspaces — none of this is framed as a one-click, no-setup install. No evidence pack item claims or demonstrates a one-click repo access flow.
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [claimed-docs] “Use `.humanlayer/workspace.json` for shared repository or team configuration. Use `.humanlayer/workspace.local.json` for optional user or ma…”
- [claimed-docs] “Configure a multi-repository workspace with this repository, ../api, and ../web. Make ../web the primary repository. Ask me before you choos…”
- [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
Review quality gates — quality gates on changes — review flow, required checks, merge protectionReview quality gates
Quality gates on changes — review flow, required checks, merge protection
Ci remediation
engineering-leadHave failed CI workflows automatically diagnosed and fixed with a proposed pull request
weight 3 · round to OpenHandsOpenHands documents a dedicated automation that detects failed CI workflows, inspects logs, identifies the likely cause, and opens a pull request with a proposed fix — matching the story directly. Missing for 10: independent/hands-on verification of this automation working in practice, and detail on configuration/setup beyond the marketing description.
- [claimed-docs] “Detect failed workflows, inspect logs, identify the likely cause, and open a pull request with a proposed fix.”
- [github] “Create automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events”
HumanLayer supports running automation sessions from CI (`humanlayer automation run`) and can open draft PRs from a session's diff/GitHub tab, and GitHub integration links tasks to issues—so the building blocks for a CI-triggered fix-and-PR flow exist. However there is no direct evidence of automatic diagnosis of failed CI logs/errors or a documented end-to-end 'CI failure → agent diagnosis → PR' pipeline. Missing for 10: explicit CI-failure-detection/diagnosis workflow docs, example of a failing pipeline auto-triggering a session, and confirmation the resulting PR addresses the CI failure specifically.
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Draft PR creation — Ask the session agent to open a draft pull request from the GitHub tab or diff view.”
- [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
developerTrigger an agent from CI/CD pipelines to fix a broken build or failing test
weight 2 · round to OpenHandsOpenHands advertises a workflow that watches for failed CI/CD workflows, inspects logs, identifies causes, and opens a PR with a fix (openhands-docs-9), plus webhook/schedule-triggered automations (openhands-gh-4) and API support for scripting (openhands-docs-13), which together support triggering an agent from CI/CD to fix broken builds. However, there is no concrete example of GitHub Actions/CI pipeline configuration, no evidence of test-failure-specific triggers, and no independent/hands-on confirmation that this works as described. Missing for 10: explicit CI pipeline integration docs/examples, test-failure-specific triggers, third-party verification.
- [claimed-docs] “Detect failed workflows, inspect logs, identify the likely cause, and open a pull request with a proposed fix.”
- [github] “Create automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events”
- [claimed-docs] “API support for automation and scripting”
- [claimed-docs] “The Software Agent SDK is a composable Python library for building agents that work with code.”
HumanLayer docs explicitly describe `humanlayer automation run` for running a Cloud-visible coding session 'from any automation environment — a CI job, a cron machine, or a script on a server,' plus launch tokens for non-interactive command execution, directly supporting CI/CD-triggered agent runs. However, there is no explicit example or integration guide showing a CI failure (broken build/failing test) triggering the agent to diagnose and fix it, nor independent/hands-on confirmation of this workflow. missing for 10: a concrete CI/CD pipeline example tied to build/test failures, evidence of automatic failure detection triggering the agent, and independent verification of this automation flow working in practice.
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
Diff review
developerConfigure an agent to automatically open a pull request when its task completes
weight 2 · round to OpenHandsOpenHands documents workflow automations that open pull requests automatically for specific triggers (failed CI, security alerts) via [openhands-docs-9] and [openhands-docs-10], showing the underlying capability exists. However, there's no direct documentation of configuring a general coding-task agent to auto-open a PR upon arbitrary task completion — the evidence only covers specific automation templates (bug-fix, security remediation) rather than a generic 'open PR on task completion' setting. Missing for 10: explicit config option/flag for auto-PR-on-completion in standard task workflows, independent/hands-on confirmation of this behavior.
- [claimed-docs] “Detect failed workflows, inspect logs, identify the likely cause, and open a pull request with a proposed fix.”
- [claimed-docs] “Review security alerts, validate the finding, update affected code, and open a pull request with the remediation.”
- [github] “Create automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events”
Docs confirm HumanLayer can open a draft pull request from a session (humanlayer-docs-19) and integrates with GitHub for issue-linked tasks (humanlayer-docs-16), but the evidence describes PR creation as a manual 'ask the session agent' action from the UI, not an automatic trigger fired upon task completion. Missing for 10: explicit configuration option/workflow setting for automatic PR creation on task completion, and any evidence of it happening without a manual prompt.
- [claimed-docs] “Draft PR creation — Ask the session agent to open a draft pull request from the GitHub tab or diff view.”
- [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
developerReview a diff of an agent's changes and approve it before it becomes a pull request
weight 3 · round to HumanLayerOpenHands has an auto-approve/manual-approve action mode (openhands-docs-3 implies a default confirmation step exists before auto-approve is enabled) and can open PRs after agent work, suggesting some human-in-the-loop gating exists, but there's no explicit documentation of a diff review UI or an approval gate specifically before PR creation. missing for 10: explicit diff-review interface, documented approve/reject step tied to PR creation, evidence of a review-before-merge workflow.
- [claimed-docs] “Auto-approve all actions (use with caution)”
- [claimed-docs] “Watch for a configurable label on pull requests, inspect full PR context, and post an AI review comment once per label event.”
Release notes explicitly reference a diff view and 'draft PR creation' workflow (docs-19, docs-21), and tasks include 'One place for comments and review' (docs-5), indicating a review-before-PR mechanism. However, there's no detailed documentation of an explicit approve/reject gate tied specifically to diff review prior to PR creation, and community discussion raises concerns about automation bias in approvals (comm-3) without disputing the core capability. Missing for 10: explicit step-by-step approval workflow docs, independent hands-on verification of the diff-review-then-approve flow, and clarity on how rejection/edits are handled before PR creation.
- [claimed-docs] “Draft PR creation — Ask the session agent to open a draft pull request from the GitHub tab or diff view.”
- [claimed-docs] “Keyboard navigation for changed files — Move through the PR changes tree with J/K, N/P, G shortcuts, and Enter.”
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
- [community] “"I feel much more comfortable with senior developers approving AI changes to our codebase, then letting loose an autonomous agent with no hu…”
- [community] “User raised concern about automation bias: once an agent proves reliable, humans may rubber-stamp approvals without careful review, letting …”
Pr review automation
ai-native userHave incoming issues automatically triaged with severity suggested and routed to the right owner
weight 2 · round to OpenHandsopenhands-docs-7 directly describes an automation that watches for new bugs/incidents, gathers logs and recent changes, suggests severity, and routes the issue to the right owner — matching the story closely, backed by GitHub-documented webhook/scheduled automation infrastructure (openhands-gh-4). Missing for 10: independent/hands-on verification of triage accuracy and routing correctness beyond vendor's own site copy.
- [claimed-docs] “Watch for new bugs or incidents, gather logs and recent changes, suggest severity, and route the issue to the right owner.”
- [github] “Create automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events”
HumanLayernone0/10HumanLayer connects issue trackers (Jira, GitHub, Linear) to create tasks from tickets, but there is no evidence of automatic triage, severity classification, or routing to an owner — integrations only create/link tasks, not assess or assign severity/ownership.
- [claimed-docs] “Connect Jira Cloud so HumanLayer can create tasks from Jira tickets and keep ticket-driven agent work connected to its source.”
- [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
- [claimed-docs] “Connect Linear so HumanLayer can create tasks from Linear issues, sync issue status, and link HumanLayer artifacts back to the source issue.”
engineering-leadHave every pull request automatically reviewed with AI-generated inline comments
weight 3 · round to OpenHandsOpenHands documents a specific automation that watches for a configurable PR label, inspects full PR context, and posts an AI review comment, which directly matches automated PR review with AI-generated comments. However, it is label-triggered rather than automatic on every PR, and posts once per label event rather than full inline (line-by-line) comments, so it's a partial match to 'every pull request' with 'inline comments'. missing for 10: evidence of automatic triggering on all PRs without manual labeling, confirmation of true inline (line-level) code comments vs a single summary comment, and independent/hands-on verification of this workflow in practice.
- [claimed-docs] “Watch for a configurable label on pull requests, inspect full PR context, and post an AI review comment once per label event.”
HumanLayernone0/10HumanLayer's evidence covers task/session management, human-in-the-loop approvals, draft PR creation, and a PR diff-viewing UI, but there is no mention of an automated review process that generates inline review comments on every pull request. Missing for 10: no evidence of automatic PR review triggers, no mention of AI-generated inline comments, no review-quality-gate CI integration for PRs.
- [claimed-docs] “Draft PR creation — Ask the session agent to open a draft pull request from the GitHub tab or diff view.”
- [claimed-docs] “Keyboard navigation for changed files — Move through the PR changes tree with J/K, N/P, G shortcuts, and Enter.”
- [claimed-docs] “Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
Readiness checks
engineering-leadRun a readiness report that evaluates how ready my repository is for autonomous agents
weight 2 · round drawnOpenHandsnone0/10No evidence of a repository readiness assessment/report feature for autonomous agents; OpenHands' evidence covers task execution, automations, and integrations but nothing about scoring or auditing repo readiness.
Security remediation
engineering-leadHave security alerts automatically validated and remediated with an opened pull request
weight 2 · round to OpenHandsopenhands-docs-10 explicitly describes an automation that reviews security alerts, validates the finding, updates affected code, and opens a pull request with the remediation, directly matching the story. This is a first-party vendor claim without independent/hands-on corroboration or detail on validation accuracy. Missing for 10: independent verification of remediation quality, and detail on how 'validated' findings are confirmed before code changes.
- [claimed-docs] “Review security alerts, validate the finding, update affected code, and open a pull request with the remediation.”
HumanLayernone0/10HumanLayer's docs show generic task creation from GitHub/Jira/Linear issues and draft PR creation from agent sessions, but there is no evidence of any security-alert-specific validation or automated vulnerability remediation workflow (no CVE, dependency-alert, or security-scanner integration mentioned).
Scale parallelism — running many jobs at once — concurrency, fleets, queueingScale parallelism
Running many jobs at once — concurrency, fleets, queueing
Concurrent execution
engineering-leadRun many agent tasks concurrently to scale delivery throughput
weight 3 · round to OpenHandsOpenHands supports running agents across many backends (local, Docker, VMs, cloud) and building automations triggered by schedules or webhooks (Slack, GitHub, Linear), plus API access for scripting — all of which enable running multiple tasks in parallel across infrastructure. However, there is no explicit documentation of a dashboard or orchestration layer for managing many concurrent agent runs, concurrency limits, or throughput metrics for an engineering lead. Missing for 10: explicit multi-task concurrency/orchestration UI, stated concurrency limits or scaling guarantees, and independent evidence of teams running many parallel agents successfully.
- [github] “Create automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events”
- [github] “Run agents locally, in Docker, on VMs, or anywhere you can run an agent server backend”
- [claimed-docs] “API support for automation and scripting”
- [claimed-docs] “OpenHands Cloud is the managed commercial service for running OpenHands without operating your own backend and sandbox infrastructure.”
Docs describe an architecture (tasks/sessions, multi-repository workspaces, remote daemons on cloud VMs, and a CLI 'automation run' for CI/cron/scripts) that could support running many agent tasks in parallel, and 'Advanced Metrics' track usage/productivity across an org. However, tutorials and guides are framed around running one task/session at a time, and there is no explicit documentation or evidence of concurrent multi-task orchestration, throughput dashboards, or scaling guarantees for many simultaneous agents. Missing for 10: explicit multi-session concurrency docs, evidence of parallel task orchestration at scale, and independent confirmation of throughput gains.
- [claimed-docs] “A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A his…”
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [claimed-docs] “Configure a multi-repository workspace with this repository, ../api, and ../web. Make ../web the primary repository. Ask me before you choos…”
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Advanced Metrics for all paid plans — View usage, cost, and productivity metrics, with optional access for every organization member.”
engineering-leadCreate agent sessions on behalf of other users in my organization
weight 2 · round drawnOpenHandsnone0/10Evidence shows enterprise features like SSO/SAML and API access, which imply some multi-user org management, but there is no evidence of an engineering-lead being able to create or launch agent sessions on behalf of other users. Missing for 10: any documentation of delegated session creation, admin-initiated sessions, or role-based session provisioning for other org members.
- [claimed-docs] “Enterprise SAML / SSO”
- [claimed-docs] “API support for automation and scripting”
- [claimed-docs] “OpenHands Enterprise provides commercial capabilities and support for organizations that need licensed self-hosting or managed deployment op…”
HumanLayernone0/10The evidence describes tasks, sessions, integrations, and org-wide metrics access but never mentions an engineering-lead or admin creating/assigning agent sessions on behalf of another named user in the organization. No account-delegation, impersonation, or 'assign session to teammate' capability is documented.
Deployment flexibility
developerUse a managed cloud offering to run agents without operating my own backend infrastructure
weight 2 · round to OpenHandsOpenHands Cloud is explicitly documented as the managed commercial service for running OpenHands without operating your own backend and sandbox infrastructure, with hosted access, API support, and bring-your-own-LLM-key options detailed on the pricing page. Missing for 10: independent/hands-on corroboration of the cloud offering's reliability and scale, and more detail on infrastructure limits or SLAs.
- [claimed-docs] “OpenHands Cloud is the managed commercial service for running OpenHands without operating your own backend and sandbox infrastructure.”
- [claimed-docs] “Hosted cloud access from desktop and mobile”
- [claimed-docs] “API support for automation and scripting”
- [claimed-docs] “OpenHands Open Source and OpenHands Cloud plans offer support to bring your own LLM keys.”
HumanLayer offers a cloud control plane (app.humanlayer.com, automation sessions marked 'Cloud-visible', advanced metrics for paid plans) that lets users monitor and manage agent runs remotely, suggesting a managed service layer. However, docs explicitly state the actual agent execution still runs on a host that the user must select and operate—'a cloud VM, workstation, or private-network machine'—meaning the developer still needs to provision/manage backend compute for the daemon rather than HumanLayer fully hosting execution. Missing for 10: evidence of a fully HumanLayer-operated compute backend (no user-managed VM/daemon required), and independent confirmation of a true zero-ops managed runtime.
- [claimed-docs] “This tutorial teaches you how to run one task on a remote machine. You will control the task from app.humanlayer.com on a machine or phone.”
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
- [claimed-docs] “Advanced Metrics for all paid plans — View usage, cost, and productivity metrics, with optional access for every organization member.”
engineering-leadSelf-host agent infrastructure locally, in containers, or on my own VMs
weight 2 · round to OpenHandsGitHub docs explicitly state agents can be run locally, in Docker, on VMs, or any agent server backend, and Enterprise offers licensed self-hosting for organizations. This directly matches the engineering-lead's need for flexible self-hosted deployment. Missing for 10: independent/hands-on verification of self-hosting setup complexity and no detailed self-host deployment guide excerpt in evidence.
- [github] “Run agents locally, in Docker, on VMs, or anywhere you can run an agent server backend”
- [claimed-docs] “OpenHands Enterprise provides commercial capabilities and support for organizations that need licensed self-hosting or managed deployment op…”
- [github] “Switch between local, remote, and cloud agents without losing focus”
- [github] “Run OpenHands, Claude Code, Codex, Gemini, or any ACP-compatible agent across local, remote, and cloud backends.”
Docs describe running the remote daemon on a cloud VM, workstation, or private-network machine that you control (humanlayer-docs-7), plus automation sessions from CI/cron/server environments (humanlayer-docs-12) and launch tokens for bootstrap scripts (humanlayer-docs-13), showing self-hosted deployment flexibility. However there is no explicit mention of container/Docker deployment or an official container image, and no independent verification of self-hosted setups at scale. Missing for 10: explicit container/Docker packaging docs, independent hands-on confirmation of self-hosted deployments.
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
- [claimed-docs] “This tutorial teaches you how to run one task on a remote machine. You will control the task from app.humanlayer.com on a machine or phone.”
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
Headless automation
developerRun an agent headlessly inside CI/CD pipelines and shell scripts
weight 2 · round to HumanLayerDocs show a CLI mode with an 'auto-approve all actions' flag for non-interactive runs, and GitHub evidence describes running agents locally/Docker/VMs and triggering automations 'on a schedule or in response to webhook events,' plus API support for automation/scripting — all consistent with headless CI/CD use. Missing for 10: an explicit CI/CD pipeline example (e.g., GitHub Actions YAML snippet) or independent hands-on confirmation of headless exit-code/scripting behavior.
- [claimed-docs] “Auto-approve all actions (use with caution)”
- [probe] “official CLI documented at https://docs.openhands.dev/usage/how-to/cli-mode”
- [github] “Create automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events”
- [github] “Run agents locally, in Docker, on VMs, or anywhere you can run an agent server backend”
- [claimed-docs] “API support for automation and scripting”
Docs explicitly describe `humanlayer automation run` for running Cloud-visible coding sessions from CI jobs, cron machines, or scripts, plus launch tokens for non-interactive/non-PTY execution suited to headless environments. This directly matches running an agent headlessly in CI/CD and shell scripts. Missing for 10: independent/hands-on verification of CI usage and concrete pipeline examples (e.g. GitHub Actions config).
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
- [claimed-docs] “Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap scri…”
- [claimed-docs] “The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private…”
Not comparable on these axes
ai-native userConnect an agent via an official MCP server
weight 3 · not comparableOpenHandsn/aOpenHands is itself an AI coding agent, so serving as an MCP server for other agents is a different product role (agent-as-client, not server) — this axis is out of scope per the agent-role exception. No evidence shows OpenHands running as an MCP server endpoint.
HumanLayernone0/10HumanLayer is a platform/control-plane for running and overseeing coding-agent sessions (Claude Code, Codex) rather than itself being an agent, so an official MCP server is a fair, applicable axis. The evidence pack documents many integrations (Jira, Slack, GitHub, Linear), a CLI, and remote daemons, but no MCP server offering is mentioned anywhere, and API/OpenAPI probes returned 404s. missing for 10: any first-party MCP server documentation, endpoint, or 'mcp serve' style capability.
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.humanlayer.com/openapi.json, https://docs.humanlayer.com/swagger.json, https://docs.hum…”
- [probe] “PROBE llms.txt: HTTP 404 at https://docs.humanlayer.com/llms.txt”
- [claimed-docs] “Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a scr…”
developerQuery generated documentation for any public or private repository
weight 1 · not comparableOpenHandsnone0/10No evidence of a documentation-generation or repo-doc-querying feature; OpenHands is a coding agent focused on tasks, automations, and integrations, not a repo documentation search/query tool.
engineering-leadAutomatically fix failing agent-readiness criteria in my repository
weight 1 · not comparableOpenHandsnone0/10The evidence shows OpenHands can fix failing CI workflows, remediate security alerts, and review PRs, but nothing ties this to a defined 'agent-readiness' criteria/checklist concept that an engineering lead could gate on and auto-remediate. Missing for 10: any mention of agent-readiness scoring, a checklist/criteria framework, or evidence that OpenHands detects and fixes failures against such a standard.
HumanLayern/aHumanLayer is a human-in-the-loop agent orchestration/approval tool for running coding tasks, not a repository readiness/compliance scanner with auto-remediation of 'agent-readiness criteria'. This axis is a category error for this product type — no evidence pack content relates to detecting or auto-fixing repo readiness criteria.