How HumanLayer’s scores are calculated

The full audit trail, recomputed from the verdict data at build time through the same code that produced the leaderboard: verdict × quality × story weight per cell, cells sum to dimension scores, dimensions blend into the PA Score. Every number on the product page is reproducible from this page alone; for why the formula looks like this, see the methodology.

verdict factors: full ×1.0 · partial ×0.6 · disputed ×0.3 · none ×0.0 · n/a excluded from both sides · cell points = weight × quality × factor · cell max = weight × 10

PA Score19/100

Agent-ready 19.4 × 0.30 = 5.82

API quality 0.0 × 0.20 = 0.00

Openness 3.6 × 0.20 = 0.72

Built-in AI 59.8 × 0.15 = 8.97

Automation 23.3 × 0.15 = 3.50

(5.82 + 0.00 + 0.72 + 8.97 + 3.50) ÷ (0.30 + 0.20 + 0.20 + 0.15 + 0.15) = 19.00 ÷ 1.00 = 19.0

Scores are stored to 1 decimal; the product page’s pills round to whole numbers for display. Each dimension below shows the stories, verdicts, and cited evidence behind its number.

Agent-ready19.4/100×0.30 of the PA blend

Outside-in: can YOUR agent reach and drive this product — API, MCP, CLI, headless runs, agent docs.

Point an agent at llms.txt or agent-oriented docsweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

[probe] https://docs.humanlayer.com/llms.txt“PROBE llms.txt: HTTP 404 at https://docs.humanlayer.com/llms.txt”

Run the product headlessly / in CI for automationweight 2

2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max

[claimed-docs] https://docs.humanlayer.com/guide/automation-sessions“Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a script on a server.”
[claimed-docs] https://docs.humanlayer.com/guide/remote-daemons“Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap script.”
[claimed-docs] https://docs.humanlayer.com/explanation/remote-daemons“The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private services that the agent needs.”
[probe] https://docs.humanlayer.com/guide/remote-daemons“official CLI documented at https://docs.humanlayer.com/guide/remote-daemons”

Plug MCP servers into this product so it can use their toolsweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Connect an agent via an official MCP serverweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

[probe] https://docs.humanlayer.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.humanlayer.com/openapi.json, https://docs.humanlayer.com/swagger.json, https://docs.humanlayer.com/api/openapi.json, https://docs.humanlayer.com/.well-known/openapi.json)”
[probe] https://docs.humanlayer.com/llms.txt“PROBE llms.txt: HTTP 404 at https://docs.humanlayer.com/llms.txt”
[claimed-docs] https://docs.humanlayer.com/guide/automation-sessions“Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a script on a server.”

Use an official CLIweight 2

2 (weight) × 7 (quality) × 1.0 (full) = 14.0 of 20 max

[claimed-docs] https://docs.humanlayer.com/guide/automation-sessions“Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a script on a server.”
[claimed-docs] https://docs.humanlayer.com/guide/remote-daemons“Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap script.”
[claimed-docs] https://docs.humanlayer.com/explanation/remote-daemons“The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private services that the agent needs.”
[probe] https://docs.humanlayer.com/guide/remote-daemons“official CLI documented at https://docs.humanlayer.com/guide/remote-daemons”

Drive the product through a documented public APIweight 3

3 (weight) × 4 (quality) × 0.6 (partial) = 7.2 of 30 max

[claimed-docs] https://docs.humanlayer.com/guide/automation-sessions“Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a script on a server.”
[claimed-docs] https://docs.humanlayer.com/guide/remote-daemons“Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap script.”
[claimed-docs] https://docs.humanlayer.com/explanation/remote-daemons“The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private services that the agent needs.”
[probe] https://docs.humanlayer.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.humanlayer.com/openapi.json, https://docs.humanlayer.com/swagger.json, https://docs.humanlayer.com/api/openapi.json, https://docs.humanlayer.com/.well-known/openapi.json)”
[probe] https://docs.humanlayer.com/llms.txt“PROBE llms.txt: HTTP 404 at https://docs.humanlayer.com/llms.txt”
[probe] https://docs.humanlayer.com/guide/remote-daemons“official CLI documented at https://docs.humanlayer.com/guide/remote-daemons”

Issue scoped/least-privilege API credentials for an agentweight 2

2 (weight) × 3 (quality) × 0.6 (partial) = 3.6 of 20 max

[claimed-docs] https://docs.humanlayer.com/guide/remote-daemons“Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap script.”
[claimed-docs] https://docs.humanlayer.com/explanation/remote-daemons“The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private services that the agent needs.”

Build against official SDKsweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

[probe] https://docs.humanlayer.com/llms.txt“PROBE llms.txt: HTTP 404 at https://docs.humanlayer.com/llms.txt”
[probe] https://docs.humanlayer.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.humanlayer.com/openapi.json, https://docs.humanlayer.com/swagger.json, https://docs.humanlayer.com/api/openapi.json, https://docs.humanlayer.com/.well-known/openapi.json)”
[claimed-docs] https://docs.humanlayer.com/guide/automation-sessions“Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a script on a server.”
[claimed-docs] https://docs.humanlayer.com/guide/remote-daemons“Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap script.”

Subscribe to events via webhooksweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

[probe] https://docs.humanlayer.com/llms.txt“PROBE llms.txt: HTTP 404 at https://docs.humanlayer.com/llms.txt”
[probe] https://docs.humanlayer.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.humanlayer.com/openapi.json, https://docs.humanlayer.com/swagger.json, https://docs.humanlayer.com/api/openapi.json, https://docs.humanlayer.com/.well-known/openapi.json)”

Agent-ready = 40.8 ÷ 210 × 100 = 19.4

API quality0.0/100×0.20 of the PA blend

The programmable surface once an agent is there — machine-readable spec, interactive docs, sandbox, versioning discipline.

Explore an interactive API reference with runnable examplesweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

[probe] https://docs.humanlayer.com/llms.txt“PROBE llms.txt: HTTP 404 at https://docs.humanlayer.com/llms.txt”
[probe] https://docs.humanlayer.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.humanlayer.com/openapi.json, https://docs.humanlayer.com/swagger.json, https://docs.humanlayer.com/api/openapi.json, https://docs.humanlayer.com/.well-known/openapi.json)”

Download a machine-readable API spec (OpenAPI or equivalent)weight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

[probe] https://docs.humanlayer.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.humanlayer.com/openapi.json, https://docs.humanlayer.com/swagger.json, https://docs.humanlayer.com/api/openapi.json, https://docs.humanlayer.com/.well-known/openapi.json)”
[probe] https://docs.humanlayer.com/llms.txt“PROBE llms.txt: HTTP 404 at https://docs.humanlayer.com/llms.txt”

Test against a sandbox environment without touching production dataweight 1

1 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 10 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Rely on versioned APIs with a documented deprecation policyweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

[probe] https://docs.humanlayer.com/llms.txt“PROBE llms.txt: HTTP 404 at https://docs.humanlayer.com/llms.txt”
[probe] https://docs.humanlayer.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.humanlayer.com/openapi.json, https://docs.humanlayer.com/swagger.json, https://docs.humanlayer.com/api/openapi.json, https://docs.humanlayer.com/.well-known/openapi.json)”

API quality = 0.0 ÷ 70 × 100 = 0.0

Openness3.6/100×0.20 of the PA blend

Can you leave, inspect, or self-host — data export, open source, portability.

Do everything through the API that I can do in the UIweight 2

2 (weight) × 3 (quality) × 0.6 (partial) = 3.6 of 20 max

[probe] https://docs.humanlayer.com/guide/remote-daemons“official CLI documented at https://docs.humanlayer.com/guide/remote-daemons”
[claimed-docs] https://docs.humanlayer.com/guide/automation-sessions“Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a script on a server.”
[claimed-docs] https://docs.humanlayer.com/guide/remote-daemons“Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap script.”
[probe] https://docs.humanlayer.com/llms.txt“PROBE llms.txt: HTTP 404 at https://docs.humanlayer.com/llms.txt”
[probe] https://docs.humanlayer.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.humanlayer.com/openapi.json, https://docs.humanlayer.com/swagger.json, https://docs.humanlayer.com/api/openapi.json, https://docs.humanlayer.com/.well-known/openapi.json)”
[claimed-docs] https://docs.humanlayer.com/release-notes“Live multiplayer response drafts — Write prompts together with shared text, named cursors, presence, read-only viewing, and draft recovery.”
[claimed-docs] https://docs.humanlayer.com/release-notes“Keyboard navigation for changed files — Move through the PR changes tree with J/K, N/P, G shortcuts, and Enter.”

Export all of my data in open formats and leaveweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Read the product's source under an open licenseweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Self-host the core productweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

[claimed-docs] https://docs.humanlayer.com/tutorials/remote-daemon“This tutorial teaches you how to run one task on a remote machine. You will control the task from app.humanlayer.com on a machine or phone.”
[claimed-docs] https://docs.humanlayer.com/explanation/remote-daemons“The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private services that the agent needs.”
[claimed-docs] https://docs.humanlayer.com/guide/remote-daemons“Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap script.”

Openness = 3.6 ÷ 100 × 100 = 3.6

Built-in AI59.8/100×0.15 of the PA blend

Inside-out: how agentic the product itself is for its users — built-in assistants, autonomous features.

Get AI-generated insights and suggestions from my data inside the productweight 2

2 (weight) × 4 (quality) × 0.6 (partial) = 4.8 of 20 max

[claimed-docs] https://docs.humanlayer.com/release-notes“Advanced Metrics for all paid plans — View usage, cost, and productivity metrics, with optional access for every organization member.”
[claimed-docs] https://docs.humanlayer.com/release-notes“Draft PR creation — Ask the session agent to open a draft pull request from the GitHub tab or diff view.”
[claimed-docs] https://docs.humanlayer.com/explanation/tasks“A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A history that can continue across agents and workstations”

Set up automations that run autonomously in the backgroundweight 2

2 (weight) × 7 (quality) × 1.0 (full) = 14.0 of 20 max

[claimed-docs] https://docs.humanlayer.com/guide/automation-sessions“Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a script on a server.”
[claimed-docs] https://docs.humanlayer.com/guide/remote-daemons“Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap script.”
[claimed-docs] https://docs.humanlayer.com/explanation/remote-daemons“The host can be a cloud VM, workstation, or private-network machine. Select a host that can access the code, tools, credentials, and private services that the agent needs.”
[claimed-docs] https://docs.humanlayer.com/reference/workspace-config“Use `.humanlayer/workspace.json` for shared repository or team configuration. Use `.humanlayer/workspace.local.json` for optional user or machine overrides.”

Delegate tasks to a built-in AI assistant inside the productweight 3

3 (weight) × 7 (quality) × 1.0 (full) = 21.0 of 30 max

[claimed-docs] https://docs.humanlayer.com/tutorials/first-session“This tutorial teaches you how to create and run one small task in HumanLayer. You will use the macOS app and Claude Code.”
[claimed-docs] https://docs.humanlayer.com/explanation/tasks“A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A history that can continue across agents and workstations”
[claimed-docs] https://docs.humanlayer.com/explanation/workflow-phases“Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape needs more review before implementation.”
[claimed-docs] https://docs.humanlayer.com/reference/subagent-models“HumanLayer registers these RPI sub-agents for Claude Code sessions”
[claimed-docs] https://docs.humanlayer.com/guide/skills-workflows“Select **Oneshot**, **RPI**, **PRD-Oriented**, or **Freeform**.”
[claimed-docs] https://docs.humanlayer.com/guide/automation-sessions“Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a script on a server.”
[claimed-docs] https://docs.humanlayer.com/guide/codex“This guide shows you how to install, authenticate, select, and check Codex for HumanLayer sessions.”

Operate the product with natural-language commandsweight 2

2 (weight) × 7 (quality) × 1.0 (full) = 14.0 of 20 max

[claimed-docs] https://docs.humanlayer.com/guide/workspaces“Configure a multi-repository workspace with this repository, ../api, and ../web. Make ../web the primary repository. Ask me before you choose setup commands or local files to copy.”
[claimed-docs] https://docs.humanlayer.com/explanation/workflow-phases“Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape needs more review before implementation.”
[claimed-docs] https://docs.humanlayer.com/guide/skills-workflows“Select **Oneshot**, **RPI**, **PRD-Oriented**, or **Freeform**.”
[claimed-docs] https://docs.humanlayer.com/tutorials/first-session“This tutorial teaches you how to create and run one small task in HumanLayer. You will use the macOS app and Claude Code.”
[claimed-docs] https://docs.humanlayer.com/explanation/tasks“A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A history that can continue across agents and workstations”

Built-in AI = 53.8 ÷ 90 × 100 = 59.8

Automation23.3/100×0.15 of the PA blend

Depth of automation primitives — rules, scheduling, bulk operations, webhooks.

Perform bulk operations across many items at onceweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

[claimed-docs] https://docs.humanlayer.com/tutorials/first-session“This tutorial teaches you how to create and run one small task in HumanLayer. You will use the macOS app and Claude Code.”
[claimed-docs] https://docs.humanlayer.com/tutorials/remote-daemon“This tutorial teaches you how to run one task on a remote machine. You will control the task from app.humanlayer.com on a machine or phone.”
[claimed-docs] https://docs.humanlayer.com/guide/automation-sessions“Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a script on a server.”
[claimed-docs] https://docs.humanlayer.com/explanation/tasks“A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A history that can continue across agents and workstations”

Define rules that trigger actions automatically on eventsweight 3

3 (weight) × 6 (quality) × 0.6 (partial) = 10.8 of 30 max

[claimed-docs] https://docs.humanlayer.com/guide/jira-integration“Connect Jira Cloud so HumanLayer can create tasks from Jira tickets and keep ticket-driven agent work connected to its source.”
[claimed-docs] https://docs.humanlayer.com/guide/github-integration“Connect GitHub to create HumanLayer tasks from issues and link task artifacts back to the source issue.”
[claimed-docs] https://docs.humanlayer.com/guide/linear-integration“Connect Linear so HumanLayer can create tasks from Linear issues, sync issue status, and link HumanLayer artifacts back to the source issue.”
[claimed-docs] https://docs.humanlayer.com/guide/automation-sessions“Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a script on a server.”
[claimed-docs] https://docs.humanlayer.com/guide/remote-daemons“Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap script.”

Schedule recurring jobs or workflowsweight 2

2 (weight) × 4 (quality) × 0.6 (partial) = 4.8 of 20 max

[claimed-docs] https://docs.humanlayer.com/guide/automation-sessions“Use `humanlayer automation run` to run one Cloud-visible coding session from any automation environment — a CI job, a cron machine, or a script on a server.”
[claimed-docs] https://docs.humanlayer.com/guide/remote-daemons“Use a launch token for one non-interactive command. Examples include a shell without an interactive PTY and a one-time remote bootstrap script.”

Version, review, and roll back my automationsweight 1

1 (weight) × 5 (quality) × 0.6 (partial) = 3.0 of 10 max

[claimed-docs] https://docs.humanlayer.com/explanation/tasks“A HumanLayer task gives one piece of work: A group of related sessions, A shared set of task files, One place for comments and review, A history that can continue across agents and workstations”
[claimed-docs] https://docs.humanlayer.com/release-notes“Draft PR creation — Ask the session agent to open a draft pull request from the GitHub tab or diff view.”
[claimed-docs] https://docs.humanlayer.com/explanation/workflow-phases“Use the shortest path that controls the main risk. A clear and small change can use Oneshot. A change with unclear behavior or code shape needs more review before implementation.”

Automation = 18.6 ÷ 80 × 100 = 23.3