Skip to content

How OpenHands’s scores are calculated

The full audit trail, recomputed from the verdict data at build time through the same code that produced the leaderboard: verdict × quality × story weight per cell, cells sum to dimension scores, dimensions blend into the PA Score. Every number on the product page is reproducible from this page alone; for why the formula looks like this, see the methodology.

verdict factors: full ×1.0 · partial ×0.6 · disputed ×0.3 · none ×0.0 · n/a excluded from both sides · cell points = weight × quality × factor · cell max = weight × 10

PA Score50/100

Agent-ready 53.3 × 0.30 = 15.99

API quality 34.3 × 0.20 = 6.86

Openness 38.4 × 0.20 = 7.68

Built-in AI 77.8 × 0.15 = 11.67

Automation 53.8 × 0.15 = 8.07

(15.99 + 6.86 + 7.68 + 11.67 + 8.07) ÷ (0.30 + 0.20 + 0.20 + 0.15 + 0.15) = 50.27 ÷ 1.00 = 50.3

Scores are stored to 1 decimal; the product page’s pills round to whole numbers for display. Each dimension below shows the stories, verdicts, and cited evidence behind its number.

Agent-ready53.3/100×0.30 of the PA blend

Outside-in: can YOUR agent reach and drive this product — API, MCP, CLI, headless runs, agent docs.

Point an agent at llms.txt or agent-oriented docsweight 2

2 (weight) × 9 (quality) × 1.0 (full) = 18.0 of 20 max

  • [probe] https://docs.openhands.dev/llms.txtPROBE llms.txt: HTTP 200 at https://docs.openhands.dev/llms.txt # OpenHands Docs > LLM-friendly index of OpenHands documentation (V1). Legacy V0 docs pages are intentionally excluded.

Run the product headlessly / in CI for automationweight 2

2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max

  • [claimed-docs] https://docs.openhands.dev/usage/how-to/cli-modeAuto-approve all actions (use with caution)
  • [claimed-docs] https://www.openhands.dev/pricingAPI support for automation and scripting
  • [github] https://github.com/OpenHands/OpenHandsCreate automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events
  • [github] https://github.com/OpenHands/OpenHandsRun agents locally, in Docker, on VMs, or anywhere you can run an agent server backend
  • [probe] https://docs.openhands.dev/usage/how-to/cli-modeofficial CLI documented at https://docs.openhands.dev/usage/how-to/cli-mode

Plug MCP servers into this product so it can use their toolsweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Connect an agent via an official MCP serverweight 3

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Use an official CLIweight 2

2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max

  • [claimed-docs] https://docs.openhands.dev/usage/how-to/cli-modeType natural language tasks and receive instant feedback
  • [claimed-docs] https://docs.openhands.dev/usage/how-to/cli-modeResume previous conversations
  • [claimed-docs] https://docs.openhands.dev/usage/how-to/cli-modeAuto-approve all actions (use with caution)
  • [probe] https://docs.openhands.dev/usage/how-to/cli-modeofficial CLI documented at https://docs.openhands.dev/usage/how-to/cli-mode

Drive the product through a documented public APIweight 3

3 (weight) × 8 (quality) × 1.0 (full) = 24.0 of 30 max

  • [probe] https://docs.openhands.dev/openapi.jsonPROBE openapi: HTTP 200 at https://docs.openhands.dev/openapi.json — contains "openapi" key
  • [probe] https://docs.openhands.dev/llms.txtPROBE llms.txt: HTTP 200 at https://docs.openhands.dev/llms.txt # OpenHands Docs > LLM-friendly index of OpenHands documentation (V1). Legacy V0 docs pages are intentionally excluded.
  • [claimed-docs] https://www.openhands.dev/pricingAPI support for automation and scripting

Issue scoped/least-privilege API credentials for an agentweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Build against official SDKsweight 2

2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max

  • [claimed-docs] https://docs.openhands.devThe Software Agent SDK is a composable Python library for building agents that work with code.
  • [probe] https://docs.openhands.dev/openapi.jsonPROBE openapi: HTTP 200 at https://docs.openhands.dev/openapi.json — contains "openapi" key
  • [probe] https://docs.openhands.dev/llms.txtPROBE llms.txt: HTTP 200 at https://docs.openhands.dev/llms.txt # OpenHands Docs > LLM-friendly index of OpenHands documentation (V1). Legacy V0 docs pages are intentionally excluded.

Subscribe to events via webhooksweight 2

2 (weight) × 5 (quality) × 0.6 (partial) = 6.0 of 20 max

  • [github] https://github.com/OpenHands/OpenHandsCreate automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events

Agent-ready = 96.0 ÷ 180 × 100 = 53.3

API quality34.3/100×0.20 of the PA blend

The programmable surface once an agent is there — machine-readable spec, interactive docs, sandbox, versioning discipline.

Explore an interactive API reference with runnable examplesweight 2

2 (weight) × 3 (quality) × 0.6 (partial) = 3.6 of 20 max

  • [probe] https://docs.openhands.dev/openapi.jsonPROBE openapi: HTTP 200 at https://docs.openhands.dev/openapi.json — contains "openapi" key
  • [claimed-docs] https://www.openhands.dev/pricingAPI support for automation and scripting

Download a machine-readable API spec (OpenAPI or equivalent)weight 2

2 (weight) × 9 (quality) × 1.0 (full) = 18.0 of 20 max

  • [probe] https://docs.openhands.dev/openapi.jsonPROBE openapi: HTTP 200 at https://docs.openhands.dev/openapi.json — contains "openapi" key
  • [claimed-docs] https://www.openhands.dev/pricingAPI support for automation and scripting

Test against a sandbox environment without touching production dataweight 1

1 (weight) × 4 (quality) × 0.6 (partial) = 2.4 of 10 max

  • [github] https://github.com/OpenHands/OpenHandsRun agents locally, in Docker, on VMs, or anywhere you can run an agent server backend

Rely on versioned APIs with a documented deprecation policyweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [probe] https://docs.openhands.dev/openapi.jsonPROBE openapi: HTTP 200 at https://docs.openhands.dev/openapi.json — contains "openapi" key
  • [claimed-docs] https://www.openhands.dev/pricingAPI support for automation and scripting
  • [probe] https://docs.openhands.dev/llms.txtPROBE llms.txt: HTTP 200 at https://docs.openhands.dev/llms.txt # OpenHands Docs > LLM-friendly index of OpenHands documentation (V1). Legacy V0 docs pages are intentionally excluded.

API quality = 24.0 ÷ 70 × 100 = 34.3

Openness38.4/100×0.20 of the PA blend

Can you leave, inspect, or self-host — data export, open source, portability.

Do everything through the API that I can do in the UIweight 2

2 (weight) × 6 (quality) × 0.6 (partial) = 7.2 of 20 max

  • [probe] https://docs.openhands.dev/openapi.jsonPROBE openapi: HTTP 200 at https://docs.openhands.dev/openapi.json — contains "openapi" key
  • [claimed-docs] https://www.openhands.dev/pricingAPI support for automation and scripting
  • [claimed-docs] https://docs.openhands.dev/usage/how-to/cli-modeType natural language tasks and receive instant feedback
  • [claimed-docs] https://docs.openhands.dev/usage/how-to/cli-modeResume previous conversations
  • [claimed-docs] https://docs.openhands.dev/usage/how-to/cli-modeAuto-approve all actions (use with caution)
  • [probe] https://docs.openhands.dev/usage/how-to/cli-modeofficial CLI documented at https://docs.openhands.dev/usage/how-to/cli-mode

Export all of my data in open formats and leaveweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Read the product's source under an open licenseweight 2

2 (weight) × 6 (quality) × 0.6 (partial) = 7.2 of 20 max

  • [github] https://github.com/OpenHands/OpenHandsSwitch between local, remote, and cloud agents without losing focus
  • [claimed-docs] https://www.openhands.dev/pricingOpenHands Open Source and OpenHands Cloud plans offer support to bring your own LLM keys.

Self-host the core productweight 3

3 (weight) × 8 (quality) × 1.0 (full) = 24.0 of 30 max

  • [github] https://github.com/OpenHands/OpenHandsRun agents locally, in Docker, on VMs, or anywhere you can run an agent server backend
  • [claimed-docs] https://docs.openhands.devOpenHands Cloud is the managed commercial service for running OpenHands without operating your own backend and sandbox infrastructure.
  • [claimed-docs] https://docs.openhands.devThe Software Agent SDK is a composable Python library for building agents that work with code.
  • [claimed-docs] https://www.openhands.dev/pricingOpenHands Open Source and OpenHands Cloud plans offer support to bring your own LLM keys.

Openness = 38.4 ÷ 100 × 100 = 38.4

Built-in AI77.8/100×0.15 of the PA blend

Inside-out: how agentic the product itself is for its users — built-in assistants, autonomous features.

Get AI-generated insights and suggestions from my data inside the productweight 2

2 (weight) × 7 (quality) × 1.0 (full) = 14.0 of 20 max

  • [claimed-docs] https://www.openhands.devWatch for new bugs or incidents, gather logs and recent changes, suggest severity, and route the issue to the right owner.
  • [claimed-docs] https://www.openhands.devWatch for a configurable label on pull requests, inspect full PR context, and post an AI review comment once per label event.
  • [claimed-docs] https://www.openhands.devDetect failed workflows, inspect logs, identify the likely cause, and open a pull request with a proposed fix.
  • [claimed-docs] https://www.openhands.devReview security alerts, validate the finding, update affected code, and open a pull request with the remediation.
  • [claimed-docs] https://www.openhands.devWatch Slack channels for @openhands mentions, open a conversation with the message context, and reply when the agent finishes.

Set up automations that run autonomously in the backgroundweight 2

2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max

  • [github] https://github.com/OpenHands/OpenHandsCreate automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events
  • [claimed-docs] https://www.openhands.devWatch for new bugs or incidents, gather logs and recent changes, suggest severity, and route the issue to the right owner.
  • [claimed-docs] https://www.openhands.devWatch for a configurable label on pull requests, inspect full PR context, and post an AI review comment once per label event.
  • [claimed-docs] https://www.openhands.devDetect failed workflows, inspect logs, identify the likely cause, and open a pull request with a proposed fix.
  • [claimed-docs] https://www.openhands.devReview security alerts, validate the finding, update affected code, and open a pull request with the remediation.
  • [claimed-docs] https://www.openhands.devWatch Slack channels for @openhands mentions, open a conversation with the message context, and reply when the agent finishes.
  • [claimed-docs] https://www.openhands.dev/pricingAPI support for automation and scripting

Delegate tasks to a built-in AI assistant inside the productweight 3

3 (weight) × 8 (quality) × 1.0 (full) = 24.0 of 30 max

  • [claimed-docs] https://docs.openhands.dev/usage/how-to/cli-modeType natural language tasks and receive instant feedback
  • [claimed-docs] https://docs.openhands.dev/usage/how-to/cli-modeResume previous conversations
  • [claimed-docs] https://docs.openhands.dev/usage/how-to/cli-modeAuto-approve all actions (use with caution)
  • [github] https://github.com/OpenHands/OpenHandsSwitch between local, remote, and cloud agents without losing focus
  • [github] https://github.com/OpenHands/OpenHandsUse with any LLM
  • [probe] https://docs.openhands.dev/usage/how-to/cli-modeofficial CLI documented at https://docs.openhands.dev/usage/how-to/cli-mode

Operate the product with natural-language commandsweight 2

2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max

  • [claimed-docs] https://docs.openhands.dev/usage/how-to/cli-modeType natural language tasks and receive instant feedback
  • [claimed-docs] https://docs.openhands.dev/usage/how-to/cli-modeResume previous conversations
  • [claimed-docs] https://docs.openhands.dev/usage/how-to/cli-modeAuto-approve all actions (use with caution)
  • [probe] https://docs.openhands.dev/usage/how-to/cli-modeofficial CLI documented at https://docs.openhands.dev/usage/how-to/cli-mode

Built-in AI = 70.0 ÷ 90 × 100 = 77.8

Automation53.8/100×0.15 of the PA blend

Depth of automation primitives — rules, scheduling, bulk operations, webhooks.

Perform bulk operations across many items at onceweight 2

2 (weight) × 5 (quality) × 0.6 (partial) = 6.0 of 20 max

  • [github] https://github.com/OpenHands/OpenHandsCreate automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events
  • [claimed-docs] https://www.openhands.dev/pricingAPI support for automation and scripting
  • [claimed-docs] https://www.openhands.devWatch for new bugs or incidents, gather logs and recent changes, suggest severity, and route the issue to the right owner.
  • [claimed-docs] https://www.openhands.devDetect failed workflows, inspect logs, identify the likely cause, and open a pull request with a proposed fix.

Define rules that trigger actions automatically on eventsweight 3

3 (weight) × 7 (quality) × 1.0 (full) = 21.0 of 30 max

  • [github] https://github.com/OpenHands/OpenHandsCreate automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events
  • [claimed-docs] https://www.openhands.devWatch for new bugs or incidents, gather logs and recent changes, suggest severity, and route the issue to the right owner.
  • [claimed-docs] https://www.openhands.devWatch for a configurable label on pull requests, inspect full PR context, and post an AI review comment once per label event.
  • [claimed-docs] https://www.openhands.devDetect failed workflows, inspect logs, identify the likely cause, and open a pull request with a proposed fix.
  • [claimed-docs] https://www.openhands.devReview security alerts, validate the finding, update affected code, and open a pull request with the remediation.
  • [claimed-docs] https://www.openhands.devWatch Slack channels for @openhands mentions, open a conversation with the message context, and reply when the agent finishes.

Schedule recurring jobs or workflowsweight 2

2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max

  • [github] https://github.com/OpenHands/OpenHandsCreate automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events
  • [claimed-docs] https://www.openhands.devWatch for new bugs or incidents, gather logs and recent changes, suggest severity, and route the issue to the right owner.
  • [claimed-docs] https://www.openhands.devWatch for a configurable label on pull requests, inspect full PR context, and post an AI review comment once per label event.
  • [claimed-docs] https://www.openhands.devDetect failed workflows, inspect logs, identify the likely cause, and open a pull request with a proposed fix.
  • [claimed-docs] https://www.openhands.devReview security alerts, validate the finding, update affected code, and open a pull request with the remediation.
  • [claimed-docs] https://www.openhands.devWatch Slack channels for @openhands mentions, open a conversation with the message context, and reply when the agent finishes.

Version, review, and roll back my automationsweight 1

1 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 10 max

  • [github] https://github.com/OpenHands/OpenHandsCreate automations and workflows that integrate with Slack, GitHub, Linear, and more. Run on a schedule or in response to webhook events
  • [claimed-docs] https://www.openhands.devWatch for new bugs or incidents, gather logs and recent changes, suggest severity, and route the issue to the right owner.
  • [claimed-docs] https://www.openhands.devWatch for a configurable label on pull requests, inspect full PR context, and post an AI review comment once per label event.
  • [claimed-docs] https://www.openhands.devDetect failed workflows, inspect logs, identify the likely cause, and open a pull request with a proposed fix.
  • [claimed-docs] https://www.openhands.devReview security alerts, validate the finding, update affected code, and open a pull request with the remediation.
  • [claimed-docs] https://www.openhands.devWatch Slack channels for @openhands mentions, open a conversation with the message context, and reply when the agent finishes.

Automation = 43.0 ÷ 80 × 100 = 53.8