Skip to content

How Codegen’s scores are calculated

The full audit trail, recomputed from the verdict data at build time through the same code that produced the leaderboard: verdict × quality × story weight per cell, cells sum to dimension scores, dimensions blend into the PA Score. Every number on the product page is reproducible from this page alone; for why the formula looks like this, see the methodology.

verdict factors: full ×1.0 · partial ×0.6 · disputed ×0.3 · none ×0.0 · n/a excluded from both sides · cell points = weight × quality × factor · cell max = weight × 10

PA Score32/100

Agent-ready 56.0 × 0.30 = 16.80

API quality 4.3 × 0.20 = 0.86

Openness 6.0 × 0.20 = 1.20

Built-in AI 66.7 × 0.15 = 10.01

Automation 18.8 × 0.15 = 2.82

(16.80 + 0.86 + 1.20 + 10.01 + 2.82) ÷ (0.30 + 0.20 + 0.20 + 0.15 + 0.15) = 31.69 ÷ 1.00 = 31.7

Scores are stored to 1 decimal; the product page’s pills round to whole numbers for display. Each dimension below shows the stories, verdicts, and cited evidence behind its number.

Agent-ready56.0/100×0.30 of the PA blend

Outside-in: can YOUR agent reach and drive this product — API, MCP, CLI, headless runs, agent docs.

Point an agent at llms.txt or agent-oriented docsweight 2

2 (weight) × 9 (quality) × 1.0 (full) = 18.0 of 20 max

  • [probe] https://docs.codegen.com/llms.txtPROBE llms.txt: HTTP 200 at https://docs.codegen.com/llms.txt # Codegen ## Docs - [Codegen](https://docs.codegen.com/introduction/overview.md) - [Use Cases](https://docs.codegen.co
  • [claimed-docs] https://docs.codegen.comAnalyze requirements, implement features, fix bugs, write tests, and improve documentation based on your prompts.
  • [claimed-docs] https://docs.codegen.comProgrammatically interact with agents using the Python SDK for advanced automation.

Run the product headlessly / in CI for automationweight 2

2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max

  • [claimed-docs] https://docs.codegen.comProgrammatically interact with agents using the Python SDK for advanced automation.
  • [claimed-docs] https://docs.codegen.com/introduction/sdkagent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")
  • [claimed-docs] https://docs.codegen.com/api-reference/overviewCreate and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic control over their execution and monitoring.
  • [claimed-docs] https://docs.codegen.com/api-reference/overviewprogrammatically retrieve detailed agent traces for analysis and debugging
  • [claimed-docs] https://docs.codegen.com/api-reference/overviewTrigger agents from CI/CD pipelines when builds fail
  • [claimed-docs] https://docs.codegen.com/introduction/cliUse it to view agents, pull their work, create new agents, and run Claude Code with full telemetry and monitoring.
  • [probe] https://docs.codegen.com/openapi.jsonPROBE openapi: all candidate paths 404 (https://docs.codegen.com/openapi.json, https://docs.codegen.com/swagger.json, https://docs.codegen.com/api/openapi.json, https://docs.codegen.com/.well-known/openapi.json)

Plug MCP servers into this product so it can use their toolsweight 3

3 (weight) × 6 (quality) × 0.6 (partial) = 10.8 of 30 max

  • [claimed-docs] https://docs.codegen.comConnect with Slack, Linear, Figma, databases, and extend capabilities with custom MCP tools.
  • [claimed-docs] https://docs.codegen.comLog local Claude Code instances to the cloud and provision MCP servers across your org.

Connect an agent via an official MCP serverweight 3

n/a — not applicable to this product: excluded from numerator and denominator

  • [claimed-docs] https://docs.codegen.comConnect with Slack, Linear, Figma, databases, and extend capabilities with custom MCP tools.
  • [claimed-docs] https://docs.codegen.comLog local Claude Code instances to the cloud and provision MCP servers across your org.

Use an official CLIweight 2

2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max

  • [claimed-docs] https://docs.codegen.com/introduction/cliUse it to view agents, pull their work, create new agents, and run Claude Code with full telemetry and monitoring.
  • [claimed-docs] https://docs.codegen.com/introduction/cliPull agent work - Download branches and code changes created by agents directly to your local environment
  • [claimed-docs] https://docs.codegen.com/introduction/cliManage organizations - Switch between organizations and configure repositories
  • [probe] https://docs.codegen.com/introduction/cliofficial CLI documented at https://docs.codegen.com/introduction/cli

Drive the product through a documented public APIweight 3

3 (weight) × 8 (quality) × 1.0 (full) = 24.0 of 30 max

  • [claimed-docs] https://docs.codegen.comProgrammatically interact with agents using the Python SDK for advanced automation.
  • [claimed-docs] https://docs.codegen.com/introduction/cliUse it to view agents, pull their work, create new agents, and run Claude Code with full telemetry and monitoring.
  • [claimed-docs] https://docs.codegen.com/introduction/sdkagent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")
  • [claimed-docs] https://docs.codegen.com/api-reference/overviewCreate and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic control over their execution and monitoring.
  • [claimed-docs] https://docs.codegen.com/api-reference/overviewprogrammatically retrieve detailed agent traces for analysis and debugging
  • [claimed-docs] https://docs.codegen.com/api-reference/overviewTrigger agents from CI/CD pipelines when builds fail
  • [probe] https://docs.codegen.com/openapi.jsonPROBE openapi: all candidate paths 404 (https://docs.codegen.com/openapi.json, https://docs.codegen.com/swagger.json, https://docs.codegen.com/api/openapi.json, https://docs.codegen.com/.well-known/openapi.json)
  • [probe] https://docs.codegen.com/introduction/cliofficial CLI documented at https://docs.codegen.com/introduction/cli

Issue scoped/least-privilege API credentials for an agentweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [claimed-docs] https://docs.codegen.com/introduction/sdkagent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")
  • [claimed-docs] https://docs.codegen.com/api-reference/overviewCreate and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic control over their execution and monitoring.

Build against official SDKsweight 2

2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max

  • [claimed-docs] https://docs.codegen.comProgrammatically interact with agents using the Python SDK for advanced automation.
  • [claimed-docs] https://docs.codegen.com/introduction/sdkagent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")
  • [claimed-docs] https://docs.codegen.com/api-reference/overviewCreate and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic control over their execution and monitoring.
  • [claimed-docs] https://docs.codegen.com/api-reference/overviewprogrammatically retrieve detailed agent traces for analysis and debugging
  • [claimed-docs] https://docs.codegen.com/api-reference/overviewTrigger agents from CI/CD pipelines when builds fail
  • [probe] https://docs.codegen.com/openapi.jsonPROBE openapi: all candidate paths 404 (https://docs.codegen.com/openapi.json, https://docs.codegen.com/swagger.json, https://docs.codegen.com/api/openapi.json, https://docs.codegen.com/.well-known/openapi.json)
  • [probe] https://docs.codegen.com/introduction/cliofficial CLI documented at https://docs.codegen.com/introduction/cli

Subscribe to events via webhooksweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Agent-ready = 100.8 ÷ 180 × 100 = 56.0

API quality4.3/100×0.20 of the PA blend

The programmable surface once an agent is there — machine-readable spec, interactive docs, sandbox, versioning discipline.

Explore an interactive API reference with runnable examplesweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [claimed-docs] https://docs.codegen.com/api-reference/overviewCreate and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic control over their execution and monitoring.
  • [claimed-docs] https://docs.codegen.com/api-reference/overviewprogrammatically retrieve detailed agent traces for analysis and debugging
  • [claimed-docs] https://docs.codegen.com/api-reference/overviewTrigger agents from CI/CD pipelines when builds fail
  • [probe] https://docs.codegen.com/openapi.jsonPROBE openapi: all candidate paths 404 (https://docs.codegen.com/openapi.json, https://docs.codegen.com/swagger.json, https://docs.codegen.com/api/openapi.json, https://docs.codegen.com/.well-known/openapi.json)

Download a machine-readable API spec (OpenAPI or equivalent)weight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [probe] https://docs.codegen.com/openapi.jsonPROBE openapi: all candidate paths 404 (https://docs.codegen.com/openapi.json, https://docs.codegen.com/swagger.json, https://docs.codegen.com/api/openapi.json, https://docs.codegen.com/.well-known/openapi.json)
  • [claimed-docs] https://docs.codegen.com/api-reference/overviewCreate and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic control over their execution and monitoring.

Test against a sandbox environment without touching production dataweight 1

1 (weight) × 5 (quality) × 0.6 (partial) = 3.0 of 10 max

  • [claimed-docs] https://docs.codegen.comSafely run code, install dependencies, and test changes in robust isolated environments

Rely on versioned APIs with a documented deprecation policyweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [claimed-docs] https://docs.codegen.com/api-reference/overviewCreate and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic control over their execution and monitoring.
  • [probe] https://docs.codegen.com/openapi.jsonPROBE openapi: all candidate paths 404 (https://docs.codegen.com/openapi.json, https://docs.codegen.com/swagger.json, https://docs.codegen.com/api/openapi.json, https://docs.codegen.com/.well-known/openapi.json)

API quality = 3.0 ÷ 70 × 100 = 4.3

Openness6.0/100×0.20 of the PA blend

Can you leave, inspect, or self-host — data export, open source, portability.

Do everything through the API that I can do in the UIweight 2

2 (weight) × 5 (quality) × 0.6 (partial) = 6.0 of 20 max

  • [claimed-docs] https://docs.codegen.comProgrammatically interact with agents using the Python SDK for advanced automation.
  • [claimed-docs] https://docs.codegen.com/introduction/sdkagent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")
  • [claimed-docs] https://docs.codegen.com/api-reference/overviewCreate and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic control over their execution and monitoring.
  • [claimed-docs] https://docs.codegen.com/api-reference/overviewprogrammatically retrieve detailed agent traces for analysis and debugging
  • [claimed-docs] https://docs.codegen.com/api-reference/overviewTrigger agents from CI/CD pipelines when builds fail
  • [probe] https://docs.codegen.com/openapi.jsonPROBE openapi: all candidate paths 404 (https://docs.codegen.com/openapi.json, https://docs.codegen.com/swagger.json, https://docs.codegen.com/api/openapi.json, https://docs.codegen.com/.well-known/openapi.json)

Export all of my data in open formats and leaveweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

  • [claimed-docs] https://docs.codegen.com/introduction/cliPull agent work - Download branches and code changes created by agents directly to your local environment
  • [claimed-docs] https://docs.codegen.com/api-reference/overviewprogrammatically retrieve detailed agent traces for analysis and debugging

Read the product's source under an open licenseweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Self-host the core productweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Openness = 6.0 ÷ 100 × 100 = 6.0

Built-in AI66.7/100×0.15 of the PA blend

Inside-out: how agentic the product itself is for its users — built-in assistants, autonomous features.

Get AI-generated insights and suggestions from my data inside the productweight 2

2 (weight) × 5 (quality) × 0.6 (partial) = 6.0 of 20 max

  • [claimed-docs] https://codegen.comReads the PR diff Summarizes changes across files Runs security and secrets scan Checks test coverage Posts a review comment on failure
  • [claimed-docs] https://docs.codegen.comReview PRs, suggest changes, comment on issues, create branches, commit code, and manage repositories.
  • [claimed-docs] https://docs.codegen.comConnect with Slack, Linear, Figma, databases, and extend capabilities with custom MCP tools.

Set up automations that run autonomously in the backgroundweight 2

2 (weight) × 7 (quality) × 1.0 (full) = 14.0 of 20 max

  • [claimed-docs] https://docs.codegen.com/api-reference/overviewTrigger agents from CI/CD pipelines when builds fail
  • [claimed-docs] https://codegen.comReads the PR diff Summarizes changes across files Runs security and secrets scan Checks test coverage Posts a review comment on failure
  • [claimed-docs] https://docs.codegen.comSend notifications, ask for clarification, report progress, and interact directly with your team in Slack channels.
  • [claimed-docs] https://docs.codegen.comUpdate statuses, add comments, link PRs to issues, and create new tasks. Support for Jira, Linear, Clickup and Monday.com.
  • [claimed-docs] https://docs.codegen.comProgrammatically interact with agents using the Python SDK for advanced automation.
  • [claimed-docs] https://docs.codegen.com/introduction/sdkagent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")
  • [claimed-docs] https://docs.codegen.com/api-reference/overviewCreate and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic control over their execution and monitoring.

Delegate tasks to a built-in AI assistant inside the productweight 3

3 (weight) × 8 (quality) × 1.0 (full) = 24.0 of 30 max

  • [claimed-docs] https://docs.codegen.comAnalyze requirements, implement features, fix bugs, write tests, and improve documentation based on your prompts.
  • [claimed-docs] https://docs.codegen.comSend notifications, ask for clarification, report progress, and interact directly with your team in Slack channels.
  • [claimed-docs] https://docs.codegen.comReview PRs, suggest changes, comment on issues, create branches, commit code, and manage repositories.
  • [claimed-docs] https://docs.codegen.com/introduction/sdkagent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")
  • [claimed-docs] https://docs.codegen.com/api-reference/overviewCreate and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic control over their execution and monitoring.

Operate the product with natural-language commandsweight 2

2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max

  • [claimed-docs] https://docs.codegen.comAnalyze requirements, implement features, fix bugs, write tests, and improve documentation based on your prompts.
  • [claimed-docs] https://docs.codegen.comSend notifications, ask for clarification, report progress, and interact directly with your team in Slack channels.
  • [claimed-docs] https://docs.codegen.comUpdate statuses, add comments, link PRs to issues, and create new tasks. Support for Jira, Linear, Clickup and Monday.com.
  • [claimed-docs] https://docs.codegen.comReview PRs, suggest changes, comment on issues, create branches, commit code, and manage repositories.
  • [claimed-docs] https://docs.codegen.com/introduction/sdkagent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")
  • [claimed-docs] https://docs.codegen.com/api-reference/overviewCreate and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic control over their execution and monitoring.

Built-in AI = 60.0 ÷ 90 × 100 = 66.7

Automation18.8/100×0.15 of the PA blend

Depth of automation primitives — rules, scheduling, bulk operations, webhooks.

Perform bulk operations across many items at onceweight 2

2 (weight) × 5 (quality) × 0.6 (partial) = 6.0 of 20 max

  • [claimed-docs] https://docs.codegen.comProgrammatically interact with agents using the Python SDK for advanced automation.
  • [claimed-docs] https://docs.codegen.com/introduction/sdkagent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")
  • [claimed-docs] https://docs.codegen.com/api-reference/overviewCreate and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic control over their execution and monitoring.
  • [claimed-docs] https://docs.codegen.com/api-reference/overviewTrigger agents from CI/CD pipelines when builds fail

Define rules that trigger actions automatically on eventsweight 3

3 (weight) × 5 (quality) × 0.6 (partial) = 9.0 of 30 max

  • [claimed-docs] https://docs.codegen.com/api-reference/overviewTrigger agents from CI/CD pipelines when builds fail
  • [claimed-docs] https://codegen.comReads the PR diff Summarizes changes across files Runs security and secrets scan Checks test coverage Posts a review comment on failure
  • [claimed-docs] https://docs.codegen.comProgrammatically interact with agents using the Python SDK for advanced automation.
  • [claimed-docs] https://docs.codegen.com/api-reference/overviewCreate and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic control over their execution and monitoring.

Schedule recurring jobs or workflowsweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Version, review, and roll back my automationsweight 1

1 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 10 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Automation = 15.0 ÷ 80 × 100 = 18.8