How Codegen’s scores are calculated
The full audit trail, recomputed from the verdict data at build time through the same code that produced the leaderboard: verdict × quality × story weight per cell, cells sum to dimension scores, dimensions blend into the PA Score. Every number on the product page is reproducible from this page alone; for why the formula looks like this, see the methodology.
verdict factors: full ×1.0 · partial ×0.6 · disputed ×0.3 · none ×0.0 · n/a excluded from both sides · cell points = weight × quality × factor · cell max = weight × 10
PA Score32/100
Agent-ready 56.0 × 0.30 = 16.80
API quality 4.3 × 0.20 = 0.86
Openness 6.0 × 0.20 = 1.20
Built-in AI 66.7 × 0.15 = 10.01
Automation 18.8 × 0.15 = 2.82
(16.80 + 0.86 + 1.20 + 10.01 + 2.82) ÷ (0.30 + 0.20 + 0.20 + 0.15 + 0.15) = 31.69 ÷ 1.00 = 31.7
Scores are stored to 1 decimal; the product page’s pills round to whole numbers for display. Each dimension below shows the stories, verdicts, and cited evidence behind its number.
Agent-ready56.0/100×0.30 of the PA blend
Outside-in: can YOUR agent reach and drive this product — API, MCP, CLI, headless runs, agent docs.
Point an agent at llms.txt or agent-oriented docsweight 2
2 (weight) × 9 (quality) × 1.0 (full) = 18.0 of 20 max
- [probe] https://docs.codegen.com/llms.txt“PROBE llms.txt: HTTP 200 at https://docs.codegen.com/llms.txt # Codegen ## Docs - [Codegen](https://docs.codegen.com/introduction/overview.md) - [Use Cases](https://docs.codegen.co”
- [claimed-docs] https://docs.codegen.com“Analyze requirements, implement features, fix bugs, write tests, and improve documentation based on your prompts.”
- [claimed-docs] https://docs.codegen.com“Programmatically interact with agents using the Python SDK for advanced automation.”
Run the product headlessly / in CI for automationweight 2
2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max
- [claimed-docs] https://docs.codegen.com“Programmatically interact with agents using the Python SDK for advanced automation.”
- [claimed-docs] https://docs.codegen.com/introduction/sdk“agent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")”
- [claimed-docs] https://docs.codegen.com/api-reference/overview“Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic control over their execution and monitoring.”
- [claimed-docs] https://docs.codegen.com/api-reference/overview“programmatically retrieve detailed agent traces for analysis and debugging”
- [claimed-docs] https://docs.codegen.com/api-reference/overview“Trigger agents from CI/CD pipelines when builds fail”
- [claimed-docs] https://docs.codegen.com/introduction/cli“Use it to view agents, pull their work, create new agents, and run Claude Code with full telemetry and monitoring.”
- [probe] https://docs.codegen.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.codegen.com/openapi.json, https://docs.codegen.com/swagger.json, https://docs.codegen.com/api/openapi.json, https://docs.codegen.com/.well-known/openapi.json)”
Plug MCP servers into this product so it can use their toolsweight 3
3 (weight) × 6 (quality) × 0.6 (partial) = 10.8 of 30 max
- [claimed-docs] https://docs.codegen.com“Connect with Slack, Linear, Figma, databases, and extend capabilities with custom MCP tools.”
- [claimed-docs] https://docs.codegen.com“Log local Claude Code instances to the cloud and provision MCP servers across your org.”
Connect an agent via an official MCP serverweight 3
n/a — not applicable to this product: excluded from numerator and denominator
- [claimed-docs] https://docs.codegen.com“Connect with Slack, Linear, Figma, databases, and extend capabilities with custom MCP tools.”
- [claimed-docs] https://docs.codegen.com“Log local Claude Code instances to the cloud and provision MCP servers across your org.”
Use an official CLIweight 2
2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max
- [claimed-docs] https://docs.codegen.com/introduction/cli“Use it to view agents, pull their work, create new agents, and run Claude Code with full telemetry and monitoring.”
- [claimed-docs] https://docs.codegen.com/introduction/cli“Pull agent work - Download branches and code changes created by agents directly to your local environment”
- [claimed-docs] https://docs.codegen.com/introduction/cli“Manage organizations - Switch between organizations and configure repositories”
- [probe] https://docs.codegen.com/introduction/cli“official CLI documented at https://docs.codegen.com/introduction/cli”
Drive the product through a documented public APIweight 3
3 (weight) × 8 (quality) × 1.0 (full) = 24.0 of 30 max
- [claimed-docs] https://docs.codegen.com“Programmatically interact with agents using the Python SDK for advanced automation.”
- [claimed-docs] https://docs.codegen.com/introduction/cli“Use it to view agents, pull their work, create new agents, and run Claude Code with full telemetry and monitoring.”
- [claimed-docs] https://docs.codegen.com/introduction/sdk“agent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")”
- [claimed-docs] https://docs.codegen.com/api-reference/overview“Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic control over their execution and monitoring.”
- [claimed-docs] https://docs.codegen.com/api-reference/overview“programmatically retrieve detailed agent traces for analysis and debugging”
- [claimed-docs] https://docs.codegen.com/api-reference/overview“Trigger agents from CI/CD pipelines when builds fail”
- [probe] https://docs.codegen.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.codegen.com/openapi.json, https://docs.codegen.com/swagger.json, https://docs.codegen.com/api/openapi.json, https://docs.codegen.com/.well-known/openapi.json)”
- [probe] https://docs.codegen.com/introduction/cli“official CLI documented at https://docs.codegen.com/introduction/cli”
Issue scoped/least-privilege API credentials for an agentweight 2
2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max
- [claimed-docs] https://docs.codegen.com/introduction/sdk“agent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")”
- [claimed-docs] https://docs.codegen.com/api-reference/overview“Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic control over their execution and monitoring.”
Build against official SDKsweight 2
2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max
- [claimed-docs] https://docs.codegen.com“Programmatically interact with agents using the Python SDK for advanced automation.”
- [claimed-docs] https://docs.codegen.com/introduction/sdk“agent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")”
- [claimed-docs] https://docs.codegen.com/api-reference/overview“Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic control over their execution and monitoring.”
- [claimed-docs] https://docs.codegen.com/api-reference/overview“programmatically retrieve detailed agent traces for analysis and debugging”
- [claimed-docs] https://docs.codegen.com/api-reference/overview“Trigger agents from CI/CD pipelines when builds fail”
- [probe] https://docs.codegen.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.codegen.com/openapi.json, https://docs.codegen.com/swagger.json, https://docs.codegen.com/api/openapi.json, https://docs.codegen.com/.well-known/openapi.json)”
- [probe] https://docs.codegen.com/introduction/cli“official CLI documented at https://docs.codegen.com/introduction/cli”
Subscribe to events via webhooksweight 2
2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
Agent-ready = 100.8 ÷ 180 × 100 = 56.0
API quality4.3/100×0.20 of the PA blend
The programmable surface once an agent is there — machine-readable spec, interactive docs, sandbox, versioning discipline.
Explore an interactive API reference with runnable examplesweight 2
2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max
- [claimed-docs] https://docs.codegen.com/api-reference/overview“Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic control over their execution and monitoring.”
- [claimed-docs] https://docs.codegen.com/api-reference/overview“programmatically retrieve detailed agent traces for analysis and debugging”
- [claimed-docs] https://docs.codegen.com/api-reference/overview“Trigger agents from CI/CD pipelines when builds fail”
- [probe] https://docs.codegen.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.codegen.com/openapi.json, https://docs.codegen.com/swagger.json, https://docs.codegen.com/api/openapi.json, https://docs.codegen.com/.well-known/openapi.json)”
Download a machine-readable API spec (OpenAPI or equivalent)weight 2
2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max
- [probe] https://docs.codegen.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.codegen.com/openapi.json, https://docs.codegen.com/swagger.json, https://docs.codegen.com/api/openapi.json, https://docs.codegen.com/.well-known/openapi.json)”
- [claimed-docs] https://docs.codegen.com/api-reference/overview“Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic control over their execution and monitoring.”
Test against a sandbox environment without touching production dataweight 1
1 (weight) × 5 (quality) × 0.6 (partial) = 3.0 of 10 max
- [claimed-docs] https://docs.codegen.com“Safely run code, install dependencies, and test changes in robust isolated environments”
Rely on versioned APIs with a documented deprecation policyweight 2
2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max
- [claimed-docs] https://docs.codegen.com/api-reference/overview“Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic control over their execution and monitoring.”
- [probe] https://docs.codegen.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.codegen.com/openapi.json, https://docs.codegen.com/swagger.json, https://docs.codegen.com/api/openapi.json, https://docs.codegen.com/.well-known/openapi.json)”
API quality = 3.0 ÷ 70 × 100 = 4.3
Openness6.0/100×0.20 of the PA blend
Can you leave, inspect, or self-host — data export, open source, portability.
Do everything through the API that I can do in the UIweight 2
2 (weight) × 5 (quality) × 0.6 (partial) = 6.0 of 20 max
- [claimed-docs] https://docs.codegen.com“Programmatically interact with agents using the Python SDK for advanced automation.”
- [claimed-docs] https://docs.codegen.com/introduction/sdk“agent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")”
- [claimed-docs] https://docs.codegen.com/api-reference/overview“Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic control over their execution and monitoring.”
- [claimed-docs] https://docs.codegen.com/api-reference/overview“programmatically retrieve detailed agent traces for analysis and debugging”
- [claimed-docs] https://docs.codegen.com/api-reference/overview“Trigger agents from CI/CD pipelines when builds fail”
- [probe] https://docs.codegen.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.codegen.com/openapi.json, https://docs.codegen.com/swagger.json, https://docs.codegen.com/api/openapi.json, https://docs.codegen.com/.well-known/openapi.json)”
Export all of my data in open formats and leaveweight 3
3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max
- [claimed-docs] https://docs.codegen.com/introduction/cli“Pull agent work - Download branches and code changes created by agents directly to your local environment”
- [claimed-docs] https://docs.codegen.com/api-reference/overview“programmatically retrieve detailed agent traces for analysis and debugging”
Read the product's source under an open licenseweight 2
2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
Self-host the core productweight 3
3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
Openness = 6.0 ÷ 100 × 100 = 6.0
Built-in AI66.7/100×0.15 of the PA blend
Inside-out: how agentic the product itself is for its users — built-in assistants, autonomous features.
Get AI-generated insights and suggestions from my data inside the productweight 2
2 (weight) × 5 (quality) × 0.6 (partial) = 6.0 of 20 max
- [claimed-docs] https://codegen.com“Reads the PR diff Summarizes changes across files Runs security and secrets scan Checks test coverage Posts a review comment on failure”
- [claimed-docs] https://docs.codegen.com“Review PRs, suggest changes, comment on issues, create branches, commit code, and manage repositories.”
- [claimed-docs] https://docs.codegen.com“Connect with Slack, Linear, Figma, databases, and extend capabilities with custom MCP tools.”
Set up automations that run autonomously in the backgroundweight 2
2 (weight) × 7 (quality) × 1.0 (full) = 14.0 of 20 max
- [claimed-docs] https://docs.codegen.com/api-reference/overview“Trigger agents from CI/CD pipelines when builds fail”
- [claimed-docs] https://codegen.com“Reads the PR diff Summarizes changes across files Runs security and secrets scan Checks test coverage Posts a review comment on failure”
- [claimed-docs] https://docs.codegen.com“Send notifications, ask for clarification, report progress, and interact directly with your team in Slack channels.”
- [claimed-docs] https://docs.codegen.com“Update statuses, add comments, link PRs to issues, and create new tasks. Support for Jira, Linear, Clickup and Monday.com.”
- [claimed-docs] https://docs.codegen.com“Programmatically interact with agents using the Python SDK for advanced automation.”
- [claimed-docs] https://docs.codegen.com/introduction/sdk“agent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")”
- [claimed-docs] https://docs.codegen.com/api-reference/overview“Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic control over their execution and monitoring.”
Delegate tasks to a built-in AI assistant inside the productweight 3
3 (weight) × 8 (quality) × 1.0 (full) = 24.0 of 30 max
- [claimed-docs] https://docs.codegen.com“Analyze requirements, implement features, fix bugs, write tests, and improve documentation based on your prompts.”
- [claimed-docs] https://docs.codegen.com“Send notifications, ask for clarification, report progress, and interact directly with your team in Slack channels.”
- [claimed-docs] https://docs.codegen.com“Review PRs, suggest changes, comment on issues, create branches, commit code, and manage repositories.”
- [claimed-docs] https://docs.codegen.com/introduction/sdk“agent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")”
- [claimed-docs] https://docs.codegen.com/api-reference/overview“Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic control over their execution and monitoring.”
Operate the product with natural-language commandsweight 2
2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max
- [claimed-docs] https://docs.codegen.com“Analyze requirements, implement features, fix bugs, write tests, and improve documentation based on your prompts.”
- [claimed-docs] https://docs.codegen.com“Send notifications, ask for clarification, report progress, and interact directly with your team in Slack channels.”
- [claimed-docs] https://docs.codegen.com“Update statuses, add comments, link PRs to issues, and create new tasks. Support for Jira, Linear, Clickup and Monday.com.”
- [claimed-docs] https://docs.codegen.com“Review PRs, suggest changes, comment on issues, create branches, commit code, and manage repositories.”
- [claimed-docs] https://docs.codegen.com/introduction/sdk“agent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")”
- [claimed-docs] https://docs.codegen.com/api-reference/overview“Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic control over their execution and monitoring.”
Built-in AI = 60.0 ÷ 90 × 100 = 66.7
Automation18.8/100×0.15 of the PA blend
Depth of automation primitives — rules, scheduling, bulk operations, webhooks.
Perform bulk operations across many items at onceweight 2
2 (weight) × 5 (quality) × 0.6 (partial) = 6.0 of 20 max
- [claimed-docs] https://docs.codegen.com“Programmatically interact with agents using the Python SDK for advanced automation.”
- [claimed-docs] https://docs.codegen.com/introduction/sdk“agent = Agent(org_id="...", token="...") # Run an agent with a prompt task = agent.run(prompt="Leave a review on PR #123")”
- [claimed-docs] https://docs.codegen.com/api-reference/overview“Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic control over their execution and monitoring.”
- [claimed-docs] https://docs.codegen.com/api-reference/overview“Trigger agents from CI/CD pipelines when builds fail”
Define rules that trigger actions automatically on eventsweight 3
3 (weight) × 5 (quality) × 0.6 (partial) = 9.0 of 30 max
- [claimed-docs] https://docs.codegen.com/api-reference/overview“Trigger agents from CI/CD pipelines when builds fail”
- [claimed-docs] https://codegen.com“Reads the PR diff Summarizes changes across files Runs security and secrets scan Checks test coverage Posts a review comment on failure”
- [claimed-docs] https://docs.codegen.com“Programmatically interact with agents using the Python SDK for advanced automation.”
- [claimed-docs] https://docs.codegen.com/api-reference/overview“Create and manage AI agents that can write code, fix bugs, and handle development tasks across your repositories with full programmatic control over their execution and monitoring.”
Schedule recurring jobs or workflowsweight 2
2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
Version, review, and roll back my automationsweight 1
1 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 10 max
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
Automation = 15.0 ÷ 80 × 100 = 18.8