Skip to content
ProductArena

How Codex’s scores are calculated

The full audit trail, recomputed from the verdict data at build time through the same code that produced the leaderboard: verdict × quality × story weight per cell, cells sum to dimension scores, dimensions blend into the PA Score. Every number on the product page is reproducible from this page alone; for why the formula looks like this, see the methodology.

verdict factors: full ×1.0 · partial ×0.6 · disputed ×0.3 · none ×0.0 · n/a excluded from both sides · cell points = weight × quality × factor · cell max = weight × 10

PA Score33/100

Agent-ready 46.2 × 0.30 = 13.86

API quality 25.7 × 0.20 = 5.14

Openness 8.4 × 0.20 = 1.68

Built-in AI 68.0 × 0.15 = 10.20

Automation 12.0 × 0.15 = 1.80

(13.86 + 5.14 + 1.68 + 10.20 + 1.80) ÷ (0.30 + 0.20 + 0.20 + 0.15 + 0.15) = 32.68 ÷ 1.00 = 32.7

Scores are stored to 1 decimal; the product page’s pills round to whole numbers for display. Each dimension below shows the stories, verdicts, and cited evidence behind its number.

Agent-ready46.2/100×0.30 of the PA blend

Outside-in: can YOUR agent reach and drive this product — API, MCP, CLI, headless runs, agent docs.

Point an agent at llms.txt or agent-oriented docsweight 2

2 (weight) × 4 (quality) × 0.6 (partial) = 4.8 of 20 max

  • [probe] https://platform.openai.com/llms.txt“PROBE llms.txt: HTTP 404 at https://platform.openai.com/llms.txt”
  • [probe] https://platform.openai.com/docs/codex.md“PROBE docs-md: HTTP 404 at https://platform.openai.com/docs/codex.md”
  • [probe] https://learn.chatgpt.com/docs/codex/cli“official CLI documented at https://learn.chatgpt.com/docs/codex/cli”

Run the product headlessly / in CI for automationweight 2

2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max

  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Run a non-interactive command in a repeatable workflow.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Compose with scripts and CI: Use Codex interactively or call codex exec from repeatable workflows and pipelines.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Choose when Codex can edit files or run commands without asking, and inspect the active sandbox and writable roots before you continue.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Set the boundaries for each run — /permissions: Choose when Codex can edit files or run commands without asking, and inspect the active sandbox and writable roots before you continue.”

Plug MCP servers into this product so it can use their toolsweight 3

3 (weight) × 9 (quality) × 1.0 (full) = 27.0 of 30 max

  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Add local or remote MCP servers, authenticate when needed, and inspect the tools available to the current session before Codex uses them.”
  • [claimed-docs] https://learn.chatgpt.com/docs/extend/mcp.md“The ChatGPT desktop app, Codex CLI, and IDE extension share this configuration. Once you configure your MCP servers, you can switch among those clients without redoing setup.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Connect external tools with MCP — codex mcp: Add local or remote MCP servers, authenticate when needed, and inspect the tools available to the current session before Codex uses them.”
  • [claimed-docs] https://learn.chatgpt.com/docs/extend/mcp.md“Model Context Protocol (MCP) connects models to tools and context. Use it to give ChatGPT or Codex access to third-party documentation, or to let it interact with developer tools like your browser or Figma.”
  • [claimed-docs] https://learn.chatgpt.com/docs/extend/mcp.md“codex mcp add <server-name> --env VAR1=VALUE1 --env VAR2=VALUE2 -- <stdio server-command>”
  • [claimed-docs] https://learn.chatgpt.com/docs/extend/mcp.md“In the `codex` TUI, use `/mcp` to see your active MCP servers.”

Connect an agent via an official MCP serverweight 3

3 (weight) × 5 (quality) × 0.6 (partial) = 9.0 of 30 max

  • [github] https://github.com/openai/codex/blob/main/codex-rs/docs/codex_mcp_interface.md“Codex MCP Server Interface [experimental]: a JSON-RPC API that runs over the Model Context Protocol (MCP) transport to control a local Codex engine. Server binary: codex mcp-server (or codex-mcp-server). Run Codex as an MCP server and connect an MCP client: codex mcp-server | your_mcp_client”
  • [claimed-docs] https://learn.chatgpt.com/docs/mcp-server“codex mcp-server is deprecated. Use the Codex app server instead. ... This page documents the deprecated command for existing integrations. You can run Codex as an MCP server and connect it from other MCP clients.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Add local or remote MCP servers, authenticate when needed, and inspect the tools available to the current session before Codex uses them.”

Use an official CLIweight 2

2 (weight) × 9 (quality) × 1.0 (full) = 18.0 of 20 max

  • [github] https://github.com/openai/codex“Codex CLI is a coding agent from OpenAI that runs locally on your computer.”
  • [github] https://github.com/openai/codex“npm install -g @openai/codex”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Inspect code, make changes, run commands, and automate repeatable work without leaving your terminal.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Run a non-interactive command in a repeatable workflow.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Compose with scripts and CI: Use Codex interactively or call codex exec from repeatable workflows and pipelines.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Connect external tools with MCP — codex mcp: Add local or remote MCP servers, authenticate when needed, and inspect the tools available to the current session before Codex uses them.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Split up a larger investigation — subagents: Ask Codex to delegate focused work to specialized agents, then bring their findings back into the main terminal session.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Choose when Codex can edit files or run commands without asking, and inspect the active sandbox and writable roots before you continue.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Install the Codex CLI with the standalone installer for macOS and Linux.”
  • [probe] https://learn.chatgpt.com/docs/codex/cli“official CLI documented at https://learn.chatgpt.com/docs/codex/cli”

Drive the product through a documented public APIweight 3

3 (weight) × 5 (quality) × 0.6 (partial) = 9.0 of 30 max

  • [github] https://github.com/openai/codex“You can also use Codex with an API key, but this requires additional setup.”
  • [github] https://github.com/openai/codex/blob/main/codex-rs/docs/codex_mcp_interface.md“Codex MCP Server Interface [experimental]: a JSON-RPC API that runs over the Model Context Protocol (MCP) transport to control a local Codex engine. Server binary: codex mcp-server (or codex-mcp-server). Run Codex as an MCP server and connect an MCP client: codex mcp-server | your_mcp_client”
  • [claimed-docs] https://learn.chatgpt.com/docs/mcp-server“codex mcp-server is deprecated. Use the Codex app server instead. ... This page documents the deprecated command for existing integrations. You can run Codex as an MCP server and connect it from other MCP clients.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Run a non-interactive command in a repeatable workflow.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Compose with scripts and CI: Use Codex interactively or call codex exec from repeatable workflows and pipelines.”
  • [community] https://news.ycombinator.com/item?id=46902638“gpt-5.3-codex isn't available on the API yet — 'We are working to safely enable API access soon.'”

Issue scoped/least-privilege API credentials for an agentweight 2

2 (weight) × 5 (quality) × 0.6 (partial) = 6.0 of 20 max

  • [claimed-docs] https://developers.openai.com/api/docs/guides/rbac“Role-based access control (RBAC) lets you decide who can do what across your organization and projects—both through the API and in the Dashboard. With RBAC you can: Group users and assign permissions at scale, Create custom roles with the exact permissions you need, Scope access at the organization or project level.”
  • [github] https://github.com/openai/codex“You can also use Codex with an API key, but this requires additional setup.”
  • [claimed-docs] https://developers.openai.com/api/docs/api-reference/responses/create“Create a model response — request/response reference with runnable code samples selectable per language: HTTP, Python, TypeScript, Go, Ruby, Java, and CLI Tool (curl), each with a live example request and response body.”

Build against official SDKsweight 2

2 (weight) × 6 (quality) × 0.6 (partial) = 7.2 of 20 max

  • [claimed-docs] https://developers.openai.com/api/docs/api-reference/responses/create“Create a model response — request/response reference with runnable code samples selectable per language: HTTP, Python, TypeScript, Go, Ruby, Java, and CLI Tool (curl), each with a live example request and response body.”
  • [github] https://github.com/openai/openai-openapi“A machine-readable description of the OpenAI REST API, authored in OpenAPI 3.1.”
  • [community] https://news.ycombinator.com/item?id=46902638“gpt-5.3-codex isn't available on the API yet — 'We are working to safely enable API access soon.'”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Add local or remote MCP servers, authenticate when needed, and inspect the tools available to the current session before Codex uses them.”

Subscribe to events via webhooksweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Agent-ready = 97.0 ÷ 210 × 100 = 46.2

API quality25.7/100×0.20 of the PA blend

The programmable surface once an agent is there — machine-readable spec, interactive docs, sandbox, versioning discipline.

Explore an interactive API reference with runnable examplesweight 2

2 (weight) × 5 (quality) × 0.6 (partial) = 6.0 of 20 max

  • [claimed-docs] https://developers.openai.com/api/docs/api-reference/responses/create“Create a model response — request/response reference with runnable code samples selectable per language: HTTP, Python, TypeScript, Go, Ruby, Java, and CLI Tool (curl), each with a live example request and response body.”
  • [github] https://github.com/openai/codex“You can also use Codex with an API key, but this requires additional setup.”

Download a machine-readable API spec (OpenAPI or equivalent)weight 2

2 (weight) × 4 (quality) × 0.6 (partial) = 4.8 of 20 max

  • [github] https://github.com/openai/openai-openapi“A machine-readable description of the OpenAI REST API, authored in OpenAPI 3.1.”
  • [github] https://github.com/openai/codex“You can also use Codex with an API key, but this requires additional setup.”
  • [claimed-docs] https://developers.openai.com/api/docs/api-reference/responses/create“Create a model response — request/response reference with runnable code samples selectable per language: HTTP, Python, TypeScript, Go, Ruby, Java, and CLI Tool (curl), each with a live example request and response body.”

Test against a sandbox environment without touching production dataweight 1

1 (weight) × 6 (quality) × 0.6 (partial) = 3.6 of 10 max

  • [claimed-docs] https://platform.openai.com/docs/codex“Run tasks in isolated cloud environments, work in parallel, and start work from the web, GitHub, GitLab, Linear, or Slack.”
  • [claimed-docs] https://platform.openai.com/docs/codex“Configure the dependencies, tools, variables, and setup steps each repository needs.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Choose when Codex can edit files or run commands without asking, and inspect the active sandbox and writable roots before you continue.”
  • [community] https://news.ycombinator.com/item?id=47796469“Does that version of Codex still read sensitive data on your file system without even asking? Just curious. [links to github.com/openai/codex/issues/2847]”

Rely on versioned APIs with a documented deprecation policyweight 2

2 (weight) × 3 (quality) × 0.6 (partial) = 3.6 of 20 max

  • [github] https://github.com/openai/openai-openapi“A machine-readable description of the OpenAI REST API, authored in OpenAPI 3.1.”
  • [claimed-docs] https://developers.openai.com/api/docs/api-reference/responses/create“Create a model response — request/response reference with runnable code samples selectable per language: HTTP, Python, TypeScript, Go, Ruby, Java, and CLI Tool (curl), each with a live example request and response body.”
  • [claimed-docs] https://learn.chatgpt.com/docs/mcp-server“codex mcp-server is deprecated. Use the Codex app server instead. ... This page documents the deprecated command for existing integrations. You can run Codex as an MCP server and connect it from other MCP clients.”

API quality = 18.0 ÷ 70 × 100 = 25.7

Openness8.4/100×0.20 of the PA blend

Can you leave, inspect, or self-host — data export, open source, portability.

Do everything through the API that I can do in the UIweight 2

2 (weight) × 3 (quality) × 0.6 (partial) = 3.6 of 20 max

  • [github] https://github.com/openai/codex“You can also use Codex with an API key, but this requires additional setup.”
  • [community] https://news.ycombinator.com/item?id=46902638“gpt-5.3-codex isn't available on the API yet — 'We are working to safely enable API access soon.'”
  • [claimed-docs] https://developers.openai.com/api/docs/guides/rbac“Role-based access control (RBAC) lets you decide who can do what across your organization and projects—both through the API and in the Dashboard. With RBAC you can: Group users and assign permissions at scale, Create custom roles with the exact permissions you need, Scope access at the organization or project level.”
  • [claimed-docs] https://developers.openai.com/api/docs/api-reference/responses/create“Create a model response — request/response reference with runnable code samples selectable per language: HTTP, Python, TypeScript, Go, Ruby, Java, and CLI Tool (curl), each with a live example request and response body.”
  • [claimed-docs] https://learn.chatgpt.com/docs/mcp-server“codex mcp-server is deprecated. Use the Codex app server instead. ... This page documents the deprecated command for existing integrations. You can run Codex as an MCP server and connect it from other MCP clients.”

Export all of my data in open formats and leaveweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Read the product's source under an open licenseweight 2

2 (weight) × 4 (quality) × 0.6 (partial) = 4.8 of 20 max

  • [github] https://github.com/openai/codex“Codex CLI is a coding agent from OpenAI that runs locally on your computer.”
  • [community] https://news.ycombinator.com/item?id=47796469“I wish Codex App was open source. I like it, but there are always a bunch of little paper cuts that, if you were using codex cli, you could have easily diagnosed and filed an issue.”

Self-host the core productweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

  • [github] https://github.com/openai/codex“We recommend signing into your ChatGPT account to use Codex as part of your Plus, Pro, Business, Edu, or Enterprise plan.”
  • [github] https://github.com/openai/codex“You can also use Codex with an API key, but this requires additional setup.”
  • [community] https://news.ycombinator.com/item?id=47796469“I wish Codex App was open source. I like it, but there are always a bunch of little paper cuts that, if you were using codex cli, you could have easily diagnosed and filed an issue.”

Openness = 8.4 ÷ 100 × 100 = 8.4

Built-in AI68.0/100×0.15 of the PA blend

Inside-out: how agentic the product itself is for its users — built-in assistants, autonomous features.

Get AI-generated insights and suggestions from my data inside the productweight 2

2 (weight) × 6 (quality) × 0.6 (partial) = 7.2 of 20 max

  • [claimed-docs] https://platform.openai.com/docs/codex“Inspect the summary and diff, request a follow-up, or open a pull request when the result is ready.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized findings without modifying your working tree”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Review changes before they ship: Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized findings without modifying your working tree.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized findings without modifying your working tree, so you can address risks before you commit or open a pull request.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Split up a larger investigation — subagents: Ask Codex to delegate focused work to specialized agents, then bring their findings back into the main terminal session.”

Set up automations that run autonomously in the backgroundweight 2

2 (weight) × 7 (quality) × 1.0 (full) = 14.0 of 20 max

  • [claimed-docs] https://platform.openai.com/docs/codex“Run tasks in isolated cloud environments, work in parallel, and start work from the web, GitHub, GitLab, Linear, or Slack.”
  • [claimed-docs] https://platform.openai.com/docs/codex“Start work in Codex cloud from GitHub pull requests, GitLab merge requests and issues, Linear issues, or Slack channels and threads.”
  • [claimed-docs] https://platform.openai.com/docs/codex“Run tasks in parallel without tying up your local machine.”
  • [claimed-docs] https://platform.openai.com/docs/codex“Delegate a longer task and return when it is ready.”
  • [claimed-docs] https://platform.openai.com/docs/codex“Inspect the summary and diff, request a follow-up, or open a pull request when the result is ready.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Run a non-interactive command in a repeatable workflow.”

Delegate tasks to a built-in AI assistant inside the productweight 3

3 (weight) × 8 (quality) × 1.0 (full) = 24.0 of 30 max

  • [claimed-docs] https://platform.openai.com/docs/codex“Delegate a longer task and return when it is ready.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Ask Codex to delegate focused work to specialized agents, then bring their findings back into the main terminal session.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Split up a larger investigation — subagents: Ask Codex to delegate focused work to specialized agents, then bring their findings back into the main terminal session.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Move work to Codex cloud — codex cloud: Browse active and completed chats, submit work to a configured environment, and apply the result to your local repository from the terminal.”
  • [github] https://github.com/openai/codex“Codex CLI is a coding agent from OpenAI that runs locally on your computer.”

Operate the product with natural-language commandsweight 2

2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max

  • [claimed-docs] https://platform.openai.com/docs/codex“Delegate a longer task and return when it is ready.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Start Codex in a repository to explore unfamiliar code, plan a change, edit files, and run your local development tools.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Ask Codex to delegate focused work to specialized agents, then bring their findings back into the main terminal session.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Run a non-interactive command in a repeatable workflow.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“`codex resume`: Reopen a recent chat from the current repository, or search across local chats when you need to return to older work.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Bring visual context into the prompt — codex --image: Pass an error screenshot, architecture diagram, or design reference with the first prompt, or paste an image into the interactive composer.”
  • [github] https://github.com/openai/codex“Codex CLI is a coding agent from OpenAI that runs locally on your computer.”
  • [community] https://news.ycombinator.com/item?id=46859054“Genuinely excited to try this out. I've started using Codex much more heavily in the past two months and honestly, it's been shockingly good... overall it's no worse than having average L3-L4 engs at your disposal. That being said, the app is stuck at the launch screen, with 'Loading projects...' taking forever.”

Built-in AI = 61.2 ÷ 90 × 100 = 68.0

Automation12.0/100×0.15 of the PA blend

Depth of automation primitives — rules, scheduling, bulk operations, webhooks.

Perform bulk operations across many items at onceweight 2

2 (weight) × 6 (quality) × 0.6 (partial) = 7.2 of 20 max

  • [claimed-docs] https://platform.openai.com/docs/codex“Run tasks in isolated cloud environments, work in parallel, and start work from the web, GitHub, GitLab, Linear, or Slack.”
  • [claimed-docs] https://platform.openai.com/docs/codex“Run tasks in parallel without tying up your local machine.”
  • [claimed-docs] https://platform.openai.com/docs/codex“Delegate a longer task and return when it is ready.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Ask Codex to delegate focused work to specialized agents, then bring their findings back into the main terminal session.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Run a non-interactive command in a repeatable workflow.”

Define rules that trigger actions automatically on eventsweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

  • [claimed-docs] https://platform.openai.com/docs/codex“Run tasks in isolated cloud environments, work in parallel, and start work from the web, GitHub, GitLab, Linear, or Slack.”
  • [claimed-docs] https://platform.openai.com/docs/codex“Start work in Codex cloud from GitHub pull requests, GitLab merge requests and issues, Linear issues, or Slack channels and threads.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Run a non-interactive command in a repeatable workflow.”

Schedule recurring jobs or workflowsweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Compose with scripts and CI: Use Codex interactively or call codex exec from repeatable workflows and pipelines.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Run a non-interactive command in a repeatable workflow.”
  • [claimed-docs] https://platform.openai.com/docs/codex“Start work in Codex cloud from GitHub pull requests, GitLab merge requests and issues, Linear issues, or Slack channels and threads.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Move work to Codex cloud — codex cloud: Browse active and completed chats, submit work to a configured environment, and apply the result to your local repository from the terminal.”

Version, review, and roll back my automationsweight 1

1 (weight) × 4 (quality) × 0.6 (partial) = 2.4 of 10 max

  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized findings without modifying your working tree”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Review changes before they ship: Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized findings without modifying your working tree.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Run a dedicated review against uncommitted changes, a commit, or a base branch. Codex reports prioritized findings without modifying your working tree, so you can address risks before you commit or open a pull request.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Package repeatable instructions as skills, then add plugins to connect Codex to your team's tools and data without leaving the CLI.”
  • [claimed-docs] https://learn.chatgpt.com/docs/codex/cli.md“Use skills and plugins: Package repeatable instructions as skills, then add plugins to connect Codex to your team's tools and data without leaving the CLI.”

Automation = 9.6 ÷ 80 × 100 = 12.0