Skip to content
ProductArena

How Codex Plugins’s scores are calculated

The full audit trail, recomputed from the verdict data at build time through the same code that produced the leaderboard: verdict × quality × story weight per cell, cells sum to dimension scores, dimensions blend into the PA Score. Every number on the product page is reproducible from this page alone; for why the formula looks like this, see the methodology.

verdict factors: full ×1.0 · partial ×0.6 · disputed ×0.3 · none ×0.0 · n/a excluded from both sides · cell points = weight × quality × factor · cell max = weight × 10

PA Score19/100

Agent-ready 32.5 × 0.30 = 9.75

API quality 0.0 × 0.20 = 0.00

Openness 3.6 × 0.20 = 0.72

Built-in AI 46.9 × 0.15 = 7.03

Automation 6.7 × 0.15 = 1.00

(9.75 + 0.00 + 0.72 + 7.03 + 1.00) ÷ (0.30 + 0.20 + 0.20 + 0.15 + 0.15) = 18.51 ÷ 1.00 = 18.5

Scores are stored to 1 decimal; the product page’s pills round to whole numbers for display. Each dimension below shows the stories, verdicts, and cited evidence behind its number.

Agent-ready32.5/100×0.30 of the PA blend

Outside-in: can YOUR agent reach and drive this product — API, MCP, CLI, headless runs, agent docs.

Point an agent at llms.txt or agent-oriented docsweight 2

2 (weight) × 5 (quality) × 0.6 (partial) = 6.0 of 20 max

  • [probe] https://developers.openai.com/llms.txt“PROBE llms.txt: HTTP 200 at https://developers.openai.com/llms.txt # OpenAI Developers > Complete documentation hub for OpenAI API, Ads, Plugins, Workspace Agents, Codex, Agentic Commerc”
  • [probe] https://developers.openai.com/codex/skills.md“PROBE docs-md: HTTP 200 at https://developers.openai.com/codex/skills.md # Build skills > For the complete documentation index, see [llms.txt](https://learn.chatgpt.com/llms.txt). Markdown ver”
  • [claimed-docs] https://learn.chatgpt.com/docs/build-skills.md“Codex scans `.agents/skills` in every directory from your current working directory up to the repository root.”

Run the product headlessly / in CI for automationweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [claimed-docs] https://learn.chatgpt.com/docs/plugins.md“In Codex CLI, enter `/plugins` to open the plugin browser.”
  • [claimed-docs] https://learn.chatgpt.com/docs/build-skills.md“In Codex CLI or the IDE extension, run `/skills` or type `$` to mention a skill.”
  • [claimed-docs] https://learn.chatgpt.com/docs/plugins.md“In Codex CLI, enter `/plugins` to open the plugin browser. Install a plugin from a configured marketplace, then start a new session before using its bundled skills or tools.”

Plug MCP servers into this product so it can use their toolsweight 3

3 (weight) × 8 (quality) × 1.0 (full) = 24.0 of 30 max

  • [claimed-docs] https://developers.openai.com/plugins/build/app-quickstart.md“Plugins use the [Model Context Protocol (MCP)] to expose server-backed capabilities to ChatGPT and Codex.”
  • [claimed-docs] https://developers.openai.com/plugins/build/plugins.md“The plugin-creator skill will create the plugin folder, create the required `.codex-plugin/plugin.json`, and add MCP server wiring for the plugin.”
  • [claimed-docs] https://learn.chatgpt.com/docs/plugins.md“In Codex CLI, enter `/plugins` to open the plugin browser. Install a plugin from a configured marketplace, then start a new session before using its bundled skills or tools.”
  • [claimed-docs] https://learn.chatgpt.com/docs/plugins.md“Open the Plugins tab to browse and install plugins. After installation, you can use plugins in Chat or Work in ChatGPT, or in Codex. Installed plugins can add skills and MCP tools to new chats.”
  • [claimed-docs] https://developers.openai.com/plugins/build/plugins.md“Use `codex plugin marketplace add` to add and track a marketplace source instead of editing `config.toml` by hand.”
  • [claimed-docs] https://developers.openai.com/plugins/build/plugins.md“For the fastest setup, use the built-in `@plugin-creator` skill. It scaffolds the required `.codex-plugin/plugin.json` manifest and can also generate a local marketplace entry for testing.”

Connect an agent via an official MCP serverweight 3

n/a — not applicable to this product: excluded from numerator and denominator

  • [claimed-docs] https://developers.openai.com/plugins/build/app-quickstart.md“Plugins use the [Model Context Protocol (MCP)] to expose server-backed capabilities to ChatGPT and Codex.”
  • [claimed-docs] https://developers.openai.com/plugins/build/plugins.md“The plugin-creator skill will create the plugin folder, create the required `.codex-plugin/plugin.json`, and add MCP server wiring for the plugin.”
  • [claimed-docs] https://learn.chatgpt.com/docs/plugins.md“Open the Plugins tab to browse and install plugins. After installation, you can use plugins in Chat or Work in ChatGPT, or in Codex. Installed plugins can add skills and MCP tools to new chats.”

Use an official CLIweight 2

2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max

  • [claimed-docs] https://learn.chatgpt.com/docs/plugins.md“In Codex CLI, enter `/plugins` to open the plugin browser.”
  • [claimed-docs] https://developers.openai.com/plugins/build/plugins.md“Use `codex plugin marketplace add` to add and track a marketplace source instead of editing `config.toml` by hand.”
  • [claimed-docs] https://learn.chatgpt.com/docs/build-skills.md“In Codex CLI or the IDE extension, run `/skills` or type `$` to mention a skill.”
  • [claimed-docs] https://learn.chatgpt.com/docs/plugins.md“In Codex CLI, enter `/plugins` to open the plugin browser. Install a plugin from a configured marketplace, then start a new session before using its bundled skills or tools.”
  • [claimed-docs] https://learn.chatgpt.com/docs/build-skills.md“To add curated skills beyond the built-ins for your own local Codex setup, use `$skill-installer`.”
  • [probe] https://developers.openai.com/codex/skills.md“PROBE docs-md: HTTP 200 at https://developers.openai.com/codex/skills.md # Build skills > For the complete documentation index, see [llms.txt](https://learn.chatgpt.com/llms.txt). Markdown ver”

Drive the product through a documented public APIweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

  • [probe] https://developers.openai.com/openapi.json“PROBE openapi: all candidate paths 404 (https://developers.openai.com/openapi.json, https://developers.openai.com/swagger.json, https://developers.openai.com/api/openapi.json, https://developers.openai.com/.well-known/openapi.json)”
  • [claimed-docs] https://developers.openai.com/plugins/build/app-quickstart.md“Plugins use the [Model Context Protocol (MCP)] to expose server-backed capabilities to ChatGPT and Codex.”
  • [claimed-docs] https://developers.openai.com/plugins/build/plugins.md“Use `codex plugin marketplace add` to add and track a marketplace source instead of editing `config.toml` by hand.”

Issue scoped/least-privilege API credentials for an agentweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [claimed-docs] https://learn.chatgpt.com/docs/plugins.md“Signing in shares only your name, email address, and profile picture, when available, with the partner. It doesn't grant the plugin access to your data or approve actions automatically.”
  • [claimed-docs] https://learn.chatgpt.com/docs/plugins.md“Signing in shares only your name, email address, and profile picture, when available, with the partner.”

Build against official SDKsweight 2

2 (weight) × 5 (quality) × 0.6 (partial) = 6.0 of 20 max

  • [claimed-docs] https://developers.openai.com/plugins/build/plugins.md“For the fastest setup, use the built-in `@plugin-creator` skill. It scaffolds the required `.codex-plugin/plugin.json` manifest and can also generate a local marketplace entry for testing.”
  • [claimed-docs] https://developers.openai.com/plugins/build/plugins.md“The plugin-creator skill will create the plugin folder, create the required `.codex-plugin/plugin.json`, and add MCP server wiring for the plugin.”
  • [claimed-docs] https://developers.openai.com/plugins/build/plugins.md“For a portable Agent Plugins package, add `plugin.json` at the plugin root and declare the Agent Plugins schema.”
  • [claimed-docs] https://developers.openai.com/plugins/build/app-quickstart.md“Plugins use the [Model Context Protocol (MCP)] to expose server-backed capabilities to ChatGPT and Codex.”
  • [claimed-docs] https://developers.openai.com/plugins/deploy/submission.md“Use the plugin submission portal to submit a plugin for review when you're ready to publish it for public use.”
  • [claimed-docs] https://developers.openai.com/plugins/build/plugins.md“Use `codex plugin marketplace add` to add and track a marketplace source instead of editing `config.toml` by hand.”
  • [probe] https://developers.openai.com/openapi.json“PROBE openapi: all candidate paths 404 (https://developers.openai.com/openapi.json, https://developers.openai.com/swagger.json, https://developers.openai.com/api/openapi.json, https://developers.openai.com/.well-known/openapi.json)”

Subscribe to events via webhooksweight 2

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Agent-ready = 52.0 ÷ 160 × 100 = 32.5

API quality0.0/100×0.20 of the PA blend

The programmable surface once an agent is there — machine-readable spec, interactive docs, sandbox, versioning discipline.

Explore an interactive API reference with runnable examplesweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [probe] https://developers.openai.com/openapi.json“PROBE openapi: all candidate paths 404 (https://developers.openai.com/openapi.json, https://developers.openai.com/swagger.json, https://developers.openai.com/api/openapi.json, https://developers.openai.com/.well-known/openapi.json)”

Download a machine-readable API spec (OpenAPI or equivalent)weight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [probe] https://developers.openai.com/openapi.json“PROBE openapi: all candidate paths 404 (https://developers.openai.com/openapi.json, https://developers.openai.com/swagger.json, https://developers.openai.com/api/openapi.json, https://developers.openai.com/.well-known/openapi.json)”

Test against a sandbox environment without touching production dataweight 1

1 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 10 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Rely on versioned APIs with a documented deprecation policyweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [probe] https://developers.openai.com/openapi.json“PROBE openapi: all candidate paths 404 (https://developers.openai.com/openapi.json, https://developers.openai.com/swagger.json, https://developers.openai.com/api/openapi.json, https://developers.openai.com/.well-known/openapi.json)”

API quality = 0.0 ÷ 70 × 100 = 0.0

Openness3.6/100×0.20 of the PA blend

Can you leave, inspect, or self-host — data export, open source, portability.

Do everything through the API that I can do in the UIweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [claimed-docs] https://learn.chatgpt.com/docs/plugins.md“In Codex CLI, enter `/plugins` to open the plugin browser.”
  • [claimed-docs] https://developers.openai.com/plugins/build/plugins.md“Use `codex plugin marketplace add` to add and track a marketplace source instead of editing `config.toml` by hand.”
  • [claimed-docs] https://learn.chatgpt.com/docs/plugins.md“Open the Plugins tab to browse and install plugins. After installation, you can use plugins in Chat or Work in ChatGPT, or in Codex. Installed plugins can add skills and MCP tools to new chats.”
  • [claimed-docs] https://learn.chatgpt.com/docs/build-skills.md“In the ChatGPT desktop app, open **Skills** in the sidebar to view and explore skills created across your projects.”
  • [probe] https://developers.openai.com/openapi.json“PROBE openapi: all candidate paths 404 (https://developers.openai.com/openapi.json, https://developers.openai.com/swagger.json, https://developers.openai.com/api/openapi.json, https://developers.openai.com/.well-known/openapi.json)”

Export all of my data in open formats and leaveweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Read the product's source under an open licenseweight 2

2 (weight) × 3 (quality) × 0.6 (partial) = 3.6 of 20 max

  • [github] https://github.com/openai/plugins“plugins/figma for `use_figma`, Code to Canvas, Code Connect, and design system rules”
  • [github] https://github.com/openai/plugins“`plugins/notion` for planning, research, meetings, and knowledge capture”
  • [claimed-docs] https://raw.githubusercontent.com/openai/plugins/HEAD/.agents/plugins/marketplace.json“"name": "slack", ... "category": "Communication"”

Self-host the core productweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Openness = 3.6 ÷ 100 × 100 = 3.6

Built-in AI46.9/100×0.15 of the PA blend

Inside-out: how agentic the product itself is for its users — built-in assistants, autonomous features.

Get AI-generated insights and suggestions from my data inside the productweight 2

2 (weight) × 6 (quality) × 0.6 (partial) = 7.2 of 20 max

  • [claimed-docs] https://learn.chatgpt.com/docs/plugins.md“Install the Gmail plugin to work with Gmail.”
  • [claimed-docs] https://learn.chatgpt.com/docs/plugins.md“Install the Google Drive plugin to work across Drive, Docs, Sheets, and Slides.”
  • [claimed-docs] https://learn.chatgpt.com/docs/plugins.md“Install the Slack plugin to summarize channels or draft replies.”
  • [github] https://github.com/openai/plugins“plugins/notion for planning, research, meetings, and knowledge capture”
  • [claimed-docs] https://learn.chatgpt.com/docs/plugins.md“Install the Codex Security plugin to scan authorized code and confirm plausible vulnerability findings.”
  • [claimed-docs] https://developers.openai.com/plugins/build/app-quickstart.md“Plugins use the [Model Context Protocol (MCP)] to expose server-backed capabilities to ChatGPT and Codex.”

Set up automations that run autonomously in the backgroundweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [claimed-docs] https://learn.chatgpt.com/docs/plugins.md“In Codex CLI, enter `/plugins` to open the plugin browser. Install a plugin from a configured marketplace, then start a new session before using its bundled skills or tools.”
  • [claimed-docs] https://learn.chatgpt.com/docs/build-skills.md“The creator asks what the skill does, when it should trigger, and whether it should stay instruction-only or include scripts.”
  • [claimed-docs] https://learn.chatgpt.com/docs/build-skills.md“In Codex CLI or the IDE extension, run `/skills` or type `$` to mention a skill.”

Delegate tasks to a built-in AI assistant inside the productweight 3

3 (weight) × 7 (quality) × 1.0 (full) = 21.0 of 30 max

  • [claimed-docs] https://learn.chatgpt.com/docs/plugins.md“Install the Gmail plugin to work with Gmail.”
  • [claimed-docs] https://learn.chatgpt.com/docs/plugins.md“Install the Google Drive plugin to work across Drive, Docs, Sheets, and Slides.”
  • [claimed-docs] https://learn.chatgpt.com/docs/plugins.md“Install the Slack plugin to summarize channels or draft replies.”
  • [claimed-docs] https://learn.chatgpt.com/docs/plugins.md“it can read and search iMessage, SMS, and RCS chats on your Mac and send messages on your behalf through the Messages app”
  • [claimed-docs] https://learn.chatgpt.com/docs/plugins.md“Open the Plugins tab to browse and install plugins. After installation, you can use plugins in Chat or Work in ChatGPT, or in Codex. Installed plugins can add skills and MCP tools to new chats.”
  • [claimed-docs] https://learn.chatgpt.com/docs/plugins.md“In Codex CLI, enter `/plugins` to open the plugin browser. Install a plugin from a configured marketplace, then start a new session before using its bundled skills or tools.”

Operate the product with natural-language commandsweight 2

2 (weight) × 7 (quality) × 1.0 (full) = 14.0 of 20 max

  • [claimed-docs] https://learn.chatgpt.com/docs/plugins.md“Install the Slack plugin to summarize channels or draft replies.”
  • [claimed-docs] https://learn.chatgpt.com/docs/build-skills.md“The creator asks what the skill does, when it should trigger, and whether it should stay instruction-only or include scripts.”
  • [claimed-docs] https://learn.chatgpt.com/docs/build-skills.md“In Codex CLI or the IDE extension, run `/skills` or type `$` to mention a skill.”
  • [claimed-docs] https://learn.chatgpt.com/docs/plugins.md“Open the Plugins tab to browse and install plugins. After installation, you can use plugins in Chat or Work in ChatGPT, or in Codex. Installed plugins can add skills and MCP tools to new chats.”
  • [community] https://hn.algolia.com/api/v1/items/46334424“Skills are the matrix scene where neo learns kungfu. Imagine they are a database of specialized knowledge that can an agent can instantly tap into on demand... Some practical skills: writing tests, fetching sentry info, using playwright (a lot of local mcps are just flat out replaced by skills), submitting a PR according to team conventions.”

Built-in AI = 42.2 ÷ 90 × 100 = 46.9

Automation6.7/100×0.15 of the PA blend

Depth of automation primitives — rules, scheduling, bulk operations, webhooks.

Perform bulk operations across many items at onceweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Define rules that trigger actions automatically on eventsweight 3

3 (weight) × 3 (quality) × 0.6 (partial) = 5.4 of 30 max

  • [claimed-docs] https://learn.chatgpt.com/docs/build-skills.md“The creator asks what the skill does, when it should trigger, and whether it should stay instruction-only or include scripts.”
  • [claimed-docs] https://learn.chatgpt.com/docs/build-skills.md“In Codex CLI or the IDE extension, run `/skills` or type `$` to mention a skill.”
  • [claimed-docs] https://learn.chatgpt.com/docs/plugins.md“In Codex CLI, enter `/plugins` to open the plugin browser.”

Schedule recurring jobs or workflowsweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Version, review, and roll back my automationsweight 1

1 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 10 max

  • [claimed-docs] https://learn.chatgpt.com/docs/build-skills.md“A skill is a directory with a `SKILL.md` file plus optional scripts and references. The `SKILL.md` file must include `name` and `description`.”
  • [claimed-docs] https://learn.chatgpt.com/docs/build-skills.md“Codex scans `.agents/skills` in every directory from your current working directory up to the repository root.”
  • [claimed-docs] https://developers.openai.com/plugins/build/plugins.md“Use `codex plugin marketplace add` to add and track a marketplace source instead of editing `config.toml` by hand.”
  • [claimed-docs] https://developers.openai.com/plugins/build/plugins.md“It scaffolds the required `.codex-plugin/plugin.json` manifest and can also generate a local marketplace entry for testing.”

Automation = 5.4 ÷ 80 × 100 = 6.7