Agent Skills & Extensions Arena
Agent Skills & Extensions — procurement report
ProductArena · rankings as of 2026-09-16 · evidence as of 2026-09-16 · 5 products · 52 judged requirements · 260 judged cells
Methodology: Every product is judged against a shared taxonomy of user stories using cited evidence — hands-on probes > repository code > independent community sources > vendor claims — never opinion. Full writeup: https://ultrametric.ai/productarena/methodology
Leaderboard
| # | Product | PA Score | Coverage score | Applicable cells | Confidence |
|---|---|---|---|---|---|
| 1 | Anthropic Skills | 24.1 | 31.7 | 44/52 | B |
| 2 | Superpowers | 22.0 | 35.1 | 40/52 | B |
| 3 | Skills for Real Engineers | 19.1 | 26.6 | 43/52 | C |
| 4 | Codex Plugins | 18.5 | 27.4 | 49/52 | C |
| 5 | skills.sh | 18.0 | 37.8 | 45/52 | B |
PA Score = agent-readiness blend (see methodology). Coverage score = weighted share of judged requirements met. Confidence = how much of the score rests on tested vs claimed evidence (A–D).
Uncertainty note
The current #1/#2 gap in this arena is not close enough to qualify for the multi-judge uncertainty pass (or the pass has not covered it yet) — no extra caveat applies beyond the per-product confidence grades above.
Buyer checklist (RFP)
The arena's 52 judged user stories as requirements, grouped by theme. Priorities mirror the story weights our scoring uses (3 = must-have, 2 = should-have, 1 = nice-to-have). Interactive version with per-requirement verdicts for the top products: /arena/agent-skills/checklist
Agent workflows — stories about agent workflows in this arenaAgent workflows
Stories about agent workflows in this arena
- ai-native userMy coding agent can install a skill by itself — a non-interactive, promptless install path an agent can run headlessly end to endmust-have
- ai-native userMy agent can author and package a new skill end to end by following the project's own spec, template, or meta-skillshould-have
Agenticness — how well agents can access and operate the productAgenticness
How well agents can access and operate the product
- ai-native userPlug MCP servers into this product so it can use their toolsmust-have
- ai-native userConnect an agent via an official MCP servermust-have
- ai-native userDrive the product through a documented public APImust-have
- ai-native userDelegate tasks to a built-in AI assistant inside the productmust-have
- ai-native userPoint an agent at llms.txt or agent-oriented docsshould-have
- ai-native userRun the product headlessly / in CI for automationshould-have
- ai-native userUse an official CLIshould-have
- ai-native userIssue scoped/least-privilege API credentials for an agentshould-have
- ai-native userBuild against official SDKsshould-have
- ai-native userSubscribe to events via webhooksshould-have
- ai-native userGet AI-generated insights and suggestions from my data inside the productshould-have
- ai-native userSet up automations that run autonomously in the backgroundshould-have
- ai-native userOperate the product with natural-language commandsshould-have
- ai-native userExplore an interactive API reference with runnable examplesshould-have
- ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)should-have
- ai-native userRely on versioned APIs with a documented deprecation policyshould-have
- ai-native userTest against a sandbox environment without touching production datanice-to-have
Automation depth — how much of the product can run unattendedAutomation depth
How much of the product can run unattended
- ai-native userDefine rules that trigger actions automatically on eventsmust-have
- ai-native userPerform bulk operations across many items at onceshould-have
- ai-native userSchedule recurring jobs or workflowsshould-have
- ai-native userVersion, review, and roll back my automationsnice-to-have
Cross agent portability — stories about cross agent portability in this arenaCross agent portability
Stories about cross agent portability in this arena
- developerInstall the same collection into multiple different coding agents — Claude Code, Codex, Cursor, and others — with per-harness instructionsmust-have
- developerSkills are plain markdown files and folders I can read, copy, and carry to another harness — not a proprietary binary formatshould-have
Discovery distribution — stories about discovery distribution in this arenaDiscovery distribution
Stories about discovery distribution in this arena
- developerInstalled skills trigger automatically from task context, with descriptions engineered so the agent activates the right skill at the right momentmust-have
- developerBrowse or search a catalog of available skills — a registry, leaderboard, or marketplace listing — before installing anythingshould-have
- engineering-leadDistribute a standard skill set to my whole team — via a marketplace, a shared repo, or files committed to the projectshould-have
Docs onboarding — stories about docs onboarding in this arenaDocs onboarding
Stories about docs onboarding in this arena
- developerA quickstart takes me from nothing to a working installed skill in under five minutesmust-have
- developerEvery skill documents what it does and when it activates, so I can predict my agent's new behavior before it surprises meshould-have
Install experience — stories about install experience in this arenaInstall experience
Stories about install experience in this arena
- developerInstall a skill collection with one documented command — a package-manager one-liner, CLI, or in-agent marketplace command — and it is active in my next sessionmust-have
- developerChoose install scope — project-local files committed with my repo, or user-global across all projectsshould-have
- developerInstall only the specific skills I want from a collection instead of taking the whole bundleshould-have
- developerList what is installed and remove skills cleanly, without orphaned files or lingering instructionsnice-to-have
Openness — open source, data portability, and self-hosting storiesOpenness
Open source, data portability, and self-hosting stories
- ai-native userExport all of my data in open formats and leavemust-have
- ai-native userSelf-host the core productmust-have
- ai-native userDo everything through the API that I can do in the UIshould-have
- ai-native userRead the product's source under an open licenseshould-have
Privacy posture — data-handling and privacy storiesPrivacy posture
Data-handling and privacy stories
- ai-native userPrevent my data from being used to train AI modelsmust-have
- ai-native userChoose where my data is stored (region/residency)should-have
- ai-native userControl data retention and deletionshould-have
- ai-native userOpt out of telemetry and usage trackingshould-have
Safety review — stories about safety review in this arenaSafety review
Stories about safety review in this arena
- engineering-leadReview exactly what instructions and scripts a skill will add — list contents before installing and read every file afterwardmust-have
- engineering-leadThe project documents its security posture — what skills can execute, the trust model for third-party skills, and any telemetry or data collectionshould-have
Skill authoring — stories about skill authoring in this arenaSkill authoring
Stories about skill authoring in this arena
- developerAuthor a new skill from a documented template — a SKILL.md with name and description frontmatter — without reverse-engineering existing skillsmust-have
- developerThe collection ships a meta-skill or tool that guides my agent through writing, improving, and packaging new skillsshould-have
- developerSkills follow the open Agent Skills specification so the same skill folder is valid beyond this one vendor's toolingshould-have
Testing quality — stories about testing quality in this arenaTesting quality
Stories about testing quality in this arena
- developerThe collection is actively maintained — recent releases, triaged issues, and accepted community contributionsshould-have
- developerThe collection maintains tests or evals for its skills so changes are verified against regressions rather than shipped on vibesshould-have
Versioning updates — stories about versioning updates in this arenaVersioning updates
Stories about versioning updates in this arena
- developerThere is a documented update path — marketplace auto-updates or an explicit update command — so I get fixes without reinstalling from scratchmust-have
- engineering-leadControl when skill changes reach my team — pinned versions or a lockfile rather than silent behind-the-back updatesnice-to-have
- developerReleases ship with notes or a changelog so I can see what changed in the skills before I take an updatenice-to-have
Appendix: recorded probes
Hands-on probe recordings — transcripts/videos a human can replay, the strongest evidence tier. Watch them at https://ultrametric.ai/productarena/proofs
- Anthropic Skills
npx -y skills add anthropics/skills --skill skill-creator -a claude-code -y --copy # in a scratch dir, then print installed SKILL.md frontmatterterminal · recorded 2026-09-06 · exit 0 - Codex Plugins
curl -s https://raw.githubusercontent.com/openai/plugins/HEAD/.agents/plugins/marketplace.json | head -40terminal · recorded 2026-09-06 · exit 0 - Codex Plugins
npx -y skills add openai/plugins --skill plugin-creator -a codex -y --copy # in a scratch dir, then print installed SKILL.md frontmatterterminal · recorded 2026-09-06 · exit 0 - Skills for Real Engineers
npx -y skills add mattpocock/skills --skill tdd -a claude-code -y --copy # in a scratch dir, then print installed SKILL.md frontmatterterminal · recorded 2026-09-06 · exit 0 - skills.sh
npx -y skills --versionterminal · recorded 2026-09-06 · exit 0 - skills.sh
npx -y skills add vercel-labs/agent-skills --listterminal · recorded 2026-09-06 · exit 0 - Superpowers
npx -y skills add obra/superpowers --skill test-driven-development -a claude-code -y --copy # in a scratch dir, then print installed SKILL.md frontmatterterminal · recorded 2026-09-06 · exit 0
Cite as: ProductArena by Ultrametric Inc, Agent Skills & Extensions arena, rankings as of 2026-09-16 — https://ultrametric.ai/productarena/arena/agent-skills
License: © 2026 Ultrametric Inc. Brief quotation of individual verdicts, scores, or evidence excerpts is permitted with attribution to "ProductArena by Ultrametric Inc (ultrametric.ai/productarena)", as is use of the data to evaluate, contest, or contribute corrections. Bulk copying, redistribution, or use to build competing datasets requires prior written permission (see DATA-LICENSE in the repository).
No liability: rankings, verdicts, and scores are research outputs derived from the cited evidence at a point in time, provided "as is", without warranties. Ultrametric Inc accepts no responsibility for procurement, purchasing, or other decisions made in reliance on them — verify against the cited evidence before acting (https://ultrametric.ai/productarena/terms).