E-commerce Platforms — procurement report
ProductArena · rankings as of 2026-09-16 · evidence as of 2026-09-16 · 5 products · 54 judged requirements · 270 judged cells
Methodology: Every product is judged against a shared taxonomy of user stories using cited evidence — hands-on probes > repository code > independent community sources > vendor claims — never opinion. Full writeup: https://ultrametric.ai/productarena/methodology
Leaderboard
| # | Product | PA Score | Coverage score | Applicable cells | Confidence |
|---|---|---|---|---|---|
| 1 | WooCommerce | 37.2 | 36.9 | 52/54 | A |
| 2 | Shopify | 32.2 | 32.1 | 52/54 | C |
| 3 | Medusa | 32.0 | 26.7 | 53/54 | A |
| 4 | BigCommerce | 25.6 | 21.5 | 53/54 | C |
| 5 | Swell | 17.3 | 23.0 | 52/54 | B |
PA Score = agent-readiness blend (see methodology). Coverage score = weighted share of judged requirements met. Confidence = how much of the score rests on tested vs claimed evidence (A–D).
Uncertainty note
The current #1/#2 gap in this arena is not close enough to qualify for the multi-judge uncertainty pass (or the pass has not covered it yet) — no extra caveat applies beyond the per-product confidence grades above.
Buyer checklist (RFP)
The arena's 54 judged user stories as requirements, grouped by theme. Priorities mirror the story weights our scoring uses (3 = must-have, 2 = should-have, 1 = nice-to-have). Interactive version with per-requirement verdicts for the top products: /arena/ecommerce-platforms/checklist
Agenticness — how well agents can access and operate the productAgenticness
How well agents can access and operate the product
- ai-native userPlug MCP servers into this product so it can use their toolsmust-have
- ai-native userConnect an agent via an official MCP servermust-have
- ai-native userDrive the product through a documented public APImust-have
- ai-native userDelegate tasks to a built-in AI assistant inside the productmust-have
- ai-native userPoint an agent at llms.txt or agent-oriented docsshould-have
- ai-native userRun the product headlessly / in CI for automationshould-have
- ai-native userUse an official CLIshould-have
- ai-native userIssue scoped/least-privilege API credentials for an agentshould-have
- ai-native userBuild against official SDKsshould-have
- ai-native userSubscribe to events via webhooksshould-have
- ai-native userGet AI-generated insights and suggestions from my data inside the productshould-have
- ai-native userSet up automations that run autonomously in the backgroundshould-have
- ai-native userOperate the product with natural-language commandsshould-have
- ai-native userExplore an interactive API reference with runnable examplesshould-have
- ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)should-have
- ai-native userRely on versioned APIs with a documented deprecation policyshould-have
- ai-native userTest against a sandbox environment without touching production datanice-to-have
Ai commerce — stories about ai commerce in this arenaAi commerce
Stories about ai commerce in this arena
- ai-native userA shopping agent can discover my store's products and build carts through a documented agent-commerce endpoint on the store (MCP/UCP), not by scrapingmust-have
- ai-native userMy products are discoverable by AI agents and answer engines through a cross-merchant catalog or feed built for agentsshould-have
- ai-native userAn agent can take an order through its post-purchase lifecycle — status, fulfillment, returns — via documented APIs and webhooks without a human in the adminshould-have
Automation depth — how much of the product can run unattendedAutomation depth
How much of the product can run unattended
- ai-native userDefine rules that trigger actions automatically on eventsmust-have
- ai-native userPerform bulk operations across many items at onceshould-have
- ai-native userSchedule recurring jobs or workflowsshould-have
- ai-native userVersion, review, and roll back my automationsnice-to-have
B2b wholesale — stories about b2b wholesale in this arenaB2b wholesale
Stories about b2b wholesale in this arena
- merchantSell B2B and wholesale — customer-specific price lists, volume pricing, quotes or net-terms invoicing — alongside DTCshould-have
Catalog inventory — stories about catalog inventory in this arenaCatalog inventory
Stories about catalog inventory in this arena
- merchantModel products with variants, options, and custom fields/metafields that fit my catalog, not a fixed templatemust-have
- operations leadBulk import, export, and edit the catalog — CSV or API — without touching products one at a timeshould-have
- operations leadInventory tracks across multiple locations with stock adjustments and fulfillment from the right warehouseshould-have
Checkout conversion — stories about checkout conversion in this arenaCheckout conversion
Stories about checkout conversion in this arena
- developerCustomize checkout — fields, branding, discounts, and validation logic — through supported extension points rather than forking checkoutmust-have
- merchantBuyers can pay with accelerated wallets — Apple Pay, Google Pay, PayPal, platform one-click — without re-entering detailsshould-have
- merchantAbandoned carts and checkouts trigger recovery emails or links so I can win the sale backnice-to-have
Extensibility apps — stories about extensibility apps in this arenaExtensibility apps
Stories about extensibility apps in this arena
- merchantExtend my store from a marketplace of vetted apps and extensions covering the long tail of featuresmust-have
- developerBuild custom apps and extensions with official SDKs, scaffolding tooling, and a documented review/distribution pathshould-have
- developerInject custom backend logic — discounts, shipping rules, validation — as serverless functions or hooks that run inside the platformshould-have
Headless apis — stories about headless apis in this arenaHeadless apis
Stories about headless apis in this arena
- developerManage the full commerce lifecycle programmatically — create products, edit orders, issue refunds, trigger fulfillments — through the admin APImust-have
- developerBuild a fully custom headless storefront on a documented storefront API or first-party frontend frameworkmust-have
Multi channel — stories about multi channel in this arenaMulti channel
Stories about multi channel in this arena
- merchantList and sync products to external channels — marketplaces, social commerce, shopping feeds — from one catalogshould-have
- merchantSell internationally with localized storefronts, multi-currency pricing, translations, and duties handledshould-have
- merchantSell in person with a point-of-sale that shares the same catalog, inventory, and customers as my online storeshould-have
Openness — open source, data portability, and self-hosting storiesOpenness
Open source, data portability, and self-hosting stories
- ai-native userExport all of my data in open formats and leavemust-have
- ai-native userSelf-host the core productmust-have
- ai-native userDo everything through the API that I can do in the UIshould-have
- ai-native userRead the product's source under an open licenseshould-have
Payments — stories about payments in this arenaPayments
Stories about payments in this arena
- merchantTake payments with the platform's integrated processing — published card rates, payouts, and fraud tooling — without wiring a third-party gatewaymust-have
- merchantChoose among third-party payment gateways and charge in multiple currencies when the built-in option doesn't fitshould-have
Privacy posture — data-handling and privacy storiesPrivacy posture
Data-handling and privacy stories
- ai-native userPrevent my data from being used to train AI modelsmust-have
- ai-native userChoose where my data is stored (region/residency)should-have
- ai-native userControl data retention and deletionshould-have
- ai-native userOpt out of telemetry and usage trackingshould-have
Storefront experience — stories about storefront experience in this arenaStorefront experience
Stories about storefront experience in this arena
- merchantLaunch a polished storefront from professionally designed themes and customize layout, sections, and branding without codemust-have
- developerCustomize storefront code with a documented templating layer, local preview, and version-controlled theme deploysshould-have
- merchantManage content — pages, blogs, media — alongside my products from the same adminnice-to-have
Taxes shipping — stories about taxes shipping in this arenaTaxes shipping
Stories about taxes shipping in this arena
- operations leadCheckout shows live carrier rates and I can buy labels and track shipments from the platformshould-have
- merchantSales tax and VAT are calculated automatically per jurisdiction, with rates kept current and reports I can file fromshould-have
Appendix: recorded probes
Hands-on probe recordings — transcripts/videos a human can replay, the strongest evidence tier. Watch them at https://ultrametric.ai/productarena/proofs
- BigCommerce
curl -s -X POST https://docs.bigcommerce.com/_mcp/server -H 'Content-Type: application/json' -d '<jsonrpc initialize>' # docs MCP from docs.bigcommerce.com/llms.txtterminal · recorded 2026-09-08 · exit 0 - Medusa
curl -si -X POST https://docs.medusajs.com/mcp -H 'Content-Type: application/json' -d '<jsonrpc initialize>'terminal · recorded 2026-09-08 · exit 0 - Medusa
mktemp -d && npm install @medusajs/js-sdk && node -e "import('@medusajs/js-sdk').then(m=>console.log('PA_PROBE_OK medusa js-sdk export:', typeof m.default))"terminal · recorded 2026-09-08 · exit 0 - Shopify
curl -s https://shopify.dev/docs/agents.md | sed -n 1,14pterminal · recorded 2026-09-08 · exit 0 - Shopify
curl -sL https://www.shopify.com/llms.txt | head -4terminal · recorded 2026-09-08 · exit 0 - Shopify
curl -si -X POST https://catalog.shopify.com/api/ucp/mcp -H 'Content-Type: application/json' -d '<jsonrpc initialize>' # full keyless handshaketerminal · recorded 2026-09-08 · exit 0 - Shopify
mktemp -d && npm install @shopify/ucp-cli && ucp --version && ucp --helpterminal · recorded 2026-09-08 · exit 0 - Shopify
curl -s -X POST https://hardware.shopify.com/api/ucp/mcp -H 'Content-Type: application/json' -d '<jsonrpc initialize>' # merchant-storefront UCP endpointterminal · recorded 2026-09-08 · exit 0 - Shopify
ucp catalog search --business https://hardware.shopify.com --set /query='card reader' && ucp cart create --business https://hardware.shopify.com --set /line_items/0/item/id=<variant> # real keyless cart on Shopify's own storeterminal · recorded 2026-09-08 · exit 0 - Shopify
npm install @shopify/ucp-cli && ucp profile init --name pa-probe && ucp catalog search --set /query='wireless headphones' # keyless live cross-merchant searchterminal · recorded 2026-09-08 · exit 0 - Shopify
curl -s -X POST https://catalog.shopify.com/api/ucp/mcp -d '<tools/call search_catalog without agent profile>' # documented profile gate answers with a structured errorterminal · recorded 2026-09-08 · exit 0 - Swell
mktemp -d && npm install @swell/cli && swell --versionterminal · recorded 2026-09-08 · exit 0 - Swell
curl -s https://raw.githubusercontent.com/swellstores/skills/main/README.md | head -8terminal · recorded 2026-09-08 · exit 0 - WooCommerce
curl -s 'https://api.wordpress.org/plugins/info/1.2/?action=plugin_information&request[slug]=woocommerce' | python3 -c '<print slug, version, active_installs>'terminal · recorded 2026-09-08 · exit 0
Cite as: ProductArena by Ultrametric Inc, E-commerce Platforms arena, rankings as of 2026-09-16 — https://ultrametric.ai/productarena/arena/ecommerce-platforms
License: © 2026 Ultrametric Inc. Brief quotation of individual verdicts, scores, or evidence excerpts is permitted with attribution to "ProductArena by Ultrametric Inc (ultrametric.ai/productarena)", as is use of the data to evaluate, contest, or contribute corrections. Bulk copying, redistribution, or use to build competing datasets requires prior written permission (see DATA-LICENSE in the repository).
No liability: rankings, verdicts, and scores are research outputs derived from the cited evidence at a point in time, provided "as is", without warranties. Ultrametric Inc accepts no responsibility for procurement, purchasing, or other decisions made in reliance on them — verify against the cited evidence before acting (https://ultrametric.ai/productarena/terms).