API platforms arenaBuyer checklist

Every requirement we judge api platforms products against, as a ready-to-send RFP checklist — with each item's priority, why it matters, and how the top-ranked products score on it today.

57 requirements · 11 themes · verdicts for 5 products · updated 2026-09-16 · priorities mirror the story weights our scoring uses (methodology)

Procurement report →

Show the markdown export

# API platforms — buyer checklist (RFP)

Derived from ProductArena's evidence-graded user-story taxonomy for API platforms: 57 judged requirements. Priorities mirror story weights (3 = must-have, 2 = should-have, 1 = nice-to-have).

## Agenticness

- [ ] **[must-have]** Plug MCP servers into this product so it can use their tools
- [ ] **[must-have]** Connect an agent via an official MCP server
- [ ] **[must-have]** Drive the product through a documented public API
- [ ] **[must-have]** Delegate tasks to a built-in AI assistant inside the product
- [ ] **[should-have]** Point an agent at llms.txt or agent-oriented docs
- [ ] **[should-have]** Run the product headlessly / in CI for automation
- [ ] **[should-have]** Use an official CLI
- [ ] **[should-have]** Issue scoped/least-privilege API credentials for an agent
- [ ] **[should-have]** Build against official SDKs
- [ ] **[should-have]** Subscribe to events via webhooks
- [ ] **[should-have]** Get AI-generated insights and suggestions from my data inside the product
- [ ] **[should-have]** Set up automations that run autonomously in the background
- [ ] **[should-have]** Operate the product with natural-language commands
- [ ] **[should-have]** Explore an interactive API reference with runnable examples
- [ ] **[should-have]** Download a machine-readable API spec (OpenAPI or equivalent)
- [ ] **[should-have]** Rely on versioned APIs with a documented deprecation policy
- [ ] **[nice-to-have]** Test against a sandbox environment without touching production data

## Automation depth

- [ ] **[must-have]** Define rules that trigger actions automatically on events
- [ ] **[should-have]** Perform bulk operations across many items at once
- [ ] **[should-have]** Schedule recurring jobs or workflows
- [ ] **[nice-to-have]** Version, review, and roll back my automations

## Ci scripting

- [ ] **[must-have]** Run my API test collections in CI with a CLI and fail the build on regressions
- [ ] **[should-have]** Schedule collection runs as monitors that alert me when an API breaks
- [ ] **[should-have]** Script pre-request and post-response logic to chain requests and massage data

## Collaboration

- [ ] **[must-have]** Share collections with my team in a synced shared workspace
- [ ] **[should-have]** Control who can view or edit APIs and collections with roles and permissions
- [ ] **[should-have]** Publish an internal catalog or portal where teams discover and consume our APIs

## Collections workspaces

- [ ] **[must-have]** Store collections as plain text files in my repo and version them with git
- [ ] **[must-have]** Organize API requests into collections and folders and rerun them anytime
- [ ] **[should-have]** Import and export collections in portable formats to move between tools
- [ ] **[should-have]** Work fully offline in a local workspace without a cloud account
- [ ] **[should-have]** Send GraphQL, gRPC, WebSocket, and SSE requests, not just REST

## Environments secrets

- [ ] **[must-have]** Switch requests between dev, staging, and prod using environment variable sets
- [ ] **[should-have]** Keep credentials out of shared files with masked secrets or an external vault integration

## Gateway traffic

- [ ] **[must-have]** Enforce authentication (API keys, OAuth2, JWT, mTLS) on upstream APIs via gateway plugins or policies
- [ ] **[must-have]** Enforce rate limits on API traffic per consumer or route
- [ ] **[should-have]** Proxy and govern LLM/MCP traffic (token limits, prompt guards, provider routing) through the platform
- [ ] **[should-have]** Manage API infrastructure as declarative config and sync it from CI (GitOps/APIOps)
- [ ] **[should-have]** Monitor API traffic with metrics, logging, and tracing integrations
- [ ] **[should-have]** Route, load-balance, and transform requests between consumers and upstream services

## Openness

- [ ] **[must-have]** Export all of my data in open formats and leave
- [ ] **[must-have]** Self-host the core product
- [ ] **[should-have]** Do everything through the API that I can do in the UI
- [ ] **[should-have]** Read the product's source under an open license

## Privacy posture

- [ ] **[must-have]** Prevent my data from being used to train AI models
- [ ] **[should-have]** Choose where my data is stored (region/residency)
- [ ] **[should-have]** Control data retention and deletion
- [ ] **[should-have]** Opt out of telemetry and usage tracking

## Spec design

- [ ] **[must-have]** Lint API specs against configurable style and governance rules
- [ ] **[must-have]** Import an OpenAPI spec to generate requests and collections
- [ ] **[should-have]** Generate and publish API documentation from my collections or specs
- [ ] **[should-have]** Design an API spec-first and export it as OpenAPI

## Testing mocking

- [ ] **[must-have]** Spin up a mock server from saved examples or a spec so consumers can build before the API exists
- [ ] **[must-have]** Run an entire collection as an automated test suite with a pass/fail report
- [ ] **[must-have]** Write test assertions against response status, body, and headers
- [ ] **[should-have]** Have AI generate test cases or assertions for an endpoint from its spec or responses
- [ ] **[should-have]** Validate live responses against the API contract to catch drift

---

Source: https://ultrametric.ai/productarena/arena/api-platforms (evidence-graded verdicts for 5 products) · methodology: https://ultrametric.ai/productarena/methodology

Chips show the top 5 ranked products' current verdict on each requirement — ✓ full · ~ partial · ! disputed · — none · n/a not applicable.

Agenticness — how well agents can access and operate the productAgenticness· 17 items

How well agents can access and operate the product

ai-native userPlug MCP servers into this product so it can use their tools
Core requirement — weighs 3× in arena scoring · 1 of 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
must-have
ai-native userConnect an agent via an official MCP server
Core requirement — weighs 3× in arena scoring · 3 of 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
must-have
ai-native userDrive the product through a documented public API
Core requirement — weighs 3× in arena scoring · 3 of 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
must-have
ai-native userDelegate tasks to a built-in AI assistant inside the product
Core requirement — weighs 3× in arena scoring · 1 of 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
must-have
ai-native userPoint an agent at llms.txt or agent-oriented docs
Important, not disqualifying — weighs 2× in arena scoring · 4 of 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
should-have
ai-native userRun the product headlessly / in CI for automation
Important, not disqualifying — weighs 2× in arena scoring · all 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
should-have
ai-native userUse an official CLI
Important, not disqualifying — weighs 2× in arena scoring · 4 of 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
should-have
ai-native userIssue scoped/least-privilege API credentials for an agent
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
Postman Kong Bruno Insomnia Hoppscotch
should-have
ai-native userBuild against official SDKs
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
Postman Kong Bruno Insomnia Hoppscotch
should-have
ai-native userSubscribe to events via webhooks
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
Postman Kong Bruno Insomnia Hoppscotch
should-have
ai-native userGet AI-generated insights and suggestions from my data inside the product
Important, not disqualifying — weighs 2× in arena scoring · 1 of 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
should-have
ai-native userSet up automations that run autonomously in the background
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
Postman Kong Bruno Insomnia Hoppscotch
should-have
ai-native userOperate the product with natural-language commands
Important, not disqualifying — weighs 2× in arena scoring · 1 of 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
should-have
ai-native userExplore an interactive API reference with runnable examples
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
Postman Kong Bruno Insomnia Hoppscotch
should-have
ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)
Important, not disqualifying — weighs 2× in arena scoring · 2 of 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
should-have
ai-native userRely on versioned APIs with a documented deprecation policy
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
Postman Kong Bruno Insomnia Hoppscotch
should-have
ai-native userTest against a sandbox environment without touching production data
Differentiator, not a dealbreaker — weighs 1× in arena scoring · no product fully delivers this yet
Postman Kong Bruno Insomnia Hoppscotch
nice-to-have

Automation depth — how much of the product can run unattendedAutomation depth· 4 items

How much of the product can run unattended

ai-native userDefine rules that trigger actions automatically on events
Core requirement — weighs 3× in arena scoring · no product fully delivers this yet
Postman Kong Bruno Insomnia Hoppscotch
must-have
ai-native userPerform bulk operations across many items at once
Important, not disqualifying — weighs 2× in arena scoring · 2 of 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
should-have
ai-native userSchedule recurring jobs or workflows
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
Postman Kong Bruno Insomnia Hoppscotch
should-have
ai-native userVersion, review, and roll back my automations
Differentiator, not a dealbreaker — weighs 1× in arena scoring · 2 of 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
nice-to-have

Ci scripting — stories about ci scripting in this arenaCi scripting· 3 items

Stories about ci scripting in this arena

developerRun my API test collections in CI with a CLI and fail the build on regressions
Core requirement — weighs 3× in arena scoring · 3 of 4 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
must-have
developerSchedule collection runs as monitors that alert me when an API breaks
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
Postman Kong Bruno Insomnia Hoppscotch
should-have
developerScript pre-request and post-response logic to chain requests and massage data
Important, not disqualifying — weighs 2× in arena scoring · 2 of 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
should-have

Collaboration — working as a team — sharing, comments, roles, simultaneous editingCollaboration· 3 items

Working as a team — sharing, comments, roles, simultaneous editing

developerShare collections with my team in a synced shared workspace
Core requirement — weighs 3× in arena scoring · 1 of 4 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
must-have
platform-engineerControl who can view or edit APIs and collections with roles and permissions
Important, not disqualifying — weighs 2× in arena scoring · 1 of 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
should-have
platform-engineerPublish an internal catalog or portal where teams discover and consume our APIs
Important, not disqualifying — weighs 2× in arena scoring · 1 of 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
should-have

Collections workspaces — stories about collections workspaces in this arenaCollections workspaces· 5 items

Stories about collections workspaces in this arena

developerStore collections as plain text files in my repo and version them with git
Core requirement — weighs 3× in arena scoring · 2 of 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
must-have
developerOrganize API requests into collections and folders and rerun them anytime
Core requirement — weighs 3× in arena scoring · all 4 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
must-have
developerImport and export collections in portable formats to move between tools
Important, not disqualifying — weighs 2× in arena scoring · 1 of 4 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
should-have
developerWork fully offline in a local workspace without a cloud account
Important, not disqualifying — weighs 2× in arena scoring · 1 of 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
should-have
developerSend GraphQL, gRPC, WebSocket, and SSE requests, not just REST
Important, not disqualifying — weighs 2× in arena scoring · 1 of 4 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
should-have

Environments secrets — stories about environments secrets in this arenaEnvironments secrets· 2 items

Stories about environments secrets in this arena

developerSwitch requests between dev, staging, and prod using environment variable sets
Core requirement — weighs 3× in arena scoring · 2 of 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
must-have
platform-engineerKeep credentials out of shared files with masked secrets or an external vault integration
Important, not disqualifying — weighs 2× in arena scoring · 3 of 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
should-have

Gateway traffic — stories about gateway traffic in this arenaGateway traffic· 6 items

Stories about gateway traffic in this arena

platform-engineerEnforce authentication (API keys, OAuth2, JWT, mTLS) on upstream APIs via gateway plugins or policies
Core requirement — weighs 3× in arena scoring · all 1 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
must-have
platform-engineerEnforce rate limits on API traffic per consumer or route
Core requirement — weighs 3× in arena scoring · all 1 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
must-have
ai-native userProxy and govern LLM/MCP traffic (token limits, prompt guards, provider routing) through the platform
Important, not disqualifying — weighs 2× in arena scoring · 1 of 3 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
should-have
platform-engineerManage API infrastructure as declarative config and sync it from CI (GitOps/APIOps)
Important, not disqualifying — weighs 2× in arena scoring · 1 of 4 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
should-have
platform-engineerMonitor API traffic with metrics, logging, and tracing integrations
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
Postman Kong Bruno Insomnia Hoppscotch
should-have
platform-engineerRoute, load-balance, and transform requests between consumers and upstream services
Important, not disqualifying — weighs 2× in arena scoring · all 1 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
should-have

Openness — open source, data portability, and self-hosting storiesOpenness· 4 items

Open source, data portability, and self-hosting stories

ai-native userExport all of my data in open formats and leave
Core requirement — weighs 3× in arena scoring · 1 of 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
must-have
ai-native userSelf-host the core product
Core requirement — weighs 3× in arena scoring · 3 of 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
must-have
ai-native userDo everything through the API that I can do in the UI
Important, not disqualifying — weighs 2× in arena scoring · 1 of 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
should-have
ai-native userRead the product's source under an open license
Important, not disqualifying — weighs 2× in arena scoring · 2 of 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
should-have

Privacy posture — data-handling and privacy storiesPrivacy posture· 4 items

Data-handling and privacy stories

ai-native userPrevent my data from being used to train AI models
Core requirement — weighs 3× in arena scoring · 1 of 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
must-have
ai-native userChoose where my data is stored (region/residency)
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
Postman Kong Bruno Insomnia Hoppscotch
should-have
ai-native userControl data retention and deletion
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
Postman Kong Bruno Insomnia Hoppscotch
should-have
ai-native userOpt out of telemetry and usage tracking
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
Postman Kong Bruno Insomnia Hoppscotch
should-have

Spec design — stories about spec design in this arenaSpec design· 4 items

Stories about spec design in this arena

platform-engineerLint API specs against configurable style and governance rules
Core requirement — weighs 3× in arena scoring · 1 of 3 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
must-have
developerImport an OpenAPI spec to generate requests and collections
Core requirement — weighs 3× in arena scoring · 1 of 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
must-have
developerGenerate and publish API documentation from my collections or specs
Important, not disqualifying — weighs 2× in arena scoring · 1 of 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
should-have
developerDesign an API spec-first and export it as OpenAPI
Important, not disqualifying — weighs 2× in arena scoring · 1 of 5 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
should-have

Testing mocking — stories about testing mocking in this arenaTesting mocking· 5 items

Stories about testing mocking in this arena

developerSpin up a mock server from saved examples or a spec so consumers can build before the API exists
Core requirement — weighs 3× in arena scoring · 2 of 3 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
must-have
developerRun an entire collection as an automated test suite with a pass/fail report
Core requirement — weighs 3× in arena scoring · all 4 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
must-have
developerWrite test assertions against response status, body, and headers
Core requirement — weighs 3× in arena scoring · 2 of 4 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
must-have
ai-native userHave AI generate test cases or assertions for an endpoint from its spec or responses
Important, not disqualifying — weighs 2× in arena scoring · 1 of 4 products fully deliver this today
Postman Kong Bruno Insomnia Hoppscotch
should-have
platform-engineerValidate live responses against the API contract to catch drift
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
Postman Kong Bruno Insomnia Hoppscotch
should-have

Full evidence behind every verdict lives on the arena page and each product page — chips above deep-link straight to the judged story.

🔌API platforms arenaBuyer checklist

🤖Agenticness — how well agents can access and operate the productAgenticness· 17 items

⚙️Automation depth — how much of the product can run unattendedAutomation depth· 4 items

💻Ci scripting — stories about ci scripting in this arenaCi scripting· 3 items

👥Collaboration — working as a team — sharing, comments, roles, simultaneous editingCollaboration· 3 items

👥Collections workspaces — stories about collections workspaces in this arenaCollections workspaces· 5 items

🛡️Environments secrets — stories about environments secrets in this arenaEnvironments secrets· 2 items

🔌Gateway traffic — stories about gateway traffic in this arenaGateway traffic· 6 items

🔓Openness — open source, data portability, and self-hosting storiesOpenness· 4 items

🔒Privacy posture — data-handling and privacy storiesPrivacy posture· 4 items

🎨Spec design — stories about spec design in this arenaSpec design· 4 items

🧪Testing mocking — stories about testing mocking in this arenaTesting mocking· 5 items