Model Gateways & Routers Arena
Model Gateways & Routers — procurement report
ProductArena · rankings as of 2026-09-14 · evidence as of 2026-09-14 · 7 products · 50 judged requirements · 350 judged cells
Methodology: Every product is judged against a shared taxonomy of user stories using cited evidence — hands-on probes > repository code > independent community sources > vendor claims — never opinion. Full writeup: https://ultrametric.ai/productarena/methodology
Leaderboard
| # | Product | PA Score | Coverage score | Applicable cells | Confidence |
|---|---|---|---|---|---|
| 1 | LiteLLM | 33.9 | 47.7 | 44/50 | B |
| 2 | OpenRouter | 29.7 | 47.5 | 44/50 | B |
| 3 | Vercel AI Gateway | 28.7 | 41.3 | 43/50 | B |
| 4 | Portkey | 19.5 | 38.7 | 47/50 | C |
| 5 | Cloudflare AI Gateway | 19.4 | 33.1 | 43/50 | C |
| 6 | Kong AI Gateway | 19.3 | 31.4 | 46/50 | C |
| 7 | Requesty | 17.8 | 30.8 | 44/50 | B |
PA Score = agent-readiness blend (see methodology). Coverage score = weighted share of judged requirements met. Confidence = how much of the score rests on tested vs claimed evidence (A–D).
Uncertainty note
The current #1/#2 gap in this arena is not close enough to qualify for the multi-judge uncertainty pass (or the pass has not covered it yet) — no extra caveat applies beyond the per-product confidence grades above.
Buyer checklist (RFP)
The arena's 50 judged user stories as requirements, grouped by theme. Priorities mirror the story weights our scoring uses (3 = must-have, 2 = should-have, 1 = nice-to-have). Interactive version with per-requirement verdicts for the top products: /arena/model-gateways/checklist
Agenticness — how well agents can access and operate the productAgenticness
How well agents can access and operate the product
- ai-native userPlug MCP servers into this product so it can use their toolsmust-have
- ai-native userConnect an agent via an official MCP servermust-have
- ai-native userDrive the product through a documented public APImust-have
- ai-native userDelegate tasks to a built-in AI assistant inside the productmust-have
- ai-native userPoint an agent at llms.txt or agent-oriented docsshould-have
- ai-native userRun the product headlessly / in CI for automationshould-have
- ai-native userUse an official CLIshould-have
- ai-native userIssue scoped/least-privilege API credentials for an agentshould-have
- ai-native userBuild against official SDKsshould-have
- ai-native userSubscribe to events via webhooksshould-have
- ai-native userGet AI-generated insights and suggestions from my data inside the productshould-have
- ai-native userSet up automations that run autonomously in the backgroundshould-have
- ai-native userOperate the product with natural-language commandsshould-have
- ai-native userExplore an interactive API reference with runnable examplesshould-have
- ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)should-have
- ai-native userRely on versioned APIs with a documented deprecation policyshould-have
- ai-native userTest against a sandbox environment without touching production datanice-to-have
Automation depth — how much of the product can run unattendedAutomation depth
How much of the product can run unattended
- ai-native userDefine rules that trigger actions automatically on eventsmust-have
- ai-native userPerform bulk operations across many items at onceshould-have
- ai-native userSchedule recurring jobs or workflowsshould-have
- ai-native userVersion, review, and roll back my automationsnice-to-have
Caching performance — stories about caching performance in this arenaCaching performance
Stories about caching performance in this arena
- developerCache responses at the gateway to cut cost and latency on repeated requestsshould-have
- platform engineerRun traffic through gateway infrastructure that adds minimal latency overhead to provider callsshould-have
Cost controls — stories about cost controls in this arenaCost controls
Stories about cost controls in this arena
- platform engineerSet hard budgets and spend limits per key, team, or usermust-have
- platform engineerTrack spend per model, key, team, or user across all providers in one placemust-have
- ai-native userGive an autonomous agent its own key with budget and rate guardrails so it cannot run away on spendshould-have
Key management — stories about key management in this arenaKey management
Stories about key management in this arena
- ai-native userProvision gateways, keys, and budgets programmatically through an admin APImust-have
- platform engineerMint gateway-managed keys for teams and apps without exposing raw provider keysmust-have
- developerBring my own provider API keys and have the gateway use them for my trafficshould-have
Observability — seeing what the system is doing — logs, metrics, traces, alertsObservability
Seeing what the system is doing — logs, metrics, traces, alerts
- platform engineerInspect logged requests and responses with latency, token counts, and cost attachedmust-have
- developerExport gateway logs and traces to my own observability stacknice-to-have
Openness — open source, data portability, and self-hosting storiesOpenness
Open source, data portability, and self-hosting stories
- ai-native userExport all of my data in open formats and leavemust-have
- ai-native userSelf-host the core productmust-have
- ai-native userDo everything through the API that I can do in the UIshould-have
- ai-native userRead the product's source under an open licenseshould-have
Privacy posture — data-handling and privacy storiesPrivacy posture
Data-handling and privacy stories
- ai-native userPrevent my data from being used to train AI modelsmust-have
- ai-native userChoose where my data is stored (region/residency)should-have
- ai-native userControl data retention and deletionshould-have
- ai-native userOpt out of telemetry and usage trackingshould-have
Routing resilience — stories about routing resilience in this arenaRouting resilience
Stories about routing resilience in this arena
- platform engineerConfigure automatic fallback to another model or provider when one failsmust-have
- ai-native userMy agent can switch models mid-task by policy — cost, capability, or availability — through gateway routing rulesmust-have
- platform engineerLoad-balance traffic across providers, deployments, or keys by weight, latency, or costshould-have
- platform engineerSmooth provider rate limits by spreading traffic across keys and queuing or throttling requestsshould-have
- platform engineerSet automatic retry policies for transient provider errorsshould-have
Streaming tools — stories about streaming tools in this arenaStreaming tools
Stories about streaming tools in this arena
- developerStream token-by-token responses through the gateway from any providermust-have
- developerMake tool and function calls across different providers with a consistent schemamust-have
- developerRequest structured JSON-schema outputs across providersnice-to-have
Unified api — stories about unified api in this arenaUnified api
Stories about unified api in this arena
- developerPoint existing OpenAI-compatible code at the gateway by changing only the base URL and keymust-have
- developerCall many model providers through one consistent APImust-have
- developerBrowse or query a catalog of available models with pricing and context-window metadatashould-have
Pricing signals
Extracted verbatim from each vendor's own pricing page — never converted, averaged, or derived. Products whose page prints no unit price are recorded as unclear, honestly.
| Product | Headline price | Unit | As of |
|---|---|---|---|
| LiteLLM | free tier | gateway fee | 2026-09-07 |
| OpenRouter | 5% | gateway fee | 2026-09-07 |
| Vercel AI Gateway | 0% | gateway fee | 2026-09-07 |
| Portkey | $49 | month (entry plan) | 2026-09-07 |
| Cloudflare AI Gateway | 5% | gateway fee | 2026-09-07 |
| Requesty | 5% | gateway fee | 2026-09-07 |
Appendix: recorded probes
Hands-on probe recordings — transcripts/videos a human can replay, the strongest evidence tier. Watch them at https://ultrametric.ai/productarena/proofs
- Kong AI Gateway
curl -s https://developer.konghq.com/ai-gateway.md | head -8terminal · recorded 2026-09-14 · exit 0 - Kong AI Gateway
curl -s https://developer.konghq.com/llms.txt | grep -A 4 '## AI Gateway'terminal · recorded 2026-09-14 · exit 0
Cite as: ProductArena by Ultrametric Inc, Model Gateways & Routers arena, rankings as of 2026-09-14 — https://ultrametric.ai/productarena/arena/model-gateways
License: © 2026 Ultrametric Inc. Brief quotation of individual verdicts, scores, or evidence excerpts is permitted with attribution to "ProductArena by Ultrametric Inc (ultrametric.ai/productarena)", as is use of the data to evaluate, contest, or contribute corrections. Bulk copying, redistribution, or use to build competing datasets requires prior written permission (see DATA-LICENSE in the repository).
No liability: rankings, verdicts, and scores are research outputs derived from the cited evidence at a point in time, provided "as is", without warranties. Ultrametric Inc accepts no responsibility for procurement, purchasing, or other decisions made in reliance on them — verify against the cited evidence before acting (https://ultrametric.ai/productarena/terms).