Skip to content
ProductArena

Model Gateways & Routers Arena

Model Gateways & Routers arenaBuyer checklist

Every requirement we judge model gateways & routers products against, as a ready-to-send RFP checklist — with each item's priority, why it matters, and how the top-ranked products score on it today.

50 requirements · 11 themes · verdicts for 7 products · updated 2026-09-14 · priorities mirror the story weights our scoring uses (methodology)

Procurement report →
Show the markdown export
# Model Gateways & Routers — buyer checklist (RFP)

Derived from ProductArena's evidence-graded user-story taxonomy for Model Gateways & Routers: 50 judged requirements. Priorities mirror story weights (3 = must-have, 2 = should-have, 1 = nice-to-have).

## Agenticness

- [ ] **[must-have]** Plug MCP servers into this product so it can use their tools
- [ ] **[must-have]** Connect an agent via an official MCP server
- [ ] **[must-have]** Drive the product through a documented public API
- [ ] **[must-have]** Delegate tasks to a built-in AI assistant inside the product
- [ ] **[should-have]** Point an agent at llms.txt or agent-oriented docs
- [ ] **[should-have]** Run the product headlessly / in CI for automation
- [ ] **[should-have]** Use an official CLI
- [ ] **[should-have]** Issue scoped/least-privilege API credentials for an agent
- [ ] **[should-have]** Build against official SDKs
- [ ] **[should-have]** Subscribe to events via webhooks
- [ ] **[should-have]** Get AI-generated insights and suggestions from my data inside the product
- [ ] **[should-have]** Set up automations that run autonomously in the background
- [ ] **[should-have]** Operate the product with natural-language commands
- [ ] **[should-have]** Explore an interactive API reference with runnable examples
- [ ] **[should-have]** Download a machine-readable API spec (OpenAPI or equivalent)
- [ ] **[should-have]** Rely on versioned APIs with a documented deprecation policy
- [ ] **[nice-to-have]** Test against a sandbox environment without touching production data

## Automation depth

- [ ] **[must-have]** Define rules that trigger actions automatically on events
- [ ] **[should-have]** Perform bulk operations across many items at once
- [ ] **[should-have]** Schedule recurring jobs or workflows
- [ ] **[nice-to-have]** Version, review, and roll back my automations

## Caching performance

- [ ] **[should-have]** Cache responses at the gateway to cut cost and latency on repeated requests
- [ ] **[should-have]** Run traffic through gateway infrastructure that adds minimal latency overhead to provider calls

## Cost controls

- [ ] **[must-have]** Set hard budgets and spend limits per key, team, or user
- [ ] **[must-have]** Track spend per model, key, team, or user across all providers in one place
- [ ] **[should-have]** Give an autonomous agent its own key with budget and rate guardrails so it cannot run away on spend

## Key management

- [ ] **[must-have]** Provision gateways, keys, and budgets programmatically through an admin API
- [ ] **[must-have]** Mint gateway-managed keys for teams and apps without exposing raw provider keys
- [ ] **[should-have]** Bring my own provider API keys and have the gateway use them for my traffic

## Observability

- [ ] **[must-have]** Inspect logged requests and responses with latency, token counts, and cost attached
- [ ] **[nice-to-have]** Export gateway logs and traces to my own observability stack

## Openness

- [ ] **[must-have]** Export all of my data in open formats and leave
- [ ] **[must-have]** Self-host the core product
- [ ] **[should-have]** Do everything through the API that I can do in the UI
- [ ] **[should-have]** Read the product's source under an open license

## Privacy posture

- [ ] **[must-have]** Prevent my data from being used to train AI models
- [ ] **[should-have]** Choose where my data is stored (region/residency)
- [ ] **[should-have]** Control data retention and deletion
- [ ] **[should-have]** Opt out of telemetry and usage tracking

## Routing resilience

- [ ] **[must-have]** Configure automatic fallback to another model or provider when one fails
- [ ] **[must-have]** My agent can switch models mid-task by policy — cost, capability, or availability — through gateway routing rules
- [ ] **[should-have]** Load-balance traffic across providers, deployments, or keys by weight, latency, or cost
- [ ] **[should-have]** Smooth provider rate limits by spreading traffic across keys and queuing or throttling requests
- [ ] **[should-have]** Set automatic retry policies for transient provider errors

## Streaming tools

- [ ] **[must-have]** Stream token-by-token responses through the gateway from any provider
- [ ] **[must-have]** Make tool and function calls across different providers with a consistent schema
- [ ] **[nice-to-have]** Request structured JSON-schema outputs across providers

## Unified api

- [ ] **[must-have]** Point existing OpenAI-compatible code at the gateway by changing only the base URL and key
- [ ] **[must-have]** Call many model providers through one consistent API
- [ ] **[should-have]** Browse or query a catalog of available models with pricing and context-window metadata

---

Source: https://ultrametric.ai/productarena/arena/model-gateways (evidence-graded verdicts for 7 products) · methodology: https://ultrametric.ai/productarena/methodology

Chips show the top 5 ranked products' current verdict on each requirement — ✓ full · ~ partial · ! disputed · — none · n/a not applicable.

Agenticness — how well agents can access and operate the productAgenticness· 17 items

How well agents can access and operate the product

Automation depth — how much of the product can run unattendedAutomation depth· 4 items

How much of the product can run unattended

Caching performance — stories about caching performance in this arenaCaching performance· 2 items

Stories about caching performance in this arena

Cost controls — stories about cost controls in this arenaCost controls· 3 items

Stories about cost controls in this arena

Key management — stories about key management in this arenaKey management· 3 items

Stories about key management in this arena

Observability — seeing what the system is doing — logs, metrics, traces, alertsObservability· 2 items

Seeing what the system is doing — logs, metrics, traces, alerts

Openness — open source, data portability, and self-hosting storiesOpenness· 4 items

Open source, data portability, and self-hosting stories

Privacy posture — data-handling and privacy storiesPrivacy posture· 4 items

Data-handling and privacy stories

Routing resilience — stories about routing resilience in this arenaRouting resilience· 5 items

Stories about routing resilience in this arena

Streaming tools — stories about streaming tools in this arenaStreaming tools· 3 items

Stories about streaming tools in this arena

Unified api — stories about unified api in this arenaUnified api· 3 items

Stories about unified api in this arena

Full evidence behind every verdict lives on the arena page and each product page — chips above deep-link straight to the judged story.