Model Gateways & Routers Arena
Portkey vs Cloudflare AI Gateway
Portkey
Portkey, Inc.
Portkey wins · 16–8 (19 drawn)
Agenticness — how well agents can access and operate the productAgenticness
How well agents can access and operate the product
Agent access
ai-native userPoint an agent at llms.txt or agent-oriented docs
weight 2 · round to PortkeyPortkey hosts a working llms.txt file (HTTP 200) with a description of the product, confirming agents can be pointed at it for agent-oriented docs. Missing for 10: no docs.md fallback (404) or OpenAPI spec confirmation, and no independent/community corroboration of agents actually using it.
Cloudflare's developer docs (which cover AI Gateway) expose a working llms.txt at developers.cloudflare.com/llms.txt, letting an agent discover agent-oriented documentation; an OpenAPI spec is also available. However, the AI Gateway-specific docs page itself returns 404 for the .md variant, so agent-native markdown access is inconsistent at the product-specific level. Missing for 10: a dedicated llms.txt or .md endpoint scoped to AI Gateway docs specifically, and confirmation the top-level llms.txt indexes AI Gateway pages.
- [probe] “PROBE llms.txt: HTTP 200 at https://developers.cloudflare.com/llms.txt # Cloudflare Developer Documentation Explore guides and tutorials to…”
- [probe] “PROBE docs-md: HTTP 404 at https://developers.cloudflare.com/ai-gateway/.md”
- [probe] “PROBE openapi: HTTP 200 at https://developers.cloudflare.com/openapi.json — contains "openapi" key”
ai-native userRun the product headlessly / in CI for automation
weight 2 · round to Cloudflare AI GatewayPortkey's gateway can be run headlessly via a CLI command (npx @portkey-ai/gateway) and REST API with a base URL, and it is open source and self-hostable, which supports CI/automation usage. However, there is no explicit documentation of CI pipeline integration, GitHub Actions examples, or headless automation testing workflows. missing for 10: explicit CI/CD examples, headless automation guides, non-interactive scripted usage documentation, independent verification of CI usage.
- [claimed-docs] “npx @portkey-ai/gateway”
- [claimed-docs] “We've open sourced our battle-tested AI gateway to the community. You can run it locally with a single command:”
- [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey’s features into your existing setup.”
- [claimed-docs] “The base URL for all REST API requests is `https://api.portkey.ai/v1`”
AI Gateway is API/header-driven by design (OpenAI-compatible endpoint, JSON-based dynamic routing config, header-based controls like cf-aig-cache-status and cf-aig-collect-log), which naturally supports headless/programmatic use in scripts or CI pipelines. However, there is no explicit documentation of CI/CD integration, automation examples, or a CLI for pipeline use. Missing for 10: explicit CI/CD workflow examples, dedicated CLI/SDK for automation, documented non-interactive auth/setup flow for pipelines.
- [claimed-docs] “Cloudflare's AI Gateway offers an OpenAI-compatible `/chat/completions` endpoint, enabling integration with multiple AI providers using a si…”
- [claimed-docs] “Dynamic routing enables you to create request routing flows through a **visual interface** or a **JSON-based configuration**.”
- [claimed-docs] “Switch providers by changing the \`model\` and \`apiKey\` parameters.”
- [claimed-docs] “The \`cf-aig-collect-log\` header allows you to bypass the default log setting for the gateway.”
- [claimed-docs] “To check whether a response comes from cache or not, **cf-aig-cache-status** will be designated as `HIT` or `MISS`.”
ai-native userConnect an agent via an official MCP server
weight 3 · round drawnPortkeynone0/10Portkey's docs describe connecting *to* remote MCP servers as a client-side feature (portkey-docs-17), but there is no evidence that Portkey itself exposes an official MCP server endpoint that external agents can connect to. Missing for 10: any documentation of Portkey hosting/serving an MCP endpoint, MCP server setup instructions, or third-party confirmation of agents connecting via Portkey's MCP server.
- [claimed-docs] “Connect to Remote MCP severs, allowing you to connect external tools and data sources.”
Cloudflare AI Gatewaynone0/10Cloudflare AI Gateway is a proxy/gateway for routing and observing LLM API calls; the evidence pack contains no mention of an MCP server, MCP protocol, or agent connectivity via MCP. This is an applicable axis (a gateway product could plausibly ship an MCP server for agentic access) but no evidence supports it.
ai-native userUse an official CLI
weight 2 · round to PortkeyThe only CLI-like evidence is running the open-source gateway locally via `npx @portkey-ai/gateway`, which self-hosts the gateway rather than offering a full-featured CLI for managing configs, logs, or other Portkey resources. Missing for 10: a documented dedicated CLI tool/command set, CLI-based workflow examples, and independent corroboration of CLI usage.
- [claimed-docs] “npx @portkey-ai/gateway”
- [claimed-docs] “We've open sourced our battle-tested AI gateway to the community. You can run it locally with a single command:”
Cloudflare AI Gatewaynone0/10No evidence in the pack mentions an official CLI for AI Gateway (e.g., Wrangler or a dedicated AI Gateway CLI); all evidence covers dashboard, API headers, and configuration features. missing for 10: any mention of a CLI tool, command-line usage, or CLI-based configuration for AI Gateway.
ai-native userDrive the product through a documented public API
weight 3 · round to PortkeyPortkey exposes a documented REST API (base URL https://api.portkey.ai/v1) with API-reference docs for its inference API, OpenAI-compatible integration, and llms.txt discovery confirms API-first positioning; gateway is also open-sourced and npx-runnable. Missing for 10: a discoverable OpenAPI/swagger spec (probe found all candidate paths 404) and independent/third-party corroboration of API usage.
- [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey’s features into your existing setup.”
- [claimed-docs] “The base URL for all REST API requests is `https://api.portkey.ai/v1`”
- [claimed-docs] “Ideal for: if you’re currently utilizing OpenAI’s Python or Node.js SDKs. By changing the base URL and adding Portkey-specific headers, you …”
- [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey's features into your existing setup.”
- [probe] “PROBE llms.txt: HTTP 200 at https://portkey.ai/llms.txt # Portkey > Portkey is a production stack for Gen AI builders, equipping AI teams w…”
- [probe] “PROBE openapi: all candidate paths 404 (https://portkey.ai/openapi.json, https://portkey.ai/swagger.json, https://portkey.ai/api/openapi.jso…”
- [claimed-docs] “We've open sourced our battle-tested AI gateway to the community. You can run it locally with a single command:”
Cloudflare AI Gateway exposes a documented public API surface: an OpenAI-compatible /chat/completions endpoint, native provider passthroughs, a Universal endpoint for fallbacks, control headers (cf-aig-*), and a patchLog method for feedback/metadata, all described in first-party docs, and Cloudflare's platform-wide openapi.json confirms a machine-readable API schema exists. missing for 10: independent/hands-on third-party corroboration of API usage and a dedicated AI-Gateway-specific OpenAPI reference beyond the platform-wide one.
- [claimed-docs] “Cloudflare's AI Gateway offers an OpenAI-compatible `/chat/completions` endpoint, enabling integration with multiple AI providers using a si…”
- [claimed-docs] “Switch providers by changing the \`model\` and \`apiKey\` parameters.”
- [claimed-docs] “Connect to AI providers using their native API formats through AI Gateway.”
- [claimed-docs] “Manage all your costs with one simple bill and access every provider through a single API.”
- [claimed-docs] “The patchLog method allows you to send feedback, score, and metadata for a specific log ID.”
- [probe] “PROBE openapi: HTTP 200 at https://developers.cloudflare.com/openapi.json — contains "openapi" key”
ai-native userIssue scoped/least-privilege API credentials for an agent
weight 2 · round to PortkeyPortkey lets admins issue API keys scoped with budgets, rate limits, and model allow-lists at org/workspace level, and virtual keys keep provider credentials out of code — this maps well to least-privilege credential issuance. However, the docs frame this around human/org governance rather than explicitly per-agent scoped credentials, and there's no independent corroboration of enforcement in practice. Missing for 10: explicit agent-specific credential scoping workflow, independent/hands-on validation of enforcement.
- [claimed-docs] “Configure budget and rate limits on API keys to effectively manage AI spending and usage across your organization”
- [claimed-docs] “Budget limits allow you to set maximum LLM spending or token usage thresholds on your API keys, automatically preventing further usage when …”
- [claimed-docs] “Fine-grained control: budgets, rate limits, and model allow-lists at both org and workspace level.”
- [claimed-docs] “Set hourly, daily, or per minute rate limits on requests or tokens sent”
- [claimed-docs] “Provider credentials stored securely, never exposed in code”
Cloudflare AI Gatewaynone0/10The evidence pack covers AI Gateway's routing, caching, rate limiting, logging, and fallback features, but there is no mention of issuing scoped or least-privilege API credentials/tokens for agents (e.g., per-agent API keys with restricted permissions). Rate limiting and per-user/project quotas are traffic controls, not credential scoping mechanisms.
ai-native userBuild against official SDKs
weight 2 · round to PortkeyPortkey documents SDK-style integration (base URL swap for OpenAI Python/Node SDKs, REST API reference, open-source gateway installable via npx) supporting AI-native builders, but the evidence pack lacks explicit mention of dedicated first-party SDKs beyond OpenAI-compatible drop-in usage, and OpenAPI spec probes returned 404s. missing for 10: dedicated official multi-language SDK docs (e.g., Python/JS Portkey SDK reference pages), working OpenAPI/spec discovery, independent developer corroboration of SDK usage.
- [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey’s features into your existing setup.”
- [claimed-docs] “The base URL for all REST API requests is `https://api.portkey.ai/v1`”
- [claimed-docs] “Ideal for: if you’re currently utilizing OpenAI’s Python or Node.js SDKs. By changing the base URL and adding Portkey-specific headers, you …”
- [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey's features into your existing setup.”
- [claimed-docs] “We've open sourced our battle-tested AI gateway to the community. You can run it locally with a single command:”
- [probe] “PROBE openapi: all candidate paths 404 (https://portkey.ai/openapi.json, https://portkey.ai/swagger.json, https://portkey.ai/api/openapi.jso…”
AI Gateway is compatible with OpenAI-style SDKs and providers' native APIs (via the OpenAI-compatible /chat/completions endpoint and native API pass-through), and Workers AI bindings imply SDK-like integration, but the evidence pack never explicitly names or documents an official Cloudflare AI Gateway SDK for major languages (e.g., a dedicated 'cloudflare-ai-gateway' npm/python package) with API references. missing for 10: explicit official SDK documentation/package listing, multi-language SDK examples, independent confirmation of SDK reliability/maintenance.
- [claimed-docs] “Cloudflare's AI Gateway offers an OpenAI-compatible `/chat/completions` endpoint, enabling integration with multiple AI providers using a si…”
- [claimed-docs] “Switch providers by changing the \`model\` and \`apiKey\` parameters.”
- [claimed-docs] “Connect to AI providers using their native API formats through AI Gateway.”
- [claimed-docs] “This feature simplifies the integration process, allowing for seamless switching between different models without significant code modificat…”
- [claimed-docs] “gateway: { id: "my-gateway", },”
ai-native userSubscribe to events via webhooks
weight 2 · round drawnPortkeynone0/10No evidence in the pack mentions webhooks or event subscriptions of any kind; Portkey's docs cover gateway routing, caching, retries, budgets, and logs but nothing about outbound webhook notifications for events.
Cloudflare AI Gatewaynone0/10No evidence in the pack mentions webhooks or any event-subscription mechanism; AI Gateway's documented features are logging, caching, rate limiting, fallbacks, and dynamic routing, none of which describe outbound webhook notifications for events. This is a plausible axis for a gateway product (it could offer webhook callbacks on events like errors or budget thresholds), so absence of evidence yields 'none' rather than 'na'.
Api quality
ai-native userExplore an interactive API reference with runnable examples
weight 2 · round to PortkeyPortkey documents a REST API reference (base URL, headers, SDK compatibility) at /docs/api-reference, but the evidence pack contains no mention of an interactive explorer or runnable code examples, and probes for a discoverable OpenAPI/Swagger spec all returned 404s. Missing for 10: evidence of an interactive 'try it' console, runnable code snippets, and a published OpenAPI/Swagger schema.
- [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey’s features into your existing setup.”
- [claimed-docs] “The base URL for all REST API requests is `https://api.portkey.ai/v1`”
- [claimed-docs] “Ideal for: if you’re currently utilizing OpenAI’s Python or Node.js SDKs. By changing the base URL and adding Portkey-specific headers, you …”
- [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey's features into your existing setup.”
- [probe] “PROBE openapi: all candidate paths 404 (https://portkey.ai/openapi.json, https://portkey.ai/swagger.json, https://portkey.ai/api/openapi.jso…”
Cloudflare AI Gatewaynone0/10Evidence shows only static documentation pages with code snippets and an openapi.json file, but no interactive API reference, playground, or runnable-example explorer is described anywhere in the pack.
- [probe] “PROBE openapi: HTTP 200 at https://developers.cloudflare.com/openapi.json — contains "openapi" key”
- [claimed-docs] “Better yet - it only takes one line of code to get started.”
- [claimed-docs] “This feature simplifies the integration process, allowing for seamless switching between different models without significant code modificat…”
ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)
weight 2 · round to Cloudflare AI GatewayPortkeynone0/10Portkey has an API reference (base URL, inference API docs) but no evidence of a downloadable OpenAPI/Swagger spec; a direct probe for common OpenAPI paths returned 404s, and docs.md also 404s.
- [probe] “PROBE openapi: all candidate paths 404 (https://portkey.ai/openapi.json, https://portkey.ai/swagger.json, https://portkey.ai/api/openapi.jso…”
- [probe] “PROBE docs-md: HTTP 404 at https://portkey.ai/docs.md”
- [claimed-docs] “The base URL for all REST API requests is `https://api.portkey.ai/v1`”
Cloudflare publishes a machine-readable OpenAPI spec at a public, verifiable endpoint (openapi.json containing an 'openapi' key), covering the Cloudflare API including AI Gateway configuration/management endpoints, plus an llms.txt for AI-native discovery. missing for 10: no dedicated AI-Gateway-specific OpenAPI file (it's part of the broader Cloudflare API spec) and no independent/community confirmation of completeness.
ai-native userTest against a sandbox environment without touching production data
weight 1 · round drawnPortkeynone0/10Portkey's evidence covers gateway routing, fallbacks, caching, logging, and gradual/canary rollout of models in production, but there is no mention of a distinct sandbox/staging environment or test mode that isolates a user from production data. The axis is fair for a gateway/dev-tool product, but no evidence supports it.
ai-native userRely on versioned APIs with a documented deprecation policy
weight 2 · round drawnPortkeynone0/10No evidence of API versioning scheme (e.g., v1/v2 in URLs beyond a single 'v1' base path) or any documented deprecation policy/changelog; OpenAPI spec probe also returned 404s. Missing for 10: versioning scheme documentation, explicit deprecation policy, changelog/migration guides, and any independent corroboration.
- [claimed-docs] “The base URL for all REST API requests is `https://api.portkey.ai/v1`”
- [probe] “PROBE openapi: all candidate paths 404 (https://portkey.ai/openapi.json, https://portkey.ai/swagger.json, https://portkey.ai/api/openapi.jso…”
Cloudflare AI Gatewaynone0/10Evidence covers AI Gateway's features (caching, rate limiting, fallbacks, logging) but nowhere mentions API versioning schemes or a documented deprecation policy for the AI Gateway API itself; the openapi.json probe confirms an API spec exists but says nothing about version lifecycle or deprecation commitments. Missing for 10: explicit versioning scheme documentation, deprecation policy/timeline, changelog or migration guides for breaking changes.
- [probe] “PROBE openapi: HTTP 200 at https://developers.cloudflare.com/openapi.json — contains "openapi" key”
Automation depth — how much of the product can run unattendedAutomation depth
How much of the product can run unattended
ai-native userPerform bulk operations across many items at once
weight 2 · round drawnPortkeynone0/10Portkey's evidence covers gateway routing, fallbacks, caching, logging, and budget/rate-limit controls, but nothing describes a bulk/batch operation capability (e.g., batch inference calls, bulk editing of configs/keys, or bulk log actions) that lets a user act on many items at once. Missing for 10: any documentation of a batch API, bulk config/key management, or bulk log/annotation actions.
ai-native userDefine rules that trigger actions automatically on events
weight 3 · round drawnPortkey ships several built-in rule/event mechanisms — fallback triggers on non-2xx status codes, automatic retries with backoff, circuit-breaker/failure handling per strategy, and budget/rate-limit thresholds that automatically send notifications or block further usage — which are automatic actions triggered by defined conditions. However, these are fixed, product-defined automations (reliability/spend rules) rather than a general user-authorable rule engine where an AI-native user can define arbitrary custom triggers/actions. Missing for 10: general-purpose custom rule/trigger builder, arbitrary event types beyond status codes/spend thresholds, user-defined action targets (e.g. webhooks, custom scripts) beyond built-in notifications/fallback/retry.
- [claimed-docs] “Specify a prioritized list of providers/models. If the primary LLM fails, Portkey automatically falls back to the next in line.”
- [claimed-docs] “By default, fallback triggers on any **non-2xx** status code. Customize with `on_status_codes`”
- [claimed-docs] “Automatically retry failed LLM requests with exponential backoff.”
- [claimed-docs] “Configure per-strategy circuit protection and failure handling”
- [claimed-docs] “When usage reaches this threshold, notifications will be sent to configured recipients”
- [claimed-docs] “Budget limits allow you to set maximum LLM spending or token usage thresholds on your API keys, automatically preventing further usage when …”
- [claimed-docs] “Set hourly, daily, or per minute rate limits on requests or tokens sent”
AI Gateway supports rule-based automation like dynamic routing that evaluates conditions and triggers fallback/model-switching actions, rate limiting, and cache overrides via headers, which are event-triggered automated behaviors. However, this is narrowly scoped to request routing/traffic control rather than a general-purpose 'define custom rules that trigger arbitrary actions on events' system (e.g., no evidence of user-defined triggers for alerts, webhooks, or custom workflows beyond routing/fallback/caching/rate-limits). missing for 10: evidence of arbitrary custom event-action rule definitions (e.g., webhook/alert triggers on thresholds), independent/hands-on corroboration of dynamic routing behavior, and confirmation of a general automation/rules engine beyond routing, caching, and rate limiting.
- [claimed-docs] “Dynamic routing enables you to create request routing flows through a **visual interface** or a **JSON-based configuration**.”
- [claimed-docs] “Directing different segments (paid/not-paid user) to different models”
- [claimed-docs] “Restricting each user/project/team with budget/rate limits”
- [claimed-docs] “you compose a small flow that evaluates conditions, enforces quotas, and chooses models with fallbacks. You can iterate without touching app…”
- [claimed-docs] “Improve resilience by defining request retry and model fallbacks in case of an error.”
- [claimed-docs] “Sends a request to Workers AI Inference API. 2. If that request fails, proceeds to OpenAI.”
- [claimed-docs] “Cloudflare can trigger your fallback provider in response to request errors or predetermined request timeouts.”
- [claimed-docs] “Rate limiting controls the traffic that reaches your application, which prevents expensive bills and suspicious activity.”
- [claimed-docs] “You can define rate limits as the number of requests that get sent in a specific time frame. For example, you can limit your application to …”
ai-native userVersion, review, and roll back my automations
weight 1 · round to Cloudflare AI GatewayPortkeynone0/10Portkey's evidence covers gateway configs, logs, retries, caching, and budgets, but nothing describes versioning, review workflows, or rollback of automations/configs — the closest hint is 'Config ID' filtering in logs, which is not version control. Missing for 10: any documentation of config/version history, diff/review process, or rollback mechanism for automations.
- [claimed-docs] “Filter logs by `Config ID` to see all requests using that config”
- [claimed-docs] “The Logs section presents a chronological list of all the requests processed through Portkey.”
- [claimed-docs] “Each log on Portkey has a unique URL. You can copy the link from the address bar and directly share it with anyone in your org.”
Docs mention that dynamic routing lets you 'publish a new route version' without touching application code, implying some versioning of routing configs, but there is no documented review/diff interface or explicit rollback mechanism for automations. missing for 10: explicit rollback/undo functionality, change-review or diff UI, versioning of prompts/logic beyond routing config.
- [claimed-docs] “you compose a small flow that evaluates conditions, enforces quotas, and chooses models with fallbacks. You can iterate without touching app…”
- [claimed-docs] “Dynamic routing enables you to create request routing flows through a **visual interface** or a **JSON-based configuration**.”
- [claimed-docs] “A/B and gradual rollouts while making it accessible to both developers and non-technical team members.”
Caching performance — stories about caching performance in this arenaCaching performance
Stories about caching performance in this arena
Caching
developerCache responses at the gateway to cut cost and latency on repeated requests
weight 2 · round drawnPortkey's docs explicitly describe gateway-level caching (simple and semantic) that serves responses up to 20x faster and cheaper, including semantic matching via cosine similarity for near-duplicate queries. This directly matches the story's cost/latency reduction on repeated requests. missing for 10: independent/hands-on benchmark corroboration beyond vendor docs.
- [claimed-docs] “Cache LLM responses to serve requests up to **20x faster** and cheaper.”
- [claimed-docs] “Matches requests with **similar meaning** using cosine similarity, not just identical text.”
Docs clearly describe gateway-level caching that serves responses from Cloudflare's cache instead of calling the model provider (cost/latency savings), with per-request header overrides and a cf-aig-cache-status HIT/MISS indicator for verification. Missing for 10: independent/hands-on benchmark corroborating actual latency/cost savings.
- [claimed-docs] “Serve requests directly from Cloudflare's cache instead of the original model provider for faster requests and cost savings.”
- [claimed-docs] “AI Gateway allows you to override default cache behaviors on a per-request basis using specific HTTP headers.”
- [claimed-docs] “To check whether a response comes from cache or not, **cf-aig-cache-status** will be designated as `HIT` or `MISS`.”
Latency
platform engineerRun traffic through gateway infrastructure that adds minimal latency overhead to provider calls
weight 2 · round to PortkeyPortkey's docs claim caching can serve responses '20x faster' and market the gateway as 'performant,' but there is no direct benchmark or first-party claim quantifying the gateway's own added latency overhead per provider call, nor independent corroboration of low overhead. Missing for 10: explicit latency/overhead benchmarks for the proxy itself, third-party performance testing, and architecture details (e.g., edge/WASM deployment) that would substantiate minimal added latency beyond the cache-hit speedup claim.
- [claimed-docs] “Cache LLM responses to serve requests up to **20x faster** and cheaper.”
- [claimed-docs] “It takes 2 mins to integrate and with that, it starts monitoring all of your LLM requests and makes your app resilient, secure, performant, …”
- [claimed-docs] “offering advanced tools for control, visibility, and security in your Generative AI apps”
- [claimed-docs] “We've open sourced our battle-tested AI gateway to the community. You can run it locally with a single command:”
Cloudflare AI Gatewaynone0/10The evidence pack documents caching, rate limiting, fallbacks, and analytics features but contains no benchmarks, latency figures, or explicit claims about the gateway's own processing overhead added to provider calls. Caching (docs-4, docs-31) improves speed only for cache hits and says nothing about baseline pass-through latency. Missing for 10: any latency benchmark, p50/p99 overhead numbers, or first-party/independent claim that the proxy adds minimal delay to non-cached requests.
- [claimed-docs] “Serve requests directly from Cloudflare's cache instead of the original model provider for faster requests and cost savings.”
- [claimed-docs] “To check whether a response comes from cache or not, **cf-aig-cache-status** will be designated as `HIT` or `MISS`.”
Cost controls — stories about cost controls in this arenaCost controls
Stories about cost controls in this arena
Agent guardrails
ai-native userGive an autonomous agent its own key with budget and rate guardrails so it cannot run away on spend
weight 2 · round to PortkeyPortkey docs explicitly support issuing API keys with budget and rate limits (spend thresholds, hourly/daily/per-minute request/token limits) that automatically block further usage when exceeded, directly matching the guardrail need for an autonomous agent's key. Fine-grained control at org/workspace level and per-key configuration is documented. Missing for 10: no explicit mention of a per-agent key issuance workflow or independent/hands-on verification that the cutoff actually stops an agent mid-run.
- [claimed-docs] “Configure budget and rate limits on API keys to effectively manage AI spending and usage across your organization”
- [claimed-docs] “Budget limits allow you to set maximum LLM spending or token usage thresholds on your API keys, automatically preventing further usage when …”
- [claimed-docs] “Set hourly, daily, or per minute rate limits on requests or tokens sent”
- [claimed-docs] “Fine-grained control: budgets, rate limits, and model allow-lists at both org and workspace level.”
- [claimed-docs] “**Reset Weekly**: Budget limits reset every Sunday at 12 AM UTC”
AI Gateway's dynamic routing explicitly supports 'restricting each user/project/team with budget/rate limits' and its rate-limiting feature lets you cap requests per time window (fixed/sliding), which maps to giving an agent bounded spend/rate guardrails when configured as its own route/project. However, the evidence never describes issuing a distinct API key per agent tied to a dollar-denominated budget cap — the mechanism shown is request-count rate limiting plus routing-level quotas, not a native 'agent key + $ budget' guardrail. missing for 10: explicit per-agent key issuance, dollar-based spend caps (not just request counts), and independent confirmation the budget/rate restriction in dynamic routing enforces hard spend caps.
- [claimed-docs] “Restricting each user/project/team with budget/rate limits”
- [claimed-docs] “Rate limiting controls the traffic that reaches your application, which prevents expensive bills and suspicious activity.”
- [claimed-docs] “You can also select if you would like a **fixed** or **sliding** rate limiting technique.”
- [claimed-docs] “You can define rate limits as the number of requests that get sent in a specific time frame. For example, you can limit your application to …”
- [claimed-docs] “Control how your application scales by limiting the number of requests your application receives.”
- [claimed-docs] “Dynamic routing enables you to create request routing flows through a **visual interface** or a **JSON-based configuration**.”
- [claimed-docs] “you compose a small flow that evaluates conditions, enforces quotas, and chooses models with fallbacks. You can iterate without touching app…”
Budgets
platform engineerSet hard budgets and spend limits per key, team, or user
weight 3 · round to PortkeyPortkey's docs explicitly describe setting budget limits (max spend or token usage) and rate limits on API keys, with org/workspace-level fine-grained control, automatic enforcement when limits are reached, notification thresholds, and weekly resets — directly matching per-key/team/user budget/spend controls. Missing for 10: no independent/hands-on corroboration beyond first-party docs.
- [claimed-docs] “Configure budget and rate limits on API keys to effectively manage AI spending and usage across your organization”
- [claimed-docs] “When usage reaches this threshold, notifications will be sent to configured recipients”
- [claimed-docs] “Budget limits allow you to set maximum LLM spending or token usage thresholds on your API keys, automatically preventing further usage when …”
- [claimed-docs] “Fine-grained control: budgets, rate limits, and model allow-lists at both org and workspace level.”
- [claimed-docs] “**Reset Weekly**: Budget limits reset every Sunday at 12 AM UTC”
- [claimed-docs] “Set hourly, daily, or per minute rate limits on requests or tokens sent”
AI Gateway supports rate limiting per app/gateway and dynamic routing docs mention 'restricting each user/project/team with budget/rate limits,' but there's no dedicated documentation on setting a hard dollar spend cap or budget limit per API key, team, or user that blocks requests once exceeded — the budget/rate-limit reference is a single bullet without configuration detail. missing for 10: dedicated budget-limit configuration docs (dollar-amount caps), per-key/per-user budget enforcement examples, and evidence of automatic cutoff/alerting when a hard budget is reached.
- [claimed-docs] “Restricting each user/project/team with budget/rate limits”
- [claimed-docs] “You can define rate limits as the number of requests that get sent in a specific time frame. For example, you can limit your application to …”
- [claimed-docs] “Rate limiting controls the traffic that reaches your application, which prevents expensive bills and suspicious activity.”
- [claimed-docs] “Control how your application scales by limiting the number of requests your application receives.”
Spend tracking
platform engineerTrack spend per model, key, team, or user across all providers in one place
weight 3 · round to PortkeyPortkey's docs describe budget/rate limits and spend caps on API keys, a unified model catalog with org/workspace-level governance and allow-lists, and logs filterable by Config ID for auditing usage, which together give spend visibility across providers/keys/teams. Missing for 10: explicit per-user spend breakdown dashboards and independent/hands-on confirmation of a unified cost dashboard beyond docs claims.
- [claimed-docs] “Configure budget and rate limits on API keys to effectively manage AI spending and usage across your organization”
- [claimed-docs] “Budget limits allow you to set maximum LLM spending or token usage thresholds on your API keys, automatically preventing further usage when …”
- [claimed-docs] “Fine-grained control: budgets, rate limits, and model allow-lists at both org and workspace level.”
- [claimed-docs] “A single pane to view and manage every AI provider and model in your organization. It provides centralized governance, discovery, and usage …”
- [claimed-docs] “Filter logs by `Config ID` to see all requests using that config”
- [claimed-docs] “The core concept remains the same - one Portkey API key gives you access to multiple providers and models.”
- [claimed-docs] “Set hourly, daily, or per minute rate limits on requests or tokens sent”
AI Gateway logs per-request cost, tokens, provider, and model, and supports dynamic routing with per-user/project/team budget and rate limits, giving a foundation for spend tracking across providers in one dashboard. However, there is no explicit evidence of built-in aggregated spend dashboards or reports broken down by 'team' or 'user' as first-class filters/rollups, or per-API-key spend attribution beyond raw logs. missing for 10: dedicated cost-aggregation/reporting UI by team/user/key, evidence of budget alerts tied to spend thresholds, independent/hands-on confirmation of cross-provider spend rollups.
- [claimed-docs] “View metrics such as the number of requests, tokens, and the cost it takes to run your application.”
- [claimed-docs] “Restricting each user/project/team with budget/rate limits”
- [claimed-docs] “Your AI Gateway dashboard shows logs of individual requests, including the user prompt, model response, provider, timestamp, request status,…”
- [claimed-docs] “Manage all your costs with one simple bill and access every provider through a single API.”
Key management — stories about key management in this arenaKey management
Stories about key management in this arena
Byok
developerBring my own provider API keys and have the gateway use them for my traffic
weight 2 · round drawnPortkey's virtual keys and model catalog explicitly let developers add/store their own provider API keys ('Add a provider, then use @provider-slug/model-name', 'Provider credentials stored securely, never exposed in code'), and the gateway routes traffic through these keys with governance/rate-limit controls layered on top. Missing for 10: independent hands-on verification of the BYO-key flow and no explicit mention of per-key traffic isolation/audit beyond docs claims.
- [claimed-docs] “The core concept remains the same - one Portkey API key gives you access to multiple providers and models.”
- [claimed-docs] “Add a provider, then use `@provider-slug/model-name` in your code.”
- [claimed-docs] “Provider credentials stored securely, never exposed in code”
- [claimed-docs] “A single pane to view and manage every AI provider and model in your organization. It provides centralized governance, discovery, and usage …”
- [claimed-docs] “Fine-grained control: budgets, rate limits, and model allow-lists at both org and workspace level.”
Docs confirm AI Gateway proxies requests to providers using the developer's own API key (e.g. 'Switch providers by changing the model and apiKey parameters' and passthrough pricing with no markup, meaning the developer's own provider account/key is billed directly), and it connects to native provider APIs and OpenAI-compatible endpoints for many providers. missing for 10: no explicit first-party doc section titled 'bring your own key' explaining storage/security of BYO keys, and no independent/hands-on confirmation of this exact flow.
- [claimed-docs] “Switch providers by changing the \`model\` and \`apiKey\` parameters.”
- [claimed-docs] “Connect to AI providers using their native API formats through AI Gateway.”
- [claimed-docs] “Inference pricing from providers is passed through with no markup — you pay the same per-token rates as you would directly with the provider…”
- [claimed-docs] “Workers AI, Anthropic, Google Gemini, OpenAI, Replicate, and more work with AI Gateway.”
- [claimed-docs] “Cloudflare's AI Gateway offers an OpenAI-compatible `/chat/completions` endpoint, enabling integration with multiple AI providers using a si…”
Programmatic admin
ai-native userProvision gateways, keys, and budgets programmatically through an admin API
weight 3 · round to Cloudflare AI GatewayPortkeynone0/10Evidence shows budget/rate-limit and key management as dashboard-configurable features (docs-13, docs-25, docs-35, docs-38) but never describes a programmatic Admin API for provisioning gateways, keys, or budgets; the only API reference documented is the inference API (docs-15, docs-16, docs-36), and probes for an OpenAPI/admin API spec returned 404 (portkey-probe-3, portkey-probe-2).
- [claimed-docs] “Configure budget and rate limits on API keys to effectively manage AI spending and usage across your organization”
- [claimed-docs] “Budget limits allow you to set maximum LLM spending or token usage thresholds on your API keys, automatically preventing further usage when …”
- [claimed-docs] “**Reset Weekly**: Budget limits reset every Sunday at 12 AM UTC”
- [claimed-docs] “The base URL for all REST API requests is `https://api.portkey.ai/v1`”
- [probe] “PROBE openapi: all candidate paths 404 (https://portkey.ai/openapi.json, https://portkey.ai/swagger.json, https://portkey.ai/api/openapi.jso…”
- [probe] “PROBE docs-md: HTTP 404 at https://portkey.ai/docs.md”
Docs show gateways can be identified/configured programmatically (e.g., gateway id in code) and budgets/rate limits can be set via JSON-based dynamic routing config, suggesting some programmatic provisioning, and Cloudflare's general openapi.json exists. However there is no explicit documentation of an admin API for creating gateways, issuing/rotating API keys, or setting budgets outside the dashboard/JSON routing config. Missing for 10: explicit AI Gateway REST/GraphQL API endpoints for gateway CRUD, key provisioning, and budget management with request/response examples.
- [claimed-docs] “Restricting each user/project/team with budget/rate limits”
- [claimed-docs] “you compose a small flow that evaluates conditions, enforces quotas, and chooses models with fallbacks. You can iterate without touching app…”
- [claimed-docs] “gateway: { id: "my-gateway", },”
- [probe] “PROBE openapi: HTTP 200 at https://developers.cloudflare.com/openapi.json — contains "openapi" key”
Virtual keys
platform engineerMint gateway-managed keys for teams and apps without exposing raw provider keys
weight 3 · round to PortkeyPortkey's virtual keys/model catalog explicitly let platform engineers mint provider-agnostic API keys that map to real provider credentials, which are stored securely and never exposed in code, while budgets, rate limits, and model allow-lists can be enforced per key at org/workspace level for teams and apps. This directly matches minting gateway-managed keys without exposing raw provider keys. Missing for 10: independent/hands-on verification beyond vendor docs, and detail on per-team/app key issuance workflow (e.g., self-service or RBAC-scoped key creation) rather than admin-level config.
- [claimed-docs] “The core concept remains the same - one Portkey API key gives you access to multiple providers and models.”
- [claimed-docs] “Provider credentials stored securely, never exposed in code”
- [claimed-docs] “Configure budget and rate limits on API keys to effectively manage AI spending and usage across your organization”
- [claimed-docs] “Budget limits allow you to set maximum LLM spending or token usage thresholds on your API keys, automatically preventing further usage when …”
- [claimed-docs] “Fine-grained control: budgets, rate limits, and model allow-lists at both org and workspace level.”
- [claimed-docs] “Set hourly, daily, or per minute rate limits on requests or tokens sent”
- [claimed-docs] “Add a provider, then use `@provider-slug/model-name` in your code.”
Observability — seeing what the system is doing — logs, metrics, traces, alertsObservability
Seeing what the system is doing — logs, metrics, traces, alerts
Integrations
developerExport gateway logs and traces to my own observability stack
weight 1 · round drawnPortkeynone0/10Evidence describes Portkey's own logs UI, sharing log URLs, and manual feedback tagging, but there is no mention of exporting logs/traces to third-party observability stacks (e.g., OpenTelemetry, Datadog, Grafana) or any API/webhook for bulk export.
Cloudflare AI Gatewaynone0/10Evidence describes Cloudflare's own dashboard logging, metrics, and header-based controls for log collection (e.g., cf-aig-collect-log, patchLog), but there is no mention of exporting logs/traces to an external observability stack (e.g., via Logpush, webhook, or third-party integration).
Logs
platform engineerInspect logged requests and responses with latency, token counts, and cost attached
weight 3 · round to Cloudflare AI GatewayPortkey's docs confirm a Logs section showing a chronological list of all requests with unique shareable URLs and manual feedback tagging, supporting log inspection generally, and repeated claims of 'monitoring all LLM requests' and 'visibility' across the gateway. However, the evidence pack never explicitly documents latency, token count, or cost fields being displayed per log entry. Missing for 10: explicit documentation or screenshots showing latency/tokens/cost columns in the logs UI, independent/hands-on confirmation of these fields.
- [claimed-docs] “The Logs section presents a chronological list of all the requests processed through Portkey.”
- [claimed-docs] “Each log on Portkey has a unique URL. You can copy the link from the address bar and directly share it with anyone in your org.”
- [claimed-docs] “you can also add manual feedback on the logs to be analysed and filtered later”
- [claimed-docs] “It takes 2 mins to integrate and with that, it starts monitoring all of your LLM requests”
- [claimed-docs] “It takes 2 mins to integrate and with that, it starts monitoring all of your LLM requests and makes your app resilient, secure, performant, …”
Docs explicitly state the dashboard shows per-request logs including user prompt, model response, provider, timestamp, status, token usage, cost, duration, and user agent, plus aggregate metrics for requests/tokens/cost. This directly matches the story's requirement to inspect logged requests/responses with latency, token counts, and cost. Missing for 10: independent/hands-on corroboration beyond first-party docs.
- [claimed-docs] “Your AI Gateway dashboard shows logs of individual requests, including the user prompt, model response, provider, timestamp, request status,…”
- [claimed-docs] “View metrics such as the number of requests, tokens, and the cost it takes to run your application.”
- [claimed-docs] “Create comprehensive monitoring and alerting systems using AI Gateway's rich logging and metrics data.”
Openness — open source, data portability, and self-hosting storiesOpenness
Open source, data portability, and self-hosting stories
ai-native userDo everything through the API that I can do in the UI
weight 2 · round to Cloudflare AI GatewayPortkey's core AI gateway (chat/completions, fallbacks, caching, retries, load balancing) is clearly exposed via a REST API with a stable base URL, and the open-source gateway can be run locally — showing strong API-first design for request routing. However there is no evidence of API access to the administrative/UI-only features such as budget & rate-limit configuration, log viewing/sharing, feedback tagging, or virtual-key management, and probes show no public OpenAPI spec, so full UI-API parity isn't demonstrated. missing for 10: API endpoints for budget/rate-limit admin, log retrieval/sharing via API, feedback API, and a public OpenAPI spec confirming complete parity.
- [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey’s features into your existing setup.”
- [claimed-docs] “The base URL for all REST API requests is `https://api.portkey.ai/v1`”
- [claimed-docs] “Ideal for: if you’re currently utilizing OpenAI’s Python or Node.js SDKs. By changing the base URL and adding Portkey-specific headers, you …”
- [claimed-docs] “We've open sourced our battle-tested AI gateway to the community. You can run it locally with a single command:”
- [claimed-docs] “Configure budget and rate limits on API keys to effectively manage AI spending and usage across your organization”
- [claimed-docs] “The Logs section presents a chronological list of all the requests processed through Portkey.”
- [probe] “PROBE openapi: all candidate paths 404 (https://portkey.ai/openapi.json, https://portkey.ai/swagger.json, https://portkey.ai/api/openapi.jso…”
AI Gateway configuration such as caching, rate limiting, fallbacks, and dynamic routing can be set via headers, API/JSON config, and the OpenAI-compatible endpoint, and an OpenAPI spec exists confirming broad API coverage. However, some features like dynamic routing are explicitly described as configurable via a 'visual interface' as an alternative to JSON, and there's no explicit confirmation that every dashboard capability (e.g., A/B rollouts, log feedback via patchLog, full analytics views) is fully API-exposed with parity to the UI. missing for 10: explicit statement of full UI/API parity, evidence that dashboard-only features (analytics visualizations, visual routing builder) have equivalent API endpoints, independent confirmation of complete parity.
- [claimed-docs] “Dynamic routing enables you to create request routing flows through a **visual interface** or a **JSON-based configuration**.”
- [claimed-docs] “Cloudflare's AI Gateway offers an OpenAI-compatible `/chat/completions` endpoint, enabling integration with multiple AI providers using a si…”
- [claimed-docs] “AI Gateway allows you to override default cache behaviors on a per-request basis using specific HTTP headers.”
- [claimed-docs] “The \`cf-aig-collect-log\` header allows you to bypass the default log setting for the gateway.”
- [claimed-docs] “The patchLog method allows you to send feedback, score, and metadata for a specific log ID.”
- [probe] “PROBE openapi: HTTP 200 at https://developers.cloudflare.com/openapi.json — contains "openapi" key”
ai-native userExport all of my data in open formats and leave
weight 3 · round drawnPortkeynone0/10Portkey's docs cover logging, gateway configuration, and self-hosting the open-source gateway, but there is no mention of exporting stored logs/configs/data in open formats or facilitating a full data export for migration away from the platform. Shareable log URLs (portkey-docs-12) are not a bulk export/portability feature.
- [claimed-docs] “The Logs section presents a chronological list of all the requests processed through Portkey.”
- [claimed-docs] “Each log on Portkey has a unique URL. You can copy the link from the address bar and directly share it with anyone in your org.”
- [claimed-docs] “We've open sourced our battle-tested AI gateway to the community. You can run it locally with a single command:”
Cloudflare AI Gatewaynone0/10Evidence pack covers logging, caching, fallback, rate-limiting, and routing features, but nowhere mentions a data export capability or open-format export/data portability for users leaving the platform. No documentation of bulk export, data download, or migration tooling is present.
ai-native userRead the product's source under an open license
weight 2 · round to PortkeyPortkey explicitly states it open-sourced its AI gateway component ('We've open sourced our battle-tested AI gateway to the community... run it locally with a single command'), which supports reading some source code. However, this only covers the gateway, not the full product (observability, admin console, model catalog, etc.), and no explicit license name or repository link is provided in the evidence. Missing for 10: explicit open-source license identification, confirmation that the broader Portkey platform (not just the gateway) is open source, and a linked public repository.
- [claimed-docs] “We've open sourced our battle-tested AI gateway to the community. You can run it locally with a single command:”
- [claimed-docs] “npx @portkey-ai/gateway”
Privacy posture — data-handling and privacy storiesPrivacy posture
Data-handling and privacy stories
ai-native userChoose where my data is stored (region/residency)
weight 2 · round drawnPortkeynone0/10No evidence pack items mention data residency, regional storage options, or geographic hosting choices; Portkey's docs focus on gateway routing, caching, retries, budgets, and observability rather than data residency controls.
Cloudflare AI Gatewaynone0/10No evidence in the pack mentions data residency, region selection, or data storage location controls for AI Gateway; only logging opt-out toggles for payload/prompt data are documented. This applicable privacy axis lacks any supporting evidence. missing for 10: region/residency selection settings, data localization guarantees, documentation on where logs/cache/metadata are stored.
ai-native userPrevent my data from being used to train AI models
weight 3 · round drawnPortkeynone0/10Evidence pack covers gateway routing, caching, retries, logging, budgeting, etc., but contains no mention of a data-training opt-out, zero-retention policy, or any privacy control preventing use of data for AI model training.
Cloudflare AI Gatewaynone0/10The evidence shows AI Gateway can control its own logging/storage of prompts (e.g., disabling payload logging) but nothing indicates it can prevent downstream AI providers from using submitted data to train their models — that policy is controlled by the provider (OpenAI, Anthropic, etc.), not by the gateway. No documentation ties Cloudflare's logging controls to opting out of provider-side training use.
- [claimed-docs] “This is useful when you want to maintain visibility into usage metrics and request metadata without persisting sensitive prompt or completio…”
- [claimed-docs] “Protect your AI applications from leaking or sending sensitive information.”
- [claimed-docs] “The `cf-aig-collect-log-payload` header allows you to control whether the raw request and response bodies (payloads) are stored for a given …”
ai-native userControl data retention and deletion
weight 2 · round to Cloudflare AI GatewayPortkeynone0/10The evidence pack covers gateway routing, caching, retries, budgets, and logging, but contains no mention of data retention policies, log/data deletion controls, or configurable retention windows.
AI Gateway offers concrete controls relevant to data retention: per-request headers to bypass logging (cf-aig-collect-log) or exclude storing raw prompt/completion payloads (cf-aig-collect-log-payload), letting users limit what sensitive data is persisted. However, there's no documented data deletion mechanism, retention period/TTL settings, or bulk/account-level data purge capability. missing for 10: explicit data retention duration controls, a documented deletion/purge workflow for stored logs, and independent confirmation that opt-out headers fully prevent persistence.
- [claimed-docs] “This is useful when you want to maintain visibility into usage metrics and request metadata without persisting sensitive prompt or completio…”
- [claimed-docs] “The \`cf-aig-collect-log\` header allows you to bypass the default log setting for the gateway.”
- [claimed-docs] “The `cf-aig-collect-log-payload` header allows you to control whether the raw request and response bodies (payloads) are stored for a given …”
- [claimed-docs] “Your AI Gateway dashboard shows logs of individual requests, including the user prompt, model response, provider, timestamp, request status,…”
ai-native userOpt out of telemetry and usage tracking
weight 2 · round to Cloudflare AI GatewayPortkeynone0/10Portkey is essentially a telemetry/observability platform (logs, monitoring of all LLM requests) with no evidence of any opt-out mechanism for its own tracking or usage telemetry; the evidence pack only documents features for logging, caching, retries, and governance, never a privacy/telemetry opt-out control.
AI Gateway exposes per-request headers (cf-aig-collect-log, cf-aig-collect-log-payload) that let a user bypass logging or avoid storing raw prompt/completion payloads, and docs note this is 'useful when you want to maintain visibility into usage metrics... without persisting sensitive prompt or completion data.' However, these are opt-outs for log/payload storage only, not for the underlying usage/metrics tracking (tokens, cost, request counts) that the gateway inherently collects for billing and rate limiting. missing for 10: a global/account-level telemetry opt-out toggle, documentation of disabling usage-metrics tracking entirely, independent confirmation that opting out actually stops Cloudflare-side usage tracking.
- [claimed-docs] “The \`cf-aig-collect-log\` header allows you to bypass the default log setting for the gateway.”
- [claimed-docs] “The `cf-aig-collect-log-payload` header allows you to control whether the raw request and response bodies (payloads) are stored for a given …”
- [claimed-docs] “This is useful when you want to maintain visibility into usage metrics and request metadata without persisting sensitive prompt or completio…”
- [claimed-docs] “Your AI Gateway dashboard shows logs of individual requests, including the user prompt, model response, provider, timestamp, request status,…”
Routing resilience — stories about routing resilience in this arenaRouting resilience
Stories about routing resilience in this arena
Fallbacks
platform engineerConfigure automatic fallback to another model or provider when one fails
weight 3 · round drawnPortkey's docs explicitly describe configuring a prioritized list of providers/models with automatic fallback when the primary fails, customizable trigger status codes, plus complementary retries and load balancing for resilience. missing for 10: independent/hands-on corroboration beyond vendor docs.
- [claimed-docs] “Specify a prioritized list of providers/models. If the primary LLM fails, Portkey automatically falls back to the next in line.”
- [claimed-docs] “By default, fallback triggers on any **non-2xx** status code. Customize with `on_status_codes`”
- [claimed-docs] “Automatically retry failed LLM requests with exponential backoff.”
- [claimed-docs] “Configure per-strategy circuit protection and failure handling”
Cloudflare AI Gateway explicitly documents automatic fallback to another model or provider on request errors or timeouts via the Universal endpoint, with clear examples (e.g., Workers AI failing over to OpenAI) and a response header (cf-aig-step) to identify which step succeeded. This is a well-documented, first-party feature with configuration details and dynamic routing support for fallbacks. Missing for 10: independent/hands-on verification beyond vendor docs, and no evidence of fallback behavior nuances (e.g., latency impact, partial failure handling) from real-world usage.
- [claimed-docs] “Improve resilience by defining request retry and model fallbacks in case of an error.”
- [claimed-docs] “Sends a request to Workers AI Inference API. 2. If that request fails, proceeds to OpenAI.”
- [claimed-docs] “Cloudflare can trigger your fallback provider in response to request errors or predetermined request timeouts.”
- [claimed-docs] “Specify model or provider fallbacks with your Universal endpoint to handle request failures and ensure reliability.”
- [claimed-docs] “the response header `cf-aig-step` indicates which model successfully processed the request by returning the step number.”
Load balancing
platform engineerLoad-balance traffic across providers, deployments, or keys by weight, latency, or cost
weight 2 · round to PortkeyPortkey's docs explicitly document load balancing to distribute traffic across multiple LLMs/providers by weight, plus gradual migration (percentage-based rollout) and fallback on failure/latency-driven retries, all configurable via Configs. This directly matches weight-based and failure/latency-based routing across providers/deployments/keys, backed by first-party docs on load-balancing, fallbacks, and virtual keys. Missing for 10: explicit cost-based routing strategy documentation and independent/hands-on verification beyond vendor docs.
- [claimed-docs] “Distribute traffic across multiple LLMs to prevent any single provider from becoming a bottleneck.”
- [claimed-docs] “**Gradual Migration** | Test new models with small percentage before full rollout”
- [claimed-docs] “**Gradual Migration**|Test new models with small percentage before full rollout”
- [claimed-docs] “Specify a prioritized list of providers/models. If the primary LLM fails, Portkey automatically falls back to the next in line.”
- [claimed-docs] “By default, fallback triggers on any **non-2xx** status code. Customize with `on_status_codes`”
- [claimed-docs] “The core concept remains the same - one Portkey API key gives you access to multiple providers and models.”
- [claimed-docs] “Configure per-strategy circuit protection and failure handling”
AI Gateway supports dynamic routing (conditions, quotas, A/B/gradual rollouts, segment-based routing) and fallback chains across providers/models with sequential retry-on-error, but the evidence describes rule-based/conditional routing and ordered fallback sequences rather than true weighted, latency-based, or cost-based load balancing across providers/keys. missing for 10: explicit weighted traffic splitting, latency-based routing decisions, cost-based routing logic, and any independent/hands-on verification of these mechanisms.
- [claimed-docs] “Improve resilience by defining request retry and model fallbacks in case of an error.”
- [claimed-docs] “Sends a request to Workers AI Inference API. 2. If that request fails, proceeds to OpenAI.”
- [claimed-docs] “Dynamic routing enables you to create request routing flows through a **visual interface** or a **JSON-based configuration**.”
- [claimed-docs] “Directing different segments (paid/not-paid user) to different models”
- [claimed-docs] “Restricting each user/project/team with budget/rate limits”
- [claimed-docs] “Cloudflare can trigger your fallback provider in response to request errors or predetermined request timeouts.”
- [claimed-docs] “A/B and gradual rollouts while making it accessible to both developers and non-technical team members.”
- [claimed-docs] “you compose a small flow that evaluates conditions, enforces quotas, and chooses models with fallbacks. You can iterate without touching app…”
- [claimed-docs] “Specify model or provider fallbacks with your Universal endpoint to handle request failures and ensure reliability.”
Policy routing
ai-native userMy agent can switch models mid-task by policy — cost, capability, or availability — through gateway routing rules
weight 3 · round drawnPortkey's gateway supports policy-based model switching via conditional fallbacks (status-code triggers), load balancing/weighted routing, gradual migration and canary rollout, plus budget/rate-limit enforcement that can trigger automatic switching for cost/availability reasons - all configurable via routing configs applied at request time. This directly matches mid-task, policy-driven model switching for cost, capability, or availability. Missing for 10: independent/hands-on corroboration of dynamic mid-task capability-based switching (vs. failure-triggered fallback), and explicit example of capability-based routing rules (e.g., routing by task type).
- [claimed-docs] “Specify a prioritized list of providers/models. If the primary LLM fails, Portkey automatically falls back to the next in line.”
- [claimed-docs] “By default, fallback triggers on any **non-2xx** status code. Customize with `on_status_codes`”
- [claimed-docs] “Distribute traffic across multiple LLMs to prevent any single provider from becoming a bottleneck.”
- [claimed-docs] “**Gradual Migration** | Test new models with small percentage before full rollout”
- [claimed-docs] “Canary test new models in production”
- [claimed-docs] “Configure budget and rate limits on API keys to effectively manage AI spending and usage across your organization”
- [claimed-docs] “Budget limits allow you to set maximum LLM spending or token usage thresholds on your API keys, automatically preventing further usage when …”
- [claimed-docs] “Set hourly, daily, or per minute rate limits on requests or tokens sent”
- [claimed-docs] “Configure per-strategy circuit protection and failure handling”
Cloudflare AI Gateway explicitly supports policy-driven model switching via Dynamic Routing (visual/JSON config for segment-based routing, budget/rate limits, A/B rollouts) and Fallbacks (automatic switch to another model/provider on error or timeout, with cf-aig-step header indicating which model handled the request). These directly satisfy switching by cost, capability, or availability without app code changes. missing for 10: independent/hands-on evidence of this working in production agent workflows, and explicit documentation of capability-based (not just error/availability-based) routing logic.
- [claimed-docs] “Improve resilience by defining request retry and model fallbacks in case of an error.”
- [claimed-docs] “Sends a request to Workers AI Inference API. 2. If that request fails, proceeds to OpenAI.”
- [claimed-docs] “Dynamic routing enables you to create request routing flows through a **visual interface** or a **JSON-based configuration**.”
- [claimed-docs] “Directing different segments (paid/not-paid user) to different models”
- [claimed-docs] “Restricting each user/project/team with budget/rate limits”
- [claimed-docs] “the response header `cf-aig-step` indicates which model successfully processed the request by returning the step number.”
- [claimed-docs] “Cloudflare can trigger your fallback provider in response to request errors or predetermined request timeouts.”
- [claimed-docs] “A/B and gradual rollouts while making it accessible to both developers and non-technical team members.”
- [claimed-docs] “you compose a small flow that evaluates conditions, enforces quotas, and chooses models with fallbacks. You can iterate without touching app…”
- [claimed-docs] “Specify model or provider fallbacks with your Universal endpoint to handle request failures and ensure reliability.”
Rate limits
platform engineerSmooth provider rate limits by spreading traffic across keys and queuing or throttling requests
weight 2 · round to PortkeyPortkey documents load balancing across multiple provider keys/models to avoid bottlenecks, automatic retries with exponential backoff or provider retry-after headers, and configurable per-key/org rate limits (hourly/daily/per-minute) alongside budget limits — directly addressing spreading traffic and throttling/queuing around rate limits. Multiple virtual keys per provider further support distributing load to smooth rate limits. Missing for 10: explicit first-party or independent evidence of a request queue (vs. retry/backoff) and hands-on/independent validation of these features working under real rate-limit pressure.
- [claimed-docs] “Distribute traffic across multiple LLMs to prevent any single provider from becoming a bottleneck.”
- [claimed-docs] “Automatically retry failed LLM requests with exponential backoff.”
- [claimed-docs] “Enable `use_retry_after_headers` to use the provider’s `retry-after-ms`, `x-ms-retry-after-ms`, or `retry-after` headers instead of exponent…”
- [claimed-docs] “Enable `use_retry_after_headers` to use the provider's `retry-after-ms`, `x-ms-retry-after-ms`, or `retry-after` headers instead of exponent…”
- [claimed-docs] “Set hourly, daily, or per minute rate limits on requests or tokens sent”
- [claimed-docs] “Configure budget and rate limits on API keys to effectively manage AI spending and usage across your organization”
- [claimed-docs] “The core concept remains the same - one Portkey API key gives you access to multiple providers and models.”
- [claimed-docs] “Fine-grained control: budgets, rate limits, and model allow-lists at both org and workspace level.”
AI Gateway provides rate limiting (fixed/sliding, request-per-time-window) and fallback/retry to alternate providers/models, which helps smooth provider limits and errors, and dynamic routing can enforce per-user/team quotas. However, there is no explicit documentation of key rotation/spreading traffic across multiple API keys for the same provider, nor of a request queuing mechanism (rate limiting appears to reject/throttle rather than queue). missing for 10: explicit multi-key load balancing/rotation across provider keys, documented request queuing behavior (vs. simple throttling/rejection), independent evidence of this working at scale.
- [claimed-docs] “Rate limiting controls the traffic that reaches your application, which prevents expensive bills and suspicious activity.”
- [claimed-docs] “You can also select if you would like a **fixed** or **sliding** rate limiting technique.”
- [claimed-docs] “You can define rate limits as the number of requests that get sent in a specific time frame. For example, you can limit your application to …”
- [claimed-docs] “Control how your application scales by limiting the number of requests your application receives.”
- [claimed-docs] “Improve resilience by defining request retry and model fallbacks in case of an error.”
- [claimed-docs] “Sends a request to Workers AI Inference API. 2. If that request fails, proceeds to OpenAI.”
- [claimed-docs] “Cloudflare can trigger your fallback provider in response to request errors or predetermined request timeouts.”
- [claimed-docs] “Specify model or provider fallbacks with your Universal endpoint to handle request failures and ensure reliability.”
- [claimed-docs] “Restricting each user/project/team with budget/rate limits”
- [claimed-docs] “you compose a small flow that evaluates conditions, enforces quotas, and chooses models with fallbacks. You can iterate without touching app…”
Retries
platform engineerSet automatic retry policies for transient provider errors
weight 2 · round drawnPortkey's docs explicitly describe automatic retries with exponential backoff and support for provider retry-after headers, directly matching the story of configuring retry policies for transient errors. missing for 10: independent/hands-on corroboration beyond vendor docs, and configuration details on per-error-code or max-retry-count customization.
- [claimed-docs] “Automatically retry failed LLM requests with exponential backoff.”
- [claimed-docs] “Enable `use_retry_after_headers` to use the provider’s `retry-after-ms`, `x-ms-retry-after-ms`, or `retry-after` headers instead of exponent…”
- [claimed-docs] “Enable `use_retry_after_headers` to use the provider's `retry-after-ms`, `x-ms-retry-after-ms`, or `retry-after` headers instead of exponent…”
Cloudflare AI Gateway explicitly supports request retries and fallbacks triggered by errors or timeouts (docs-7, docs-20, docs-33), with response headers indicating which step/model succeeded (docs-17). This directly satisfies automatic retry policies for transient provider errors. missing for 10: no evidence of configurable retry counts/backoff intervals specifically for 'retry' (vs fallback) and no independent/hands-on corroboration beyond first-party docs.
- [claimed-docs] “Improve resilience by defining request retry and model fallbacks in case of an error.”
- [claimed-docs] “Sends a request to Workers AI Inference API. 2. If that request fails, proceeds to OpenAI.”
- [claimed-docs] “the response header `cf-aig-step` indicates which model successfully processed the request by returning the step number.”
- [claimed-docs] “Cloudflare can trigger your fallback provider in response to request errors or predetermined request timeouts.”
- [claimed-docs] “Specify model or provider fallbacks with your Universal endpoint to handle request failures and ensure reliability.”
Streaming tools — stories about streaming tools in this arenaStreaming tools
Stories about streaming tools in this arena
Streaming
developerStream token-by-token responses through the gateway from any provider
weight 3 · round drawnPortkeynone0/10The evidence describes Portkey's gateway, fallbacks, retries, load balancing, caching, and OpenAI-compatible base URL integration, but none of the provided docs mention streaming or token-by-token response support. Streaming is a fair and expected capability for an LLM gateway product, so its absence in the evidence pack means this axis applies but is unproven.
- [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey’s features into your existing setup.”
- [claimed-docs] “The base URL for all REST API requests is `https://api.portkey.ai/v1`”
- [claimed-docs] “Ideal for: if you’re currently utilizing OpenAI’s Python or Node.js SDKs. By changing the base URL and adding Portkey-specific headers, you …”
- [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey's features into your existing setup.”
Cloudflare AI Gatewaynone0/10The evidence describes AI Gateway as a proxy supporting multiple providers, an OpenAI-compatible /chat/completions endpoint, and native API pass-through, but nowhere explicitly confirms token-by-token streaming support or documents how streaming responses are handled through the gateway. missing for 10: explicit documentation or example of streaming/SSE support, confirmation that streaming works uniformly across all connected providers.
- [claimed-docs] “Cloudflare's AI Gateway offers an OpenAI-compatible `/chat/completions` endpoint, enabling integration with multiple AI providers using a si…”
- [claimed-docs] “Connect to AI providers using their native API formats through AI Gateway.”
- [claimed-docs] “This feature simplifies the integration process, allowing for seamless switching between different models without significant code modificat…”
Tool calling
developerRequest structured JSON-schema outputs across providers
weight 1 · round drawnPortkeynone0/10The evidence pack covers gateway features like fallbacks, retries, load balancing, caching, virtual keys, and logging, but contains no mention of structured JSON-schema output support, response_format normalization, or schema validation across providers. Since Portkey acts as a universal AI gateway, this capability would be plausible, but no documentation cites it.
Cloudflare AI Gatewaynone0/10Cloudflare AI Gateway is a proxy/observability layer that passes requests through to native or OpenAI-compatible provider APIs; the evidence describes routing, caching, fallbacks, logging, and rate limiting, but nothing about enforcing or normalizing structured JSON-schema output across providers. Missing for 10: any mention of JSON-schema/response_format enforcement, schema validation, or cross-provider structured-output normalization.
- [claimed-docs] “Cloudflare's AI Gateway offers an OpenAI-compatible `/chat/completions` endpoint, enabling integration with multiple AI providers using a si…”
- [claimed-docs] “Switch providers by changing the \`model\` and \`apiKey\` parameters.”
- [claimed-docs] “Connect to AI providers using their native API formats through AI Gateway.”
developerMake tool and function calls across different providers with a consistent schema
weight 3 · round drawnPortkey's gateway offers a unified OpenAI-SDK-compatible API (base URL + headers) across 250+ models, which implies a consistent request format including function/tool call fields, and it supports connecting external tools via remote MCP servers. However, there is no explicit documentation describing how tool/function-calling schemas are normalized or translated across different providers' native formats. Missing for 10: explicit docs on function/tool-call schema normalization, examples of tool-calling requests across multiple providers, and independent verification that tool calls work consistently across providers.
- [claimed-docs] “It serves as a unified interface for interacting with over 250 AI models, offering advanced tools for control, visibility, and security in y…”
- [claimed-docs] “Ideal for: if you’re currently utilizing OpenAI’s Python or Node.js SDKs. By changing the base URL and adding Portkey-specific headers, you …”
- [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey’s features into your existing setup.”
- [claimed-docs] “Connect to Remote MCP severs, allowing you to connect external tools and data sources.”
AI Gateway provides an OpenAI-compatible /chat/completions endpoint and unified switching between providers via model/apiKey parameters, plus native API pass-through, which supports a consistent schema for chat-style requests across providers. However, the evidence never explicitly discusses tool/function-calling schema normalization (e.g., translating tool_call formats between Anthropic, Gemini, OpenAI, etc.) — it only covers general chat completion routing, caching, logging, and fallback headers. missing for 10: explicit documentation or examples of function/tool-call schema normalization across providers, evidence of consistent tool-call response parsing, and independent/hands-on confirmation that function calling works uniformly across all supported providers.
- [claimed-docs] “This feature simplifies the integration process, allowing for seamless switching between different models without significant code modificat…”
- [claimed-docs] “Cloudflare's AI Gateway offers an OpenAI-compatible `/chat/completions` endpoint, enabling integration with multiple AI providers using a si…”
- [claimed-docs] “Switch providers by changing the \`model\` and \`apiKey\` parameters.”
- [claimed-docs] “Connect to AI providers using their native API formats through AI Gateway.”
- [claimed-docs] “Workers AI, Anthropic, Google Gemini, OpenAI, Replicate, and more work with AI Gateway.”
Unified api — stories about unified api in this arenaUnified api
Stories about unified api in this arena
Catalog
developerBrowse or query a catalog of available models with pricing and context-window metadata
weight 2 · round to PortkeyPortkey documents a 'Model Catalog' providing a single pane to view/manage all providers and models with discovery and governance controls, and claims access to 250+ models, but no evidence shows the catalog surfaces pricing or context-window metadata for models. missing for 10: explicit documentation of per-model pricing info, explicit documentation of context-window size metadata, and any UI/query example showing these fields.
- [claimed-docs] “A single pane to view and manage every AI provider and model in your organization. It provides centralized governance, discovery, and usage …”
- [claimed-docs] “It serves as a unified interface for interacting with over 250 AI models, offering advanced tools for control, visibility, and security in y…”
- [claimed-docs] “Add a provider, then use `@provider-slug/model-name` in your code.”
- [claimed-docs] “Fine-grained control: budgets, rate limits, and model allow-lists at both org and workspace level.”
Cloudflare AI Gatewaynone0/10The evidence pack shows AI Gateway supports multiple providers, per-token pricing pass-through, logging, caching, and routing, but there is no mention of a browsable/queryable catalog of models with pricing and context-window metadata—developers must know provider/model names themselves and check pricing docs separately (docs-16 only states pricing pass-through policy, not a queryable catalog).
- [claimed-docs] “Workers AI, Anthropic, Google Gemini, OpenAI, Replicate, and more work with AI Gateway.”
- [claimed-docs] “Inference pricing from providers is passed through with no markup — you pay the same per-token rates as you would directly with the provider…”
Compatibility
developerPoint existing OpenAI-compatible code at the gateway by changing only the base URL and key
weight 3 · round to PortkeyDocs explicitly state that developers using OpenAI's Python/Node SDKs can integrate Portkey by only changing the base URL and adding Portkey-specific headers, with the base URL documented as https://api.portkey.ai/v1. This is a first-party documented, purpose-built migration path. Missing for 10: independent/hands-on third-party confirmation beyond vendor docs.
- [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey’s features into your existing setup.”
- [claimed-docs] “The base URL for all REST API requests is `https://api.portkey.ai/v1`”
- [claimed-docs] “Ideal for: if you’re currently utilizing OpenAI’s Python or Node.js SDKs. By changing the base URL and adding Portkey-specific headers, you …”
- [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey's features into your existing setup.”
Docs confirm an OpenAI-compatible /chat/completions endpoint that allows switching providers by only changing model/apiKey parameters against a single URL, directly matching the story. missing for 10: independent/hands-on developer confirmation outside Cloudflare's own docs, and explicit example showing literal drop-in replacement of OpenAI SDK base_url with no other code changes.
- [claimed-docs] “Cloudflare's AI Gateway offers an OpenAI-compatible `/chat/completions` endpoint, enabling integration with multiple AI providers using a si…”
- [claimed-docs] “Switch providers by changing the \`model\` and \`apiKey\` parameters.”
- [claimed-docs] “This feature simplifies the integration process, allowing for seamless switching between different models without significant code modificat…”
- [claimed-docs] “Manage all your costs with one simple bill and access every provider through a single API.”
One endpoint
developerCall many model providers through one consistent API
weight 3 · round drawnPortkey documents a unified API/gateway (single base URL, single API key) that supports 250+ providers/models via consistent SDK/REST calls, with virtual keys and model catalog abstracting provider differences. Missing for 10: independent third-party benchmark or hands-on confirmation of the claimed 250+ provider coverage beyond vendor docs.
- [claimed-docs] “The core concept remains the same - one Portkey API key gives you access to multiple providers and models.”
- [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey’s features into your existing setup.”
- [claimed-docs] “The base URL for all REST API requests is `https://api.portkey.ai/v1`”
- [claimed-docs] “It serves as a unified interface for interacting with over 250 AI models, offering advanced tools for control, visibility, and security in y…”
- [claimed-docs] “Add a provider, then use `@provider-slug/model-name` in your code.”
- [claimed-docs] “Ideal for: if you’re currently utilizing OpenAI’s Python or Node.js SDKs. By changing the base URL and adding Portkey-specific headers, you …”
Docs clearly show a unified OpenAI-compatible /chat/completions endpoint and a Universal endpoint that route to many providers (Workers AI, Anthropic, Gemini, OpenAI, Replicate, etc.) via consistent APIs, with model/provider switching via simple parameter changes and pass-through pricing. Missing for 10: independent/hands-on developer corroboration beyond first-party docs.
- [claimed-docs] “Cloudflare's AI Gateway offers an OpenAI-compatible `/chat/completions` endpoint, enabling integration with multiple AI providers using a si…”
- [claimed-docs] “Workers AI, Anthropic, Google Gemini, OpenAI, Replicate, and more work with AI Gateway.”
- [claimed-docs] “This feature simplifies the integration process, allowing for seamless switching between different models without significant code modificat…”
- [claimed-docs] “Switch providers by changing the \`model\` and \`apiKey\` parameters.”
- [claimed-docs] “Connect to AI providers using their native API formats through AI Gateway.”
- [claimed-docs] “Manage all your costs with one simple bill and access every provider through a single API.”
Not comparable on these axes
ai-native userPlug MCP servers into this product so it can use their tools
weight 3 · not comparablePortkey's docs explicitly state it can 'Connect to Remote MCP servers, allowing you to connect external tools and data sources' via the AI Gateway, directly supporting the story. However, evidence lacks detail on setup steps, supported MCP transport, or independent/hands-on confirmation of this working in practice. Missing for 10: detailed configuration docs for adding MCP servers, independent verification of MCP tool invocation, and examples of tool-calling via MCP in a live agent workflow.
- [claimed-docs] “Connect to Remote MCP severs, allowing you to connect external tools and data sources.”
Cloudflare AI Gatewayn/aCloudflare AI Gateway is an LLM proxy/observability layer for routing, caching, rate-limiting and logging requests to model providers — it is not an agent that consumes tools, and there is no evidence of MCP server integration in the pack. This axis is a category error for this type of product.
ai-native userGet AI-generated insights and suggestions from my data inside the product
weight 2 · not comparablePortkeynone0/10Portkey's evidence pack covers gateway routing, fallbacks, caching, logging, and budget controls, but there is no mention of AI-generated insights, summaries, or suggestions derived from a user's data within the product itself — logs support manual feedback tagging (portkey-docs-34), not AI-driven analysis. missing for 10: any AI-generated insight/summary feature, anomaly detection or recommendation engine over logs/usage data, in-product AI assistant surfacing suggestions.
- [claimed-docs] “The Logs section presents a chronological list of all the requests processed through Portkey.”
- [claimed-docs] “you can also add manual feedback on the logs to be analysed and filtered later”
- [claimed-docs] “Configure budget and rate limits on API keys to effectively manage AI spending and usage across your organization”
ai-native userSet up automations that run autonomously in the background
weight 2 · not comparablePortkeyn/aPortkey is an AI gateway/observability layer for routing, caching, and governing LLM requests, not a workflow/automation platform for running autonomous background tasks; the evidence covers gateway features (fallbacks, retries, caching, load balancing, logs, budgets) with no scheduler or autonomous-agent execution capability. This story targets a different product category and does not fit Portkey's axis.
Cloudflare AI Gatewayn/aCloudflare AI Gateway is a request-routing/observability proxy layer sitting between an application and LLM providers (caching, rate limiting, fallbacks, logging) — it processes calls reactively as they occur rather than running independent background automations. This axis (autonomous background automation) fits agent/automation platforms, not a gateway product, so it does not apply to this product category.
ai-native userDelegate tasks to a built-in AI assistant inside the product
weight 3 · not comparablePortkeyn/aPortkey is an AI gateway/observability platform for routing and managing LLM traffic, not a product with a built-in chat/assistant UI that a user delegates tasks to; this axis is a category error for its product type.
Cloudflare AI Gatewayn/aCloudflare AI Gateway is an infrastructure/proxy layer for routing, caching, logging and observability of AI API calls — it is not itself an AI assistant product with a built-in agent persona to delegate tasks to. This story applies to conversational/agentic assistant products, not to a gateway/middleware tool, making it a category mismatch.
ai-native userOperate the product with natural-language commands
weight 2 · not comparablePortkeynone0/10Portkey's evidence describes an API/SDK-based AI gateway, dashboard configs, logs, and MCP server connectivity for tools/data sources, but nothing shows an interface where a user issues natural-language commands to operate Portkey itself (e.g., a chat-based admin/control plane). The axis is plausible for a platform like this, but no evidence supports it.
Cloudflare AI Gatewayn/aCloudflare AI Gateway is an infrastructure/proxy layer for routing, caching, logging and rate-limiting LLM API calls, configured via dashboard, JSON, or headers — it is not an interface a user interacts with via natural-language commands. This axis is a category error for a gateway/proxy product, not a missing feature.
ai-native userSchedule recurring jobs or workflows
weight 2 · not comparablePortkeyn/aPortkey is an AI gateway/observability platform for routing, caching, and monitoring LLM requests, not a workflow/job scheduling or orchestration product; nothing in the evidence pack relates to scheduling recurring jobs or workflows, making this a category mismatch rather than a missing feature.
ai-native userSelf-host the core product
weight 3 · not comparableDocs confirm the AI Gateway — described as the 'battle-tested' core routing component — is open-sourced and can be run locally via a single npx command, which is genuine self-hosting evidence. However, other core parts of the product (observability/logs UI, admin budget/rate-limit controls, model catalog governance) are described only as hosted SaaS features with no mention of self-hosting. Missing for 10: evidence that the full platform (observability, admin, model catalog) can be self-hosted, not just the gateway component, and any independent confirmation of self-hosting in practice.
- [claimed-docs] “npx @portkey-ai/gateway”
- [claimed-docs] “We've open sourced our battle-tested AI gateway to the community. You can run it locally with a single command:”
- [claimed-docs] “A single pane to view and manage every AI provider and model in your organization. It provides centralized governance, discovery, and usage …”
- [claimed-docs] “Configure budget and rate limits on API keys to effectively manage AI spending and usage across your organization”
Cloudflare AI Gatewayn/aCloudflare AI Gateway is a managed, hosted cloud service built into Cloudflare's edge network; there is no evidence of, or plausible path to, self-hosting the core product outside Cloudflare's infrastructure. Self-hosting is a category error for this SaaS/edge-service product type.