OpenRouter vs Portkey

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to OpenRouter
   OpenRouter✓fullprobed9/10

   OpenRouter serves a working llms.txt at openrouter.ai/llms.txt (HTTP 200, agent-oriented summary), plus an OpenAPI spec and MCP server that expose docs/data programmatically to agents. This directly satisfies pointing an agent at llms.txt or agent-oriented docs. Missing for 10: independent/community confirmation that agents successfully consume llms.txt in practice.

   • [probe] “PROBE llms.txt: HTTP 200 at https://openrouter.ai/llms.txt # OpenRouter > A unified API for every major LLM. One endpoint, hundreds of mode…”
   • [probe] “PROBE openapi: HTTP 200 at https://openrouter.ai/openapi.json — contains "openapi" key”
   • [probe] “official MCP server documented at https://openrouter.ai/docs/guides/overview/mcp-server.md”
   • [claimed-docs] “The OpenRouter MCP server plugs OpenRouter into the AI tools you already use. Once connected, your assistant can pull live OpenRouter data (…”

   proof ↗⚑ flag

   Portkey✓fullprobed8/10

   Portkey hosts a working llms.txt file (HTTP 200) with a description of the product, confirming agents can be pointed at it for agent-oriented docs. Missing for 10: no docs.md fallback (404) or OpenAPI spec confirmation, and no independent/community corroboration of agents actually using it.

   • [probe] “PROBE llms.txt: HTTP 200 at https://portkey.ai/llms.txt # Portkey > Portkey is a production stack for Gen AI builders, equipping AI teams w…”
   • [probe] “PROBE docs-md: HTTP 404 at https://portkey.ai/docs.md”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to OpenRouter
   OpenRouter✓fullprobed7/10

   OpenRouter is fundamentally a headless REST API (single endpoint, OpenAI-SDK compatible, streaming, programmatic API key management) that requires no UI and can be called from any script or pipeline, making it inherently suited to CI/automation use; the keyless models endpoint and management API keys reinforce programmatic, non-interactive use. Missing for 10: no explicit CI/CD pipeline examples (e.g., GitHub Actions), no dedicated 'headless/automation' documentation section, and no independent report of someone running it in a CI pipeline.

   • [claimed-docs] “OpenRouter provides endpoints to programmatically manage your API keys, enabling key creation and management for applications that need to d…”
   • [claimed-docs] “You can also use the OpenAI SDK pointed at OpenRouter as a drop-in replacement.”
   • [claimed-docs] “You can also use the [OpenAI SDK](#using-the-openai-sdk) pointed at OpenRouter as a drop-in replacement.”
   • [claimed-docs] “The OpenRouter API allows streaming responses from any model. This is useful for building chat interfaces or other applications where the UI…”
   • [probe] “PROBE keyless-endpoint: GET https://openrouter.ai/api/v1/models returned HTTP 200 application/json with 426 models (unauthenticated) - live …”
   • [probe] “PROBE openapi: HTTP 200 at https://openrouter.ai/openapi.json — contains "openapi" key”

   proof ↗⚑ flag

   Portkey~partialclaimed5/10

   Portkey's gateway can be run headlessly via a CLI command (npx @portkey-ai/gateway) and REST API with a base URL, and it is open source and self-hostable, which supports CI/automation usage. However, there is no explicit documentation of CI pipeline integration, GitHub Actions examples, or headless automation testing workflows. missing for 10: explicit CI/CD examples, headless automation guides, non-interactive scripted usage documentation, independent verification of CI usage.

   • [claimed-docs] “npx @portkey-ai/gateway”
   • [claimed-docs] “We've open sourced our battle-tested AI gateway to the community. You can run it locally with a single command:”
   • [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey’s features into your existing setup.”
   • [claimed-docs] “The base URL for all REST API requests is `https://api.portkey.ai/v1`”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round to Portkey
   OpenRouter—none0/10

   Evidence only shows OpenRouter publishing its own MCP server so external AI tools/editors can pull OpenRouter data (docs-14, docs-26, docs-34, probe-3) — this is OpenRouter acting as an MCP server for other clients, not OpenRouter itself consuming/plugging in external MCP servers to gain their tools. There is no evidence that OpenRouter's API or platform lets a user attach third-party MCP servers so OpenRouter-routed models can call their tools; the standardized tool-calling docs (docs-5) refer to generic function-calling, not MCP integration specifically.

   • [claimed-docs] “The OpenRouter MCP server plugs OpenRouter into the AI tools you already use. Once connected, your assistant can pull live OpenRouter data (…”
   • [claimed-docs] “The **OpenRouter MCP server** plugs OpenRouter into the AI tools you already use. Once connected, your assistant can pull live OpenRouter da…”
   • [claimed-docs] “Once connected, your assistant can pull live OpenRouter data (models, prices, your credits, rankings, and docs) and send quick test messages…”
   • [probe] “official MCP server documented at https://openrouter.ai/docs/guides/overview/mcp-server.md”
   • [claimed-docs] “OpenRouter standardizes the tool calling interface across models and providers, making it easy to integrate external tools with any supporte…”

   proof ↗⚑ flag

   Portkey~partialclaimed6/10

   Portkey's docs explicitly state it can 'Connect to Remote MCP servers, allowing you to connect external tools and data sources' via the AI Gateway, directly supporting the story. However, evidence lacks detail on setup steps, supported MCP transport, or independent/hands-on confirmation of this working in practice. Missing for 10: detailed configuration docs for adding MCP servers, independent verification of MCP tool invocation, and examples of tool-calling via MCP in a live agent workflow.

   • [claimed-docs] “Connect to Remote MCP severs, allowing you to connect external tools and data sources.”

   proof ↗⚑ flag

4. ai-native userConnect an agent via an official MCP server

   weight 3 · round to OpenRouter
   OpenRouter✓fullprobed8/10

   OpenRouter publishes an official MCP server that lets AI assistants/agents connect and pull live OpenRouter data (models, prices, credits, rankings, docs) directly from within the editor, confirmed both in docs and via a live probe of the documentation page. Missing for 10: independent/hands-on community validation of the MCP server's real-world reliability beyond first-party docs.

   • [claimed-docs] “The OpenRouter MCP server plugs OpenRouter into the AI tools you already use. Once connected, your assistant can pull live OpenRouter data (…”
   • [claimed-docs] “The **OpenRouter MCP server** plugs OpenRouter into the AI tools you already use. Once connected, your assistant can pull live OpenRouter da…”
   • [claimed-docs] “Once connected, your assistant can pull live OpenRouter data (models, prices, your credits, rankings, and docs) and send quick test messages…”
   • [probe] “official MCP server documented at https://openrouter.ai/docs/guides/overview/mcp-server.md”

   proof ↗⚑ flag

   Portkey—none0/10

   Portkey's docs describe connecting *to* remote MCP servers as a client-side feature (portkey-docs-17), but there is no evidence that Portkey itself exposes an official MCP server endpoint that external agents can connect to. Missing for 10: any documentation of Portkey hosting/serving an MCP endpoint, MCP server setup instructions, or third-party confirmation of agents connecting via Portkey's MCP server.

   • [claimed-docs] “Connect to Remote MCP severs, allowing you to connect external tools and data sources.”

   proof ↗⚑ flag

5. ai-native userUse an official CLI

   weight 2 · round to Portkey
   OpenRouter—none0/10

   OpenRouter is an API/model-routing platform for which an official CLI would be a plausible ecosystem tool, but the evidence pack contains no mention of any official OpenRouter CLI — only SDK compatibility, REST API, MCP server, and web UI are documented.

   ⚑ flag

   Portkey~partialclaimed3/10

   The only CLI-like evidence is running the open-source gateway locally via `npx @portkey-ai/gateway`, which self-hosts the gateway rather than offering a full-featured CLI for managing configs, logs, or other Portkey resources. Missing for 10: a documented dedicated CLI tool/command set, CLI-based workflow examples, and independent corroboration of CLI usage.

   • [claimed-docs] “npx @portkey-ai/gateway”
   • [claimed-docs] “We've open sourced our battle-tested AI gateway to the community. You can run it locally with a single command:”

   proof ↗⚑ flag

6. ai-native userDrive the product through a documented public API

   weight 3 · round to OpenRouter
   OpenRouter✓fullprobed10/10

   OpenRouter is fundamentally an API product: extensive first-party docs cover the unified chat completions endpoint, streaming, tool calling, model routing/fallbacks, key management, and an OpenAPI spec plus a live, keyless /api/v1/models endpoint confirm a documented, programmatically accessible public API. Community evidence corroborates real-world programmatic use (switching models, key minting, billing caps) reinforcing that it is driven via API in production.

   • [claimed-docs] “OpenRouter gives you access to hundreds of AI models through a single API endpoint.”
   • [claimed-docs] “The OpenRouter API allows streaming responses from any model. This is useful for building chat interfaces or other applications where the UI…”
   • [claimed-docs] “OpenRouter standardizes the tool calling interface across models and providers, making it easy to integrate external tools with any supporte…”
   • [claimed-docs] “Browse the full catalog at openrouter.ai/models, or list every available slug programmatically via the GET /api/v1/models endpoint.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://openrouter.ai/llms.txt # OpenRouter > A unified API for every major LLM. One endpoint, hundreds of mode…”
   • [probe] “PROBE openapi: HTTP 200 at https://openrouter.ai/openapi.json — contains "openapi" key”
   • [probe] “PROBE keyless-endpoint: GET https://openrouter.ai/api/v1/models returned HTTP 200 application/json with 426 models (unauthenticated) - live …”
   • [community] “Happy for the OpenRouter team... the ability to run any AI model with the same api key and same request has been great to experiment with ne…”
   • [community] “One thing that OpenRouter makes easy is the ability to manage API keys (mint new ones, expiry/limits per key, etc.) that I wish other provid…”

   proof ↗⚑ flag

   Portkey✓fullprobed8/10

   Portkey exposes a documented REST API (base URL https://api.portkey.ai/v1) with API-reference docs for its inference API, OpenAI-compatible integration, and llms.txt discovery confirms API-first positioning; gateway is also open-sourced and npx-runnable. Missing for 10: a discoverable OpenAPI/swagger spec (probe found all candidate paths 404) and independent/third-party corroboration of API usage.

   • [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey’s features into your existing setup.”
   • [claimed-docs] “The base URL for all REST API requests is `https://api.portkey.ai/v1`”
   • [claimed-docs] “Ideal for: if you’re currently utilizing OpenAI’s Python or Node.js SDKs. By changing the base URL and adding Portkey-specific headers, you …”
   • [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey's features into your existing setup.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://portkey.ai/llms.txt # Portkey > Portkey is a production stack for Gen AI builders, equipping AI teams w…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://portkey.ai/openapi.json, https://portkey.ai/swagger.json, https://portkey.ai/api/openapi.jso…”
   • [claimed-docs] “We've open sourced our battle-tested AI gateway to the community. You can run it locally with a single command:”

   proof ↗⚑ flag

7. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round drawn
   OpenRouter~partialcommunity7/10

   OpenRouter offers a documented Management API for creating/rotating API keys, workspace spend budgets, and per-key limits/expiry, which supports issuing distinct, capped credentials for agents (openrouter-docs-8, openrouter-docs-10, openrouter-comm-12, openrouter-comm-11). However, there's no explicit documentation of fine-grained scoping (e.g., restricting a key to specific models, tools, or actions) beyond spend caps, so 'least-privilege' is only partially realized. Missing for 10: explicit per-key model/tool/action scoping, granular permission controls beyond spend limits, and independent hands-on verification of the key-management API's robustness.

   • [claimed-docs] “OpenRouter provides endpoints to programmatically manage your API keys, enabling key creation and management for applications that need to d…”
   • [claimed-docs] “Workspace budgets let you cap how much a workspace can spend on OpenRouter inference. Set a dollar limit on any combination of intervals — d…”
   • [community] “They offer billing caps! Most model providers still don't do this, but if you're going to run anything in public it's very useful to have ha…”
   • [community] “One thing that OpenRouter makes easy is the ability to manage API keys (mint new ones, expiry/limits per key, etc.) that I wish other provid…”

   proof ↗⚑ flag

   Portkey~partialclaimed7/10

   Portkey lets admins issue API keys scoped with budgets, rate limits, and model allow-lists at org/workspace level, and virtual keys keep provider credentials out of code — this maps well to least-privilege credential issuance. However, the docs frame this around human/org governance rather than explicitly per-agent scoped credentials, and there's no independent corroboration of enforcement in practice. Missing for 10: explicit agent-specific credential scoping workflow, independent/hands-on validation of enforcement.

   • [claimed-docs] “Configure budget and rate limits on API keys to effectively manage AI spending and usage across your organization”
   • [claimed-docs] “Budget limits allow you to set maximum LLM spending or token usage thresholds on your API keys, automatically preventing further usage when …”
   • [claimed-docs] “Fine-grained control: budgets, rate limits, and model allow-lists at both org and workspace level.”
   • [claimed-docs] “Set hourly, daily, or per minute rate limits on requests or tokens sent”
   • [claimed-docs] “Provider credentials stored securely, never exposed in code”

   proof ↗⚑ flag

8. ai-native userBuild against official SDKs

   weight 2 · round to Portkey
   OpenRouter~partialprobed5/10

   OpenRouter documents an OpenAI-compatible API and explicitly recommends using the OpenAI SDK as a drop-in replacement, plus offers an OpenAPI spec and request builder for generating client code, which supports building against it programmatically. However, there is no evidence of dedicated first-party OpenRouter SDK packages (e.g., official Python/Node/Go libraries) beyond OpenAI SDK compatibility. Missing for 10: dedicated official multi-language SDKs, independent developer corroboration of SDK usage (only OpenAI-SDK compatibility and generic API docs are cited).

   • [claimed-docs] “You can also use the OpenAI SDK pointed at OpenRouter as a drop-in replacement.”
   • [claimed-docs] “You can also use the [OpenAI SDK](#using-the-openai-sdk) pointed at OpenRouter as a drop-in replacement.”
   • [probe] “PROBE openapi: HTTP 200 at https://openrouter.ai/openapi.json — contains "openapi" key”
   • [claimed-docs] “You can use the interactive Request Builder to generate OpenRouter API requests in the language of your choice.”

   proof ↗⚑ flag

   Portkey~partialprobed6/10

   Portkey documents SDK-style integration (base URL swap for OpenAI Python/Node SDKs, REST API reference, open-source gateway installable via npx) supporting AI-native builders, but the evidence pack lacks explicit mention of dedicated first-party SDKs beyond OpenAI-compatible drop-in usage, and OpenAPI spec probes returned 404s. missing for 10: dedicated official multi-language SDK docs (e.g., Python/JS Portkey SDK reference pages), working OpenAPI/spec discovery, independent developer corroboration of SDK usage.

   • [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey’s features into your existing setup.”
   • [claimed-docs] “The base URL for all REST API requests is `https://api.portkey.ai/v1`”
   • [claimed-docs] “Ideal for: if you’re currently utilizing OpenAI’s Python or Node.js SDKs. By changing the base URL and adding Portkey-specific headers, you …”
   • [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey's features into your existing setup.”
   • [claimed-docs] “We've open sourced our battle-tested AI gateway to the community. You can run it locally with a single command:”
   • [probe] “PROBE openapi: all candidate paths 404 (https://portkey.ai/openapi.json, https://portkey.ai/swagger.json, https://portkey.ai/api/openapi.jso…”

   proof ↗⚑ flag

9. ai-native userSubscribe to events via webhooks

   weight 2 · round drawn
   OpenRouter—none0/10

   No evidence in the pack mentions webhooks or event subscription mechanisms of any kind for OpenRouter; the documented features (routing, caching, key management, MCP server) do not include webhook support.

   ⚑ flag

   Portkey—none0/10

   No evidence in the pack mentions webhooks or event subscriptions of any kind; Portkey's docs cover gateway routing, caching, retries, budgets, and logs but nothing about outbound webhook notifications for events.

   ⚑ flag

Agentic features

1. ai-native userOperate the product with natural-language commands

   weight 2 · round to OpenRouter
   OpenRouter~partialprobed4/10

   OpenRouter is an API/routing platform, not a natural-language interface itself, but it does provide an official MCP server that lets an AI assistant issue natural-language commands to pull model data, pricing, credits, and send test messages without leaving the editor — this is the main evidence of NL-command operability. Beyond MCP, there's no chat-style NL command surface for managing keys, budgets, or routing directly. missing for 10: a native conversational/NL interface for account and routing operations, independent hands-on validation of the MCP server's NL command handling, and evidence of NL commands controlling core features like BYOK, budgets, or provider selection.

   • [claimed-docs] “The OpenRouter MCP server plugs OpenRouter into the AI tools you already use. Once connected, your assistant can pull live OpenRouter data (…”
   • [claimed-docs] “The **OpenRouter MCP server** plugs OpenRouter into the AI tools you already use. Once connected, your assistant can pull live OpenRouter da…”
   • [claimed-docs] “Once connected, your assistant can pull live OpenRouter data (models, prices, your credits, rankings, and docs) and send quick test messages…”
   • [probe] “official MCP server documented at https://openrouter.ai/docs/guides/overview/mcp-server.md”

   proof ↗⚑ flag

   Portkey—none0/10

   Portkey's evidence describes an API/SDK-based AI gateway, dashboard configs, logs, and MCP server connectivity for tools/data sources, but nothing shows an interface where a user issues natural-language commands to operate Portkey itself (e.g., a chat-based admin/control plane). The axis is plausible for a platform like this, but no evidence supports it.

   ⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round to OpenRouter
   OpenRouter✓fullprobed8/10

   OpenRouter's docs explicitly mention an interactive Request Builder for generating API requests in any language, plus a live OpenAPI spec is served and confirmed by probe, and the docs site is extensively organized with runnable-style examples across guides. Missing for 10: X, Y — no independent/hands-on account of actually running the interactive Request Builder or evidence of embedded runnable code sandboxes beyond the Request Builder claim.

   • [claimed-docs] “You can use the interactive Request Builder to generate OpenRouter API requests in the language of your choice.”
   • [probe] “PROBE openapi: HTTP 200 at https://openrouter.ai/openapi.json — contains "openapi" key”
   • [claimed-docs] “Browse the full catalog at openrouter.ai/models, or list every available slug programmatically via the GET /api/v1/models endpoint.”

   proof ↗⚑ flag

   Portkey~partialprobed4/10

   Portkey documents a REST API reference (base URL, headers, SDK compatibility) at /docs/api-reference, but the evidence pack contains no mention of an interactive explorer or runnable code examples, and probes for a discoverable OpenAPI/Swagger spec all returned 404s. Missing for 10: evidence of an interactive 'try it' console, runnable code snippets, and a published OpenAPI/Swagger schema.

   • [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey’s features into your existing setup.”
   • [claimed-docs] “The base URL for all REST API requests is `https://api.portkey.ai/v1`”
   • [claimed-docs] “Ideal for: if you’re currently utilizing OpenAI’s Python or Node.js SDKs. By changing the base URL and adding Portkey-specific headers, you …”
   • [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey's features into your existing setup.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://portkey.ai/openapi.json, https://portkey.ai/swagger.json, https://portkey.ai/api/openapi.jso…”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round to OpenRouter
   OpenRouter✓fullprobed9/10

   A probe confirms https://openrouter.ai/openapi.json returns HTTP 200 with an actual "openapi" key, giving a downloadable machine-readable OpenAPI spec, and docs also mention an interactive Request Builder and llms.txt for AI-native consumption. missing for 10: independent/community confirmation that the spec is actively used or kept in sync with the API by third-party developers.

   • [probe] “PROBE openapi: HTTP 200 at https://openrouter.ai/openapi.json — contains "openapi" key”
   • [probe] “PROBE llms.txt: HTTP 200 at https://openrouter.ai/llms.txt # OpenRouter > A unified API for every major LLM. One endpoint, hundreds of mode…”
   • [claimed-docs] “You can use the interactive Request Builder to generate OpenRouter API requests in the language of your choice.”

   proof ↗⚑ flag

   Portkey—none0/10

   Portkey has an API reference (base URL, inference API docs) but no evidence of a downloadable OpenAPI/Swagger spec; a direct probe for common OpenAPI paths returned 404s, and docs.md also 404s.

   • [probe] “PROBE openapi: all candidate paths 404 (https://portkey.ai/openapi.json, https://portkey.ai/swagger.json, https://portkey.ai/api/openapi.jso…”
   • [probe] “PROBE docs-md: HTTP 404 at https://portkey.ai/docs.md”
   • [claimed-docs] “The base URL for all REST API requests is `https://api.portkey.ai/v1`”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round drawn
   OpenRouter—none0/10

   OpenRouter's evidence describes production routing, billing, BYOK, and caching features but nothing about a sandbox/test environment isolated from production data or billing. The keyless models endpoint (openrouter-probe-4) only exposes a public catalog, not an isolated test mode for requests. No documentation of test/staging keys, sandbox mode, or non-production data isolation exists in the pack.

   ⚑ flag

   Portkey—none0/10

   Portkey's evidence covers gateway routing, fallbacks, caching, logging, and gradual/canary rollout of models in production, but there is no mention of a distinct sandbox/staging environment or test mode that isolates a user from production data. The axis is fair for a gateway/dev-tool product, but no evidence supports it.

   ⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   OpenRouter—none0/10

   The evidence pack covers routing, fallbacks, caching, BYOK, key management, and privacy settings, but nowhere documents API versioning (e.g., v1/v2 endpoints) or a formal deprecation policy/timeline for endpoints or models. The 'latest alias' for models is about model freshness, not API version stability guarantees.

   ⚑ flag

   Portkey—none0/10

   No evidence of API versioning scheme (e.g., v1/v2 in URLs beyond a single 'v1' base path) or any documented deprecation policy/changelog; OpenAPI spec probe also returned 404s. Missing for 10: versioning scheme documentation, explicit deprecation policy, changelog/migration guides, and any independent corroboration.

   • [claimed-docs] “The base URL for all REST API requests is `https://api.portkey.ai/v1`”
   • [probe] “PROBE openapi: all candidate paths 404 (https://portkey.ai/openapi.json, https://portkey.ai/swagger.json, https://portkey.ai/api/openapi.jso…”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round drawn
   OpenRouter—none0/10

   The evidence pack shows OpenRouter provides a single-request API, model routing/fallbacks, and programmatic key management, but there is no documentation of a batch/bulk inference endpoint or any mechanism to submit many items/prompts in one bulk operation. missing for 10: batch API or bulk job submission endpoint, evidence of processing many items in a single call, bulk data/export operations.

   ⚑ flag

   Portkey—none0/10

   Portkey's evidence covers gateway routing, fallbacks, caching, logging, and budget/rate-limit controls, but nothing describes a bulk/batch operation capability (e.g., batch inference calls, bulk editing of configs/keys, or bulk log actions) that lets a user act on many items at once. Missing for 10: any documentation of a batch API, bulk config/key management, or bulk log/annotation actions.

   ⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to Portkey
   OpenRouter—none0/10

   OpenRouter is an LLM routing/API gateway; nothing in the evidence describes user-defined event-driven rules or triggers (e.g., webhooks, if-this-then-that automations) that fire actions automatically. Model fallback and provider routing are static request-time configs, not event-triggered automation rules.

   ⚑ flag

   Portkey~partialclaimed6/10

   Portkey ships several built-in rule/event mechanisms — fallback triggers on non-2xx status codes, automatic retries with backoff, circuit-breaker/failure handling per strategy, and budget/rate-limit thresholds that automatically send notifications or block further usage — which are automatic actions triggered by defined conditions. However, these are fixed, product-defined automations (reliability/spend rules) rather than a general user-authorable rule engine where an AI-native user can define arbitrary custom triggers/actions. Missing for 10: general-purpose custom rule/trigger builder, arbitrary event types beyond status codes/spend thresholds, user-defined action targets (e.g. webhooks, custom scripts) beyond built-in notifications/fallback/retry.

   • [claimed-docs] “Specify a prioritized list of providers/models. If the primary LLM fails, Portkey automatically falls back to the next in line.”
   • [claimed-docs] “By default, fallback triggers on any **non-2xx** status code. Customize with `on_status_codes`”
   • [claimed-docs] “Automatically retry failed LLM requests with exponential backoff.”
   • [claimed-docs] “Configure per-strategy circuit protection and failure handling”
   • [claimed-docs] “When usage reaches this threshold, notifications will be sent to configured recipients”
   • [claimed-docs] “Budget limits allow you to set maximum LLM spending or token usage thresholds on your API keys, automatically preventing further usage when …”
   • [claimed-docs] “Set hourly, daily, or per minute rate limits on requests or tokens sent”

   proof ↗⚑ flag

Caching

1. developerCache responses at the gateway to cut cost and latency on repeated requests

   weight 2 · round to Portkey
   OpenRouter✓fullclaimed7/10

   OpenRouter's docs explicitly describe gateway-level response caching that returns cached results with zero billable usage (cutting cost) and near-instant latency, plus provider sticky routing specifically designed to maximize cache hit rates on subsequent requests. This directly matches the story's ask for gateway-side caching to reduce cost/latency on repeated calls. Missing for 10: independent or hands-on community confirmation that caching actually reduces cost/latency in practice, and more detail on cache TTL/configurability beyond the sticky-routing mechanism.

   • [claimed-docs] “When a cached response is available, OpenRouter returns it immediately from cache with no billing (all billable usage counters are reported …”
   • [claimed-docs] “When a cached response is available, OpenRouter returns it immediately from cache with no billing (all billable usage counters are reported …”
   • [claimed-docs] “OpenRouter uses provider sticky routing to route your subsequent requests to the same provider endpoint after a cached request.”
   • [claimed-docs] “OpenRouter uses **provider sticky routing** to route your subsequent requests to the same provider endpoint after a cached request.”
   • [claimed-docs] “To maximize cache hit rates, OpenRouter uses provider sticky routing to route your subsequent requests to the same provider endpoint after a…”

   proof ↗⚑ flag

   Portkey✓fullclaimed9/10

   Portkey's docs explicitly describe gateway-level caching (simple and semantic) that serves responses up to 20x faster and cheaper, including semantic matching via cosine similarity for near-duplicate queries. This directly matches the story's cost/latency reduction on repeated requests. missing for 10: independent/hands-on benchmark corroboration beyond vendor docs.

   • [claimed-docs] “Cache LLM responses to serve requests up to **20x faster** and cheaper.”
   • [claimed-docs] “Matches requests with **similar meaning** using cosine similarity, not just identical text.”

   proof ↗⚑ flag

Latency

1. platform engineerRun traffic through gateway infrastructure that adds minimal latency overhead to provider calls

   weight 2 · round to OpenRouter
   OpenRouter~partialcommunity5/10

   OpenRouter's docs describe latency-reducing features (response caching bypasses billing and reduces latency, sticky routing to maximize cache hits, load-balanced provider selection, streaming support) that imply a gateway designed for low overhead, but there is no explicit benchmark or first-party claim quantifying the gateway's own latency overhead versus direct provider calls. Community feedback is mixed—some praise ease of switching models and reliability (comm-2), while others report unexplained inconsistency and debugging difficulty (comm-8)—but none of this directly measures gateway latency overhead. Missing for 10: quantified latency-overhead benchmarks (vendor or independent), explicit performance comparison vs. direct provider calls, and hands-on latency testing evidence.

   • [claimed-docs] “When a cached response is available, OpenRouter returns it immediately from cache with no billing (all billable usage counters are reported …”
   • [claimed-docs] “OpenRouter uses provider sticky routing to route your subsequent requests to the same provider endpoint after a cached request.”
   • [claimed-docs] “OpenRouter routes requests to the best available providers for your model. By default, requests are load balanced across the top providers t…”
   • [claimed-docs] “The OpenRouter API allows streaming responses from any model. This is useful for building chat interfaces or other applications where the UI…”
   • [community] “Happy for the OpenRouter team... the ability to run any AI model with the same api key and same request has been great to experiment with ne…”
   • [community] “I had consistent problems (5 months back) with OpenRouter and other providers. And very hard to debug. I used the provider directly (Google)…”

   proof ↗⚑ flag

   Portkey~partialclaimed4/10

   Portkey's docs claim caching can serve responses '20x faster' and market the gateway as 'performant,' but there is no direct benchmark or first-party claim quantifying the gateway's own added latency overhead per provider call, nor independent corroboration of low overhead. Missing for 10: explicit latency/overhead benchmarks for the proxy itself, third-party performance testing, and architecture details (e.g., edge/WASM deployment) that would substantiate minimal added latency beyond the cache-hit speedup claim.

   • [claimed-docs] “Cache LLM responses to serve requests up to **20x faster** and cheaper.”
   • [claimed-docs] “It takes 2 mins to integrate and with that, it starts monitoring all of your LLM requests and makes your app resilient, secure, performant, …”
   • [claimed-docs] “offering advanced tools for control, visibility, and security in your Generative AI apps”
   • [claimed-docs] “We've open sourced our battle-tested AI gateway to the community. You can run it locally with a single command:”

   proof ↗⚑ flag

Agent guardrails

1. ai-native userGive an autonomous agent its own key with budget and rate guardrails so it cannot run away on spend

   weight 2 · round drawn
   OpenRouter✓fullcommunity8/10

   OpenRouter supports programmatic API key management (creation, per-key limits/expiry) and workspace-level budget caps (daily/weekly/monthly/lifetime) that auto-block requests once reached, directly enabling an agent to be given its own scoped key with hard spend guardrails; community confirms billing caps and per-key limits are a valued differentiator. missing for 10: explicit per-key rate-limit (requests/sec) controls beyond spend caps, and independent hands-on verification that runaway agent spend is actually blocked in practice.

   • [claimed-docs] “OpenRouter provides endpoints to programmatically manage your API keys, enabling key creation and management for applications that need to d…”
   • [claimed-docs] “Workspace budgets let you cap how much a workspace can spend on OpenRouter inference. Set a dollar limit on any combination of intervals — d…”
   • [community] “They offer billing caps! Most model providers still don't do this, but if you're going to run anything in public it's very useful to have ha…”
   • [community] “One thing that OpenRouter makes easy is the ability to manage API keys (mint new ones, expiry/limits per key, etc.) that I wish other provid…”

   proof ↗⚑ flag

   Portkey✓fullclaimed8/10

   Portkey docs explicitly support issuing API keys with budget and rate limits (spend thresholds, hourly/daily/per-minute request/token limits) that automatically block further usage when exceeded, directly matching the guardrail need for an autonomous agent's key. Fine-grained control at org/workspace level and per-key configuration is documented. Missing for 10: no explicit mention of a per-agent key issuance workflow or independent/hands-on verification that the cutoff actually stops an agent mid-run.

   • [claimed-docs] “Configure budget and rate limits on API keys to effectively manage AI spending and usage across your organization”
   • [claimed-docs] “Budget limits allow you to set maximum LLM spending or token usage thresholds on your API keys, automatically preventing further usage when …”
   • [claimed-docs] “Set hourly, daily, or per minute rate limits on requests or tokens sent”
   • [claimed-docs] “Fine-grained control: budgets, rate limits, and model allow-lists at both org and workspace level.”
   • [claimed-docs] “**Reset Weekly**: Budget limits reset every Sunday at 12 AM UTC”

   proof ↗⚑ flag

Budgets

1. platform engineerSet hard budgets and spend limits per key, team, or user

   weight 3 · round to Portkey
   OpenRouter~partialcommunity6/10

   OpenRouter's docs clearly show hard, auto-enforced 'workspace budgets' with configurable daily/weekly/monthly/lifetime dollar caps that block requests once reached (openrouter-docs-10), covering team-level budgets. Per-key limits are only supported by an informal community mention of 'expiry/limits per key' (openrouter-comm-12, openrouter-comm-11) rather than first-party documentation, and per-user budget controls are not evidenced at all. Missing for 10: official docs confirming per-key hard spend limits, explicit per-user budget/quota controls, and independent verification of enforcement.

   • [claimed-docs] “Workspace budgets let you cap how much a workspace can spend on OpenRouter inference. Set a dollar limit on any combination of intervals — d…”
   • [claimed-docs] “OpenRouter provides endpoints to programmatically manage your API keys, enabling key creation and management for applications that need to d…”
   • [community] “They offer billing caps! Most model providers still don't do this, but if you're going to run anything in public it's very useful to have ha…”
   • [community] “One thing that OpenRouter makes easy is the ability to manage API keys (mint new ones, expiry/limits per key, etc.) that I wish other provid…”

   proof ↗⚑ flag

   Portkey✓fullclaimed9/10

   Portkey's docs explicitly describe setting budget limits (max spend or token usage) and rate limits on API keys, with org/workspace-level fine-grained control, automatic enforcement when limits are reached, notification thresholds, and weekly resets — directly matching per-key/team/user budget/spend controls. Missing for 10: no independent/hands-on corroboration beyond first-party docs.

   • [claimed-docs] “Configure budget and rate limits on API keys to effectively manage AI spending and usage across your organization”
   • [claimed-docs] “When usage reaches this threshold, notifications will be sent to configured recipients”
   • [claimed-docs] “Budget limits allow you to set maximum LLM spending or token usage thresholds on your API keys, automatically preventing further usage when …”
   • [claimed-docs] “Fine-grained control: budgets, rate limits, and model allow-lists at both org and workspace level.”
   • [claimed-docs] “**Reset Weekly**: Budget limits reset every Sunday at 12 AM UTC”
   • [claimed-docs] “Set hourly, daily, or per minute rate limits on requests or tokens sent”

   proof ↗⚑ flag

Spend tracking

1. platform engineerTrack spend per model, key, team, or user across all providers in one place

   weight 3 · round to Portkey
   OpenRouter~partialcommunity6/10

   OpenRouter provides key management (creation, limits per key), workspace budgets with dollar caps by interval, and BYOK cost control, plus community confirmation of billing caps and per-key limits — but the evidence pack lacks explicit documentation of a unified spend dashboard broken down by model, team, or user across providers in one place. missing for 10: consolidated analytics/dashboard showing spend segmented by model, by team, and by individual user; documentation of team-level spend attribution; independent confirmation of cross-provider spend reporting granularity.

   • [claimed-docs] “OpenRouter provides endpoints to programmatically manage your API keys, enabling key creation and management for applications that need to d…”
   • [claimed-docs] “Workspace budgets let you cap how much a workspace can spend on OpenRouter inference. Set a dollar limit on any combination of intervals — d…”
   • [claimed-docs] “OpenRouter supports both OpenRouter credits and the option to bring your own provider keys (BYOK)... Using provider keys enables direct cont…”
   • [community] “They offer billing caps! Most model providers still don't do this, but if you're going to run anything in public it's very useful to have ha…”
   • [community] “One thing that OpenRouter makes easy is the ability to manage API keys (mint new ones, expiry/limits per key, etc.) that I wish other provid…”

   proof ↗⚑ flag

   Portkey✓fullclaimed7/10

   Portkey's docs describe budget/rate limits and spend caps on API keys, a unified model catalog with org/workspace-level governance and allow-lists, and logs filterable by Config ID for auditing usage, which together give spend visibility across providers/keys/teams. Missing for 10: explicit per-user spend breakdown dashboards and independent/hands-on confirmation of a unified cost dashboard beyond docs claims.

   • [claimed-docs] “Configure budget and rate limits on API keys to effectively manage AI spending and usage across your organization”
   • [claimed-docs] “Budget limits allow you to set maximum LLM spending or token usage thresholds on your API keys, automatically preventing further usage when …”
   • [claimed-docs] “Fine-grained control: budgets, rate limits, and model allow-lists at both org and workspace level.”
   • [claimed-docs] “A single pane to view and manage every AI provider and model in your organization. It provides centralized governance, discovery, and usage …”
   • [claimed-docs] “Filter logs by `Config ID` to see all requests using that config”
   • [claimed-docs] “The core concept remains the same - one Portkey API key gives you access to multiple providers and models.”
   • [claimed-docs] “Set hourly, daily, or per minute rate limits on requests or tokens sent”

   proof ↗⚑ flag

Byok

1. developerBring my own provider API keys and have the gateway use them for my traffic

   weight 2 · round to OpenRouter
   OpenRouter✓fullcommunity9/10

   OpenRouter's docs explicitly describe BYOK (bring your own provider keys), letting developers use their own provider API keys for direct control over rate limits and costs, and this is corroborated by community testimony citing consolidated billing/key management as a core benefit. Missing for 10: independent hands-on verification specifically of BYOK traffic routing (most community evidence discusses OpenRouter credits/key management generally rather than BYOK specifically).

   • [claimed-docs] “OpenRouter supports both OpenRouter credits and the option to bring your own provider keys (BYOK)... Using provider keys enables direct cont…”
   • [claimed-docs] “OpenRouter supports both OpenRouter credits and the option to bring your own provider keys (BYOK). ... Using provider keys enables direct co…”
   • [claimed-docs] “OpenRouter supports both OpenRouter credits and the option to bring your own provider keys (BYOK).”
   • [community] “One thing that OpenRouter makes easy is the ability to manage API keys (mint new ones, expiry/limits per key, etc.) that I wish other provid…”

   proof ↗⚑ flag

   Portkey✓fullclaimed8/10

   Portkey's virtual keys and model catalog explicitly let developers add/store their own provider API keys ('Add a provider, then use @provider-slug/model-name', 'Provider credentials stored securely, never exposed in code'), and the gateway routes traffic through these keys with governance/rate-limit controls layered on top. Missing for 10: independent hands-on verification of the BYO-key flow and no explicit mention of per-key traffic isolation/audit beyond docs claims.

   • [claimed-docs] “The core concept remains the same - one Portkey API key gives you access to multiple providers and models.”
   • [claimed-docs] “Add a provider, then use `@provider-slug/model-name` in your code.”
   • [claimed-docs] “Provider credentials stored securely, never exposed in code”
   • [claimed-docs] “A single pane to view and manage every AI provider and model in your organization. It provides centralized governance, discovery, and usage …”
   • [claimed-docs] “Fine-grained control: budgets, rate limits, and model allow-lists at both org and workspace level.”

   proof ↗⚑ flag

Programmatic admin

1. ai-native userProvision gateways, keys, and budgets programmatically through an admin API

   weight 3 · round to OpenRouter
   OpenRouter~partialcommunity6/10

   OpenRouter documents a management API for provisioning and rotating API keys programmatically (openrouter-docs-8) and community users confirm the ability to mint/limit keys (openrouter-comm-12, openrouter-comm-11). Workspace budgets are documented as a feature to cap spend (openrouter-docs-10), but the evidence doesn't explicitly confirm budgets are settable via the same admin API, and there's no mention of provisioning 'gateways' as a distinct programmable resource. Missing for 10: explicit API endpoint documentation for setting/adjusting workspace budgets programmatically, and any concept of provisioning separate 'gateways' via API.

   • [claimed-docs] “OpenRouter provides endpoints to programmatically manage your API keys, enabling key creation and management for applications that need to d…”
   • [claimed-docs] “Workspace budgets let you cap how much a workspace can spend on OpenRouter inference. Set a dollar limit on any combination of intervals — d…”
   • [community] “They offer billing caps! Most model providers still don't do this, but if you're going to run anything in public it's very useful to have ha…”
   • [community] “One thing that OpenRouter makes easy is the ability to manage API keys (mint new ones, expiry/limits per key, etc.) that I wish other provid…”

   proof ↗⚑ flag

   Portkey—none0/10

   Evidence shows budget/rate-limit and key management as dashboard-configurable features (docs-13, docs-25, docs-35, docs-38) but never describes a programmatic Admin API for provisioning gateways, keys, or budgets; the only API reference documented is the inference API (docs-15, docs-16, docs-36), and probes for an OpenAPI/admin API spec returned 404 (portkey-probe-3, portkey-probe-2).

   • [claimed-docs] “Configure budget and rate limits on API keys to effectively manage AI spending and usage across your organization”
   • [claimed-docs] “Budget limits allow you to set maximum LLM spending or token usage thresholds on your API keys, automatically preventing further usage when …”
   • [claimed-docs] “**Reset Weekly**: Budget limits reset every Sunday at 12 AM UTC”
   • [claimed-docs] “The base URL for all REST API requests is `https://api.portkey.ai/v1`”
   • [probe] “PROBE openapi: all candidate paths 404 (https://portkey.ai/openapi.json, https://portkey.ai/swagger.json, https://portkey.ai/api/openapi.jso…”
   • [probe] “PROBE docs-md: HTTP 404 at https://portkey.ai/docs.md”

   proof ↗⚑ flag

Virtual keys

1. platform engineerMint gateway-managed keys for teams and apps without exposing raw provider keys

   weight 3 · round drawn
   OpenRouter✓fullcommunity8/10

   OpenRouter's docs explicitly describe programmatic key management endpoints for creating/managing/rotating API keys for apps without exposing provider keys (docs-8), backed by BYOK for provider key control (docs-9/24/32) and workspace budgets to cap spend per workspace/team (docs-10). Community corroborates ease of minting and limiting keys (comm-12, comm-11). Missing for 10: independent verification of team/app-scoped key isolation guarantees and no first-hand platform-engineer account of using the management API at scale.

   • [claimed-docs] “OpenRouter provides endpoints to programmatically manage your API keys, enabling key creation and management for applications that need to d…”
   • [claimed-docs] “OpenRouter supports both OpenRouter credits and the option to bring your own provider keys (BYOK)... Using provider keys enables direct cont…”
   • [claimed-docs] “Workspace budgets let you cap how much a workspace can spend on OpenRouter inference. Set a dollar limit on any combination of intervals — d…”
   • [community] “One thing that OpenRouter makes easy is the ability to manage API keys (mint new ones, expiry/limits per key, etc.) that I wish other provid…”
   • [community] “They offer billing caps! Most model providers still don't do this, but if you're going to run anything in public it's very useful to have ha…”

   proof ↗⚑ flag

   Portkey✓fullclaimed8/10

   Portkey's virtual keys/model catalog explicitly let platform engineers mint provider-agnostic API keys that map to real provider credentials, which are stored securely and never exposed in code, while budgets, rate limits, and model allow-lists can be enforced per key at org/workspace level for teams and apps. This directly matches minting gateway-managed keys without exposing raw provider keys. Missing for 10: independent/hands-on verification beyond vendor docs, and detail on per-team/app key issuance workflow (e.g., self-service or RBAC-scoped key creation) rather than admin-level config.

   • [claimed-docs] “The core concept remains the same - one Portkey API key gives you access to multiple providers and models.”
   • [claimed-docs] “Provider credentials stored securely, never exposed in code”
   • [claimed-docs] “Configure budget and rate limits on API keys to effectively manage AI spending and usage across your organization”
   • [claimed-docs] “Budget limits allow you to set maximum LLM spending or token usage thresholds on your API keys, automatically preventing further usage when …”
   • [claimed-docs] “Fine-grained control: budgets, rate limits, and model allow-lists at both org and workspace level.”
   • [claimed-docs] “Set hourly, daily, or per minute rate limits on requests or tokens sent”
   • [claimed-docs] “Add a provider, then use `@provider-slug/model-name` in your code.”

   proof ↗⚑ flag

Integrations

1. developerExport gateway logs and traces to my own observability stack

   weight 1 · round drawn
   OpenRouter—none0/10

   Evidence pack shows OpenRouter's API gateway features (routing, fallbacks, caching, key management, privacy settings) but contains no mention of exporting logs/traces, webhooks, log streaming, or integration with observability stacks like Datadog, OpenTelemetry, or similar. missing for 10: log/trace export mechanism, OpenTelemetry or webhook integration, documentation of any observability-stack connector.

   ⚑ flag

   Portkey—none0/10

   Evidence describes Portkey's own logs UI, sharing log URLs, and manual feedback tagging, but there is no mention of exporting logs/traces to third-party observability stacks (e.g., OpenTelemetry, Datadog, Grafana) or any API/webhook for bulk export.

   ⚑ flag

Logs

1. platform engineerInspect logged requests and responses with latency, token counts, and cost attached

   weight 3 · round to Portkey
   OpenRouter—none0/10

   The evidence pack covers routing, caching, BYOK, budgets, and key management, but contains no mention of a request/response log viewer, latency dashboards, or per-request token/cost breakdowns that a platform engineer could inspect. missing for 10: activity/logs dashboard documentation, per-request latency metrics, per-request token count and cost attribution evidence.

   ⚑ flag

   Portkey~partialclaimed5/10

   Portkey's docs confirm a Logs section showing a chronological list of all requests with unique shareable URLs and manual feedback tagging, supporting log inspection generally, and repeated claims of 'monitoring all LLM requests' and 'visibility' across the gateway. However, the evidence pack never explicitly documents latency, token count, or cost fields being displayed per log entry. Missing for 10: explicit documentation or screenshots showing latency/tokens/cost columns in the logs UI, independent/hands-on confirmation of these fields.

   • [claimed-docs] “The Logs section presents a chronological list of all the requests processed through Portkey.”
   • [claimed-docs] “Each log on Portkey has a unique URL. You can copy the link from the address bar and directly share it with anyone in your org.”
   • [claimed-docs] “you can also add manual feedback on the logs to be analysed and filtered later”
   • [claimed-docs] “It takes 2 mins to integrate and with that, it starts monitoring all of your LLM requests”
   • [claimed-docs] “It takes 2 mins to integrate and with that, it starts monitoring all of your LLM requests and makes your app resilient, secure, performant, …”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to OpenRouter
   OpenRouter~partialprobed6/10

   OpenRouter's core capabilities—model routing, fallbacks, provider selection, key management, workspace budgets, privacy settings, and even the model catalog—are all documented as programmatically accessible via the API (openrouter-docs-8, 10, 12, 21, 35, openapi.json probe), suggesting strong API/UI parity for a product that is API-first. However, there's no explicit vendor claim of full UI-API parity, and dashboard-only features like rankings/analytics visualizations aren't confirmed as API-exposed. Missing for 10: explicit parity statement, evidence that UI-exclusive features (e.g., rankings, usage analytics dashboards) are also fully API-accessible, and independent confirmation from hands-on API users of complete feature parity.

   • [claimed-docs] “OpenRouter provides endpoints to programmatically manage your API keys, enabling key creation and management for applications that need to d…”
   • [claimed-docs] “Workspace budgets let you cap how much a workspace can spend on OpenRouter inference. Set a dollar limit on any combination of intervals — d…”
   • [claimed-docs] “You can restrict individual requests to only use providers with a certain data policy. This is also available as an account-wide setting in …”
   • [claimed-docs] “You can customize how your requests are routed using the `provider` object in the request body for [Chat Completions]”
   • [claimed-docs] “Browse the full catalog at openrouter.ai/models, or list every available slug programmatically via the GET /api/v1/models endpoint.”
   • [probe] “PROBE openapi: HTTP 200 at https://openrouter.ai/openapi.json — contains "openapi" key”
   • [probe] “PROBE keyless-endpoint: GET https://openrouter.ai/api/v1/models returned HTTP 200 application/json with 426 models (unauthenticated) - live …”
   • [community] “One thing that OpenRouter makes easy is the ability to manage API keys (mint new ones, expiry/limits per key, etc.) that I wish other provid…”

   proof ↗⚑ flag

   Portkey~partialprobed5/10

   Portkey's core AI gateway (chat/completions, fallbacks, caching, retries, load balancing) is clearly exposed via a REST API with a stable base URL, and the open-source gateway can be run locally — showing strong API-first design for request routing. However there is no evidence of API access to the administrative/UI-only features such as budget & rate-limit configuration, log viewing/sharing, feedback tagging, or virtual-key management, and probes show no public OpenAPI spec, so full UI-API parity isn't demonstrated. missing for 10: API endpoints for budget/rate-limit admin, log retrieval/sharing via API, feedback API, and a public OpenAPI spec confirming complete parity.

   • [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey’s features into your existing setup.”
   • [claimed-docs] “The base URL for all REST API requests is `https://api.portkey.ai/v1`”
   • [claimed-docs] “Ideal for: if you’re currently utilizing OpenAI’s Python or Node.js SDKs. By changing the base URL and adding Portkey-specific headers, you …”
   • [claimed-docs] “We've open sourced our battle-tested AI gateway to the community. You can run it locally with a single command:”
   • [claimed-docs] “Configure budget and rate limits on API keys to effectively manage AI spending and usage across your organization”
   • [claimed-docs] “The Logs section presents a chronological list of all the requests processed through Portkey.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://portkey.ai/openapi.json, https://portkey.ai/swagger.json, https://portkey.ai/api/openapi.jso…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round drawn
   OpenRouter—none0/10

   The evidence pack covers API routing, model access, MCP server, and billing features, but contains no mention of any user data export capability (usage logs, account data, chat history) in open formats or account portability/closure process — the axis is applicable to a SaaS platform storing usage/billing data, but no supporting evidence exists.

   ⚑ flag

   Portkey—none0/10

   Portkey's docs cover logging, gateway configuration, and self-hosting the open-source gateway, but there is no mention of exporting stored logs/configs/data in open formats or facilitating a full data export for migration away from the platform. Shareable log URLs (portkey-docs-12) are not a bulk export/portability feature.

   • [claimed-docs] “The Logs section presents a chronological list of all the requests processed through Portkey.”
   • [claimed-docs] “Each log on Portkey has a unique URL. You can copy the link from the address bar and directly share it with anyone in your org.”
   • [claimed-docs] “We've open sourced our battle-tested AI gateway to the community. You can run it locally with a single command:”

   proof ↗⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round to Portkey
   OpenRouter—none0/10

   No evidence that OpenRouter's own source code (the API service, routing platform, or website) is published under an open license; evidence pack only covers API docs, features, and OpenAPI spec, not source availability. missing for 10: any repository link, open-source license file, or statement that OpenRouter's codebase is publicly available.

   ⚑ flag

   Portkey~partialclaimed5/10

   Portkey explicitly states it open-sourced its AI gateway component ('We've open sourced our battle-tested AI gateway to the community... run it locally with a single command'), which supports reading some source code. However, this only covers the gateway, not the full product (observability, admin console, model catalog, etc.), and no explicit license name or repository link is provided in the evidence. Missing for 10: explicit open-source license identification, confirmation that the broader Portkey platform (not just the gateway) is open source, and a linked public repository.

   • [claimed-docs] “We've open sourced our battle-tested AI gateway to the community. You can run it locally with a single command:”
   • [claimed-docs] “npx @portkey-ai/gateway”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round to OpenRouter
   OpenRouter~partialcommunity6/10

   OpenRouter documents EU/US in-region routing for enterprise customers and per-request/account-wide data-policy restrictions, giving users some control over where data is processed, but this is gated to enterprise accounts rather than available to all AI-native users, and community comments note there's no true ZDR/data-sovereignty guarantee in practice. missing for 10: general-tier (non-enterprise) region selection, independent verification that in-region routing works as claimed, and clarity on residency guarantees beyond enterprise contracts.

   • [claimed-docs] “For enterprise customers, OpenRouter supports in-region routing in the EU and US. When enabled for your account, your prompts and completion…”
   • [claimed-docs] “You can restrict individual requests to only use providers with a certain data policy. This is also available as an account-wide setting in …”
   • [claimed-docs] “On your account settings page, you can set whether you would like to allow routing to providers that may train on your data (according to th…”
   • [community] “OpenRouter doesn't expose all the LLM sampling parameters/research that llamacpp, vllm, sglang expose (so no high temperature/highly diverse…”

   proof ↗⚑ flag

   Portkey—none0/10

   No evidence pack items mention data residency, regional storage options, or geographic hosting choices; Portkey's docs focus on gateway routing, caching, retries, budgets, and observability rather than data residency controls.

   ⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round to OpenRouter
   OpenRouter~partialcommunity6/10

   OpenRouter lets users restrict requests/account-wide to providers with certain data policies and opt out of routing to providers that may train on data, plus EU/US in-region processing for enterprise, which directly supports preventing training-data use; however community evidence notes there is 'no ZDR' (zero data retention) and 'no true guarantees of ZDR/privacy/data sovereignty,' indicating limits to how absolute this protection is. missing for 10: independent verification that opt-out settings are consistently honored by all providers, clarity on default behavior, and resolution of the ZDR gap raised by users.

   • [claimed-docs] “You can restrict individual requests to only use providers with a certain data policy. This is also available as an account-wide setting in …”
   • [claimed-docs] “On your account settings page, you can set whether you would like to allow routing to providers that may train on your data (according to th…”
   • [claimed-docs] “You can restrict individual requests to only use providers with a certain data policy.”
   • [claimed-docs] “For enterprise customers, OpenRouter supports in-region routing in the EU and US. When enabled for your account, your prompts and completion…”
   • [community] “No ZDR. No dice.”
   • [community] “OpenRouter doesn't expose all the LLM sampling parameters/research that llamacpp, vllm, sglang expose (so no high temperature/highly diverse…”

   proof ↗⚑ flag

   Portkey—none0/10

   Evidence pack covers gateway routing, caching, retries, logging, budgeting, etc., but contains no mention of a data-training opt-out, zero-retention policy, or any privacy control preventing use of data for AI model training.

   ⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round to OpenRouter
   OpenRouter~partialcommunity5/10

   OpenRouter lets users restrict routing to providers with certain data policies (e.g. no-training, zero-retention) both per-request and account-wide, and offers EU/US in-region processing for enterprise, which are real retention/data-policy controls. However there is no documented mechanism for deleting already-retained data or setting explicit retention periods, and community comments (e.g. 'No ZDR. No dice') indicate the zero-data-retention toggle has real-world limitations/gaps rather than full guaranteed retention control. Missing for 10: explicit data deletion/erasure endpoint, documented retention-period settings, and independent confirmation that ZDR/data-policy filters reliably cover all providers.

   • [claimed-docs] “You can restrict individual requests to only use providers with a certain data policy. This is also available as an account-wide setting in …”
   • [claimed-docs] “For enterprise customers, OpenRouter supports in-region routing in the EU and US. When enabled for your account, your prompts and completion…”
   • [claimed-docs] “On your account settings page, you can set whether you would like to allow routing to providers that may train on your data (according to th…”
   • [claimed-docs] “You can restrict individual requests to only use providers with a certain data policy.”
   • [community] “No ZDR. No dice.”
   • [community] “Since it's only discounted on the standard 'OpenAI,' non-ZDR route (old pricing on Azure), a lot of users won't see this benefit since many …”
   • [community] “OpenRouter doesn't expose all the LLM sampling parameters/research that llamacpp, vllm, sglang expose (so no high temperature/highly diverse…”

   proof ↗⚑ flag

   Portkey—none0/10

   The evidence pack covers gateway routing, caching, retries, budgets, and logging, but contains no mention of data retention policies, log/data deletion controls, or configurable retention windows.

   ⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round to OpenRouter
   OpenRouter~partialcommunity4/10

   OpenRouter documents privacy controls—restricting requests to providers with certain data policies, disallowing providers that may train on your data, and EU/US in-region routing for enterprise—which give users some control over how their prompt data is handled downstream, but this is about provider-side data handling, not an explicit opt-out of OpenRouter's own telemetry/usage tracking. Community evidence directly notes 'No ZDR. No dice' and 'no true guarantees of ZDR/privacy/data sovereignty,' indicating real limits on data-handling assurances. Missing for 10: an explicit telemetry/usage-tracking opt-out toggle for OpenRouter's own platform analytics, and independent confirmation that opting out actually stops tracking.

   • [claimed-docs] “You can restrict individual requests to only use providers with a certain data policy. This is also available as an account-wide setting in …”
   • [claimed-docs] “On your account settings page, you can set whether you would like to allow routing to providers that may train on your data (according to th…”
   • [claimed-docs] “For enterprise customers, OpenRouter supports in-region routing in the EU and US. When enabled for your account, your prompts and completion…”
   • [community] “No ZDR. No dice.”
   • [community] “OpenRouter doesn't expose all the LLM sampling parameters/research that llamacpp, vllm, sglang expose (so no high temperature/highly diverse…”

   proof ↗⚑ flag

   Portkey—none0/10

   Portkey is essentially a telemetry/observability platform (logs, monitoring of all LLM requests) with no evidence of any opt-out mechanism for its own tracking or usage telemetry; the evidence pack only documents features for logging, caching, retries, and governance, never a privacy/telemetry opt-out control.

   ⚑ flag

Fallbacks

1. platform engineerConfigure automatic fallback to another model or provider when one fails

   weight 3 · round drawn
   OpenRouter✓fullcommunity9/10

   OpenRouter documents explicit model fallback via a priority-ordered `models` array that auto-retries the next model on error, plus provider-level fallback/load balancing across top providers, and community confirms this works ('They also support fallback by default so you don't have to write wrappers'). This directly matches the platform engineer's story of configuring automatic fallback across models/providers on failure. Missing for 10: independent quantitative reliability data or a documented edge-case failure of the fallback mechanism itself.

   • [claimed-docs] “Provide an array of model IDs in priority order. If the first model returns an error, OpenRouter will automatically try the next model in th…”
   • [claimed-docs] “The models parameter lets you automatically try other models if the primary model's providers are down, rate-limited, or refuse to reply due…”
   • [claimed-docs] “OpenRouter routes requests to the best available providers for your model. By default, requests are load balanced across the top providers t…”
   • [claimed-docs] “Reliable AI models via our distributed infrastructure. Fall back to other providers when one goes down.”
   • [community] “Happy for the OpenRouter team... the ability to run any AI model with the same api key and same request has been great to experiment with ne…”

   proof ↗⚑ flag

   Portkey✓fullclaimed9/10

   Portkey's docs explicitly describe configuring a prioritized list of providers/models with automatic fallback when the primary fails, customizable trigger status codes, plus complementary retries and load balancing for resilience. missing for 10: independent/hands-on corroboration beyond vendor docs.

   • [claimed-docs] “Specify a prioritized list of providers/models. If the primary LLM fails, Portkey automatically falls back to the next in line.”
   • [claimed-docs] “By default, fallback triggers on any **non-2xx** status code. Customize with `on_status_codes`”
   • [claimed-docs] “Automatically retry failed LLM requests with exponential backoff.”
   • [claimed-docs] “Configure per-strategy circuit protection and failure handling”

   proof ↗⚑ flag

Load balancing

1. platform engineerLoad-balance traffic across providers, deployments, or keys by weight, latency, or cost

   weight 2 · round to Portkey
   OpenRouter✓fullcommunity7/10

   Docs show default load balancing across top providers, customizable provider routing (sort by price/throughput), and ordered model fallbacks, plus BYOK for provider-key control, corroborated by community mentions of built-in fallback behavior. Missing for 10: explicit documentation of weighted routing across specific API keys and independent benchmarks validating latency/cost-based routing performance in practice.

   • [claimed-docs] “OpenRouter routes requests to the best available providers for your model. By default, requests are load balanced across the top providers t…”
   • [claimed-docs] “Provide an array of model IDs in priority order. If the first model returns an error, OpenRouter will automatically try the next model in th…”
   • [claimed-docs] “You can customize how your requests are routed using the `provider` object in the request body for [Chat Completions]”
   • [claimed-docs] “You can customize how your requests are routed using the provider object in the request body for Chat Completions.”
   • [claimed-docs] “OpenRouter supports both OpenRouter credits and the option to bring your own provider keys (BYOK)... Using provider keys enables direct cont…”
   • [community] “Happy for the OpenRouter team... the ability to run any AI model with the same api key and same request has been great to experiment with ne…”

   proof ↗⚑ flag

   Portkey✓fullclaimed8/10

   Portkey's docs explicitly document load balancing to distribute traffic across multiple LLMs/providers by weight, plus gradual migration (percentage-based rollout) and fallback on failure/latency-driven retries, all configurable via Configs. This directly matches weight-based and failure/latency-based routing across providers/deployments/keys, backed by first-party docs on load-balancing, fallbacks, and virtual keys. Missing for 10: explicit cost-based routing strategy documentation and independent/hands-on verification beyond vendor docs.

   • [claimed-docs] “Distribute traffic across multiple LLMs to prevent any single provider from becoming a bottleneck.”
   • [claimed-docs] “**Gradual Migration** | Test new models with small percentage before full rollout”
   • [claimed-docs] “**Gradual Migration**|Test new models with small percentage before full rollout”
   • [claimed-docs] “Specify a prioritized list of providers/models. If the primary LLM fails, Portkey automatically falls back to the next in line.”
   • [claimed-docs] “By default, fallback triggers on any **non-2xx** status code. Customize with `on_status_codes`”
   • [claimed-docs] “The core concept remains the same - one Portkey API key gives you access to multiple providers and models.”
   • [claimed-docs] “Configure per-strategy circuit protection and failure handling”

   proof ↗⚑ flag

Policy routing

1. ai-native userMy agent can switch models mid-task by policy — cost, capability, or availability — through gateway routing rules

   weight 3 · round drawn
   OpenRouter✓fullcommunity8/10

   OpenRouter's docs and community evidence confirm model fallback lists (priority order, automatic retry on error/rate-limit/down), provider-selection routing rules, load balancing across providers, a 'latest' alias for staying current, and an auto-routing 'meta' model that picks capability-appropriate models to control cost — directly matching policy-based mid-task model switching by cost/capability/availability. missing for 10: no first-party documentation or hands-on evidence of dynamically switching models *within* a single in-flight task/conversation (vs. per-request fallback/config), and no independent benchmark validating routing-rule reliability in production.

   • [claimed-docs] “The models parameter lets you automatically try other models if the primary model's providers are down, rate-limited, or refuse to reply due…”
   • [claimed-docs] “Provide an array of model IDs in priority order. If the first model returns an error, OpenRouter will automatically try the next model in th…”
   • [claimed-docs] “OpenRouter routes requests to the best available providers for your model. By default, requests are load balanced across the top providers t…”
   • [claimed-docs] “You can customize how your requests are routed using the `provider` object in the request body for [Chat Completions]”
   • [claimed-docs] “a latest alias that always resolves to the newest OpenAI flagship model, so your code keeps using the freshest version without redeploying”
   • [community] “Happy for the OpenRouter team... the ability to run any AI model with the same api key and same request has been great to experiment with ne…”
   • [community] “One thing I really like about OpenRouter is their 'meta' model that automatically routes the prompt to an appropriately capable model. Saves…”
   • [claimed-docs] “Reliable AI models via our distributed infrastructure. Fall back to other providers when one goes down.”

   proof ↗⚑ flag

   Portkey✓fullclaimed8/10

   Portkey's gateway supports policy-based model switching via conditional fallbacks (status-code triggers), load balancing/weighted routing, gradual migration and canary rollout, plus budget/rate-limit enforcement that can trigger automatic switching for cost/availability reasons - all configurable via routing configs applied at request time. This directly matches mid-task, policy-driven model switching for cost, capability, or availability. Missing for 10: independent/hands-on corroboration of dynamic mid-task capability-based switching (vs. failure-triggered fallback), and explicit example of capability-based routing rules (e.g., routing by task type).

   • [claimed-docs] “Specify a prioritized list of providers/models. If the primary LLM fails, Portkey automatically falls back to the next in line.”
   • [claimed-docs] “By default, fallback triggers on any **non-2xx** status code. Customize with `on_status_codes`”
   • [claimed-docs] “Distribute traffic across multiple LLMs to prevent any single provider from becoming a bottleneck.”
   • [claimed-docs] “**Gradual Migration** | Test new models with small percentage before full rollout”
   • [claimed-docs] “Canary test new models in production”
   • [claimed-docs] “Configure budget and rate limits on API keys to effectively manage AI spending and usage across your organization”
   • [claimed-docs] “Budget limits allow you to set maximum LLM spending or token usage thresholds on your API keys, automatically preventing further usage when …”
   • [claimed-docs] “Set hourly, daily, or per minute rate limits on requests or tokens sent”
   • [claimed-docs] “Configure per-strategy circuit protection and failure handling”

   proof ↗⚑ flag

Rate limits

1. platform engineerSmooth provider rate limits by spreading traffic across keys and queuing or throttling requests

   weight 2 · round to Portkey
   OpenRouter~partialcommunity5/10

   OpenRouter's default load balancing across providers and automatic model/provider fallbacks (docs-3,4,19) inherently smooths provider-side rate limits, and its key-management API (docs-8) plus BYOK (docs-9,24,32) let engineers create/distribute multiple keys with per-key limits (echoed by comm-12). But there is no explicit documentation of client-side request queuing or throttling, or of deliberately spreading traffic across multiple OpenRouter keys for load smoothing — the evidence covers provider-side resilience, not key-pool traffic shaping. Missing for 10: explicit queuing/throttling mechanism, documented multi-key load-balancing pattern, independent confirmation of rate-limit smoothing at scale.

   • [claimed-docs] “The models parameter lets you automatically try other models if the primary model's providers are down, rate-limited, or refuse to reply due…”
   • [claimed-docs] “OpenRouter routes requests to the best available providers for your model. By default, requests are load balanced across the top providers t…”
   • [claimed-docs] “OpenRouter provides endpoints to programmatically manage your API keys, enabling key creation and management for applications that need to d…”
   • [claimed-docs] “OpenRouter supports both OpenRouter credits and the option to bring your own provider keys (BYOK)... Using provider keys enables direct cont…”
   • [claimed-docs] “Provide an array of model IDs in priority order. If the first model returns an error, OpenRouter will automatically try the next model in th…”
   • [community] “One thing that OpenRouter makes easy is the ability to manage API keys (mint new ones, expiry/limits per key, etc.) that I wish other provid…”

   proof ↗⚑ flag

   Portkey✓fullclaimed8/10

   Portkey documents load balancing across multiple provider keys/models to avoid bottlenecks, automatic retries with exponential backoff or provider retry-after headers, and configurable per-key/org rate limits (hourly/daily/per-minute) alongside budget limits — directly addressing spreading traffic and throttling/queuing around rate limits. Multiple virtual keys per provider further support distributing load to smooth rate limits. Missing for 10: explicit first-party or independent evidence of a request queue (vs. retry/backoff) and hands-on/independent validation of these features working under real rate-limit pressure.

   • [claimed-docs] “Distribute traffic across multiple LLMs to prevent any single provider from becoming a bottleneck.”
   • [claimed-docs] “Automatically retry failed LLM requests with exponential backoff.”
   • [claimed-docs] “Enable `use_retry_after_headers` to use the provider’s `retry-after-ms`, `x-ms-retry-after-ms`, or `retry-after` headers instead of exponent…”
   • [claimed-docs] “Enable `use_retry_after_headers` to use the provider's `retry-after-ms`, `x-ms-retry-after-ms`, or `retry-after` headers instead of exponent…”
   • [claimed-docs] “Set hourly, daily, or per minute rate limits on requests or tokens sent”
   • [claimed-docs] “Configure budget and rate limits on API keys to effectively manage AI spending and usage across your organization”
   • [claimed-docs] “The core concept remains the same - one Portkey API key gives you access to multiple providers and models.”
   • [claimed-docs] “Fine-grained control: budgets, rate limits, and model allow-lists at both org and workspace level.”

   proof ↗⚑ flag

Retries

1. platform engineerSet automatic retry policies for transient provider errors

   weight 2 · round to Portkey
   OpenRouter~partialcommunity7/10

   OpenRouter's model-fallbacks and provider load-balancing features automatically retry a request against alternate providers/models when the primary is down, rate-limited, or errors out, and this is corroborated by community reports of built-in fallback behavior (openrouter-docs-3, openrouter-docs-4, openrouter-docs-17, openrouter-docs-19, openrouter-comm-2). However, this is provider/model-level fallback rather than a granular, configurable retry policy (e.g., retry count, backoff intervals, specific transient-error codes) that a platform engineer might expect. Missing for 10: explicit retry-count/backoff configuration options, granular control over which transient error codes trigger retries, and independent verification of retry behavior under real outages.

   • [claimed-docs] “The models parameter lets you automatically try other models if the primary model's providers are down, rate-limited, or refuse to reply due…”
   • [claimed-docs] “OpenRouter routes requests to the best available providers for your model. By default, requests are load balanced across the top providers t…”
   • [claimed-docs] “Reliable AI models via our distributed infrastructure. Fall back to other providers when one goes down.”
   • [claimed-docs] “Provide an array of model IDs in priority order. If the first model returns an error, OpenRouter will automatically try the next model in th…”
   • [community] “Happy for the OpenRouter team... the ability to run any AI model with the same api key and same request has been great to experiment with ne…”

   proof ↗⚑ flag

   Portkey✓fullclaimed8/10

   Portkey's docs explicitly describe automatic retries with exponential backoff and support for provider retry-after headers, directly matching the story of configuring retry policies for transient errors. missing for 10: independent/hands-on corroboration beyond vendor docs, and configuration details on per-error-code or max-retry-count customization.

   • [claimed-docs] “Automatically retry failed LLM requests with exponential backoff.”
   • [claimed-docs] “Enable `use_retry_after_headers` to use the provider’s `retry-after-ms`, `x-ms-retry-after-ms`, or `retry-after` headers instead of exponent…”
   • [claimed-docs] “Enable `use_retry_after_headers` to use the provider's `retry-after-ms`, `x-ms-retry-after-ms`, or `retry-after` headers instead of exponent…”

   proof ↗⚑ flag

Streaming

1. developerStream token-by-token responses through the gateway from any provider

   weight 3 · round to OpenRouter
   OpenRouter✓fullcommunity8/10

   OpenRouter's docs explicitly state streaming is supported for any model through the gateway API, and community feedback corroborates using a single API/key across many providers with model switching in production. missing for 10: independent hands-on confirmation specifically of token-by-token streaming behavior (e.g., latency/chunking quality) and any provider-specific streaming caveats or limitations.

   • [claimed-docs] “The OpenRouter API allows streaming responses from any model. This is useful for building chat interfaces or other applications where the UI…”
   • [claimed-docs] “The OpenRouter API allows streaming responses from *any model*. This is useful for building chat interfaces or other applications where the …”
   • [claimed-docs] “OpenRouter gives you access to hundreds of AI models through a single API endpoint.”
   • [community] “Happy for the OpenRouter team... the ability to run any AI model with the same api key and same request has been great to experiment with ne…”

   proof ↗⚑ flag

   Portkey—none0/10

   The evidence describes Portkey's gateway, fallbacks, retries, load balancing, caching, and OpenAI-compatible base URL integration, but none of the provided docs mention streaming or token-by-token response support. Streaming is a fair and expected capability for an LLM gateway product, so its absence in the evidence pack means this axis applies but is unproven.

   • [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey’s features into your existing setup.”
   • [claimed-docs] “The base URL for all REST API requests is `https://api.portkey.ai/v1`”
   • [claimed-docs] “Ideal for: if you’re currently utilizing OpenAI’s Python or Node.js SDKs. By changing the base URL and adding Portkey-specific headers, you …”
   • [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey's features into your existing setup.”

   proof ↗⚑ flag

Tool calling

1. developerRequest structured JSON-schema outputs across providers

   weight 1 · round drawn
   OpenRouter—none0/10

   The evidence pack covers tool-calling standardization, streaming, routing, fallbacks, and key management, but contains no mention of structured JSON-schema/response_format output support across providers. missing for 10: any documentation or example of JSON-schema response_format, schema validation, or structured output enforcement across providers.

   ⚑ flag

   Portkey—none0/10

   The evidence pack covers gateway features like fallbacks, retries, load balancing, caching, virtual keys, and logging, but contains no mention of structured JSON-schema output support, response_format normalization, or schema validation across providers. Since Portkey acts as a universal AI gateway, this capability would be plausible, but no documentation cites it.

   ⚑ flag

2. developerMake tool and function calls across different providers with a consistent schema

   weight 3 · round to OpenRouter
   OpenRouter✓fullcommunity8/10

   OpenRouter explicitly documents a standardized tool-calling interface across models/providers (docs-5), consistent with its single-API model access across 500+ models and providers (docs-1, docs-16), and community confirms using the same API key/request format to switch models seamlessly (openrouter-comm-2). Streaming is also documented as working across any model (docs-11/20), supporting the streaming-tools theme. Missing for 10: no independent hands-on verification specifically of function/tool-call schema consistency across multiple providers, and no detailed example showing identical tool-call JSON working unchanged across e.g. OpenAI vs Anthropic vs Google models.

   • [claimed-docs] “OpenRouter standardizes the tool calling interface across models and providers, making it easy to integrate external tools with any supporte…”
   • [claimed-docs] “OpenRouter gives you access to hundreds of AI models through a single API endpoint.”
   • [claimed-docs] “The OpenRouter API allows streaming responses from any model. This is useful for building chat interfaces or other applications where the UI…”
   • [claimed-docs] “The OpenRouter API allows streaming responses from *any model*. This is useful for building chat interfaces or other applications where the …”
   • [community] “Happy for the OpenRouter team... the ability to run any AI model with the same api key and same request has been great to experiment with ne…”

   proof ↗⚑ flag

   Portkey~partialclaimed5/10

   Portkey's gateway offers a unified OpenAI-SDK-compatible API (base URL + headers) across 250+ models, which implies a consistent request format including function/tool call fields, and it supports connecting external tools via remote MCP servers. However, there is no explicit documentation describing how tool/function-calling schemas are normalized or translated across different providers' native formats. Missing for 10: explicit docs on function/tool-call schema normalization, examples of tool-calling requests across multiple providers, and independent verification that tool calls work consistently across providers.

   • [claimed-docs] “It serves as a unified interface for interacting with over 250 AI models, offering advanced tools for control, visibility, and security in y…”
   • [claimed-docs] “Ideal for: if you’re currently utilizing OpenAI’s Python or Node.js SDKs. By changing the base URL and adding Portkey-specific headers, you …”
   • [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey’s features into your existing setup.”
   • [claimed-docs] “Connect to Remote MCP severs, allowing you to connect external tools and data sources.”

   proof ↗⚑ flag

Catalog

1. developerBrowse or query a catalog of available models with pricing and context-window metadata

   weight 2 · round to OpenRouter
   OpenRouter✓fullprobed9/10

   OpenRouter documents a browsable model catalog at openrouter.ai/models and a programmatic GET /api/v1/models endpoint listing model slugs, and a live probe confirms this endpoint returns 426 models with pricing/context-window metadata unauthenticated. Community feedback corroborates ease of comparing/switching models via the unified catalog. Missing for 10: no independent third-party review specifically validating completeness/accuracy of pricing or context-window fields beyond the probe.

   • [claimed-docs] “Browse the full catalog at openrouter.ai/models, or list every available slug programmatically via the GET /api/v1/models endpoint.”
   • [probe] “PROBE keyless-endpoint: GET https://openrouter.ai/api/v1/models returned HTTP 200 application/json with 426 models (unauthenticated) - live …”
   • [claimed-docs] “300T+ Monthly Tokens 10M+ Global Users 80+ Providers 500+ Models”
   • [community] “Happy for the OpenRouter team... the ability to run any AI model with the same api key and same request has been great to experiment with ne…”

   proof ↗⚑ flag

   Portkey~partialclaimed5/10

   Portkey documents a 'Model Catalog' providing a single pane to view/manage all providers and models with discovery and governance controls, and claims access to 250+ models, but no evidence shows the catalog surfaces pricing or context-window metadata for models. missing for 10: explicit documentation of per-model pricing info, explicit documentation of context-window size metadata, and any UI/query example showing these fields.

   • [claimed-docs] “A single pane to view and manage every AI provider and model in your organization. It provides centralized governance, discovery, and usage …”
   • [claimed-docs] “It serves as a unified interface for interacting with over 250 AI models, offering advanced tools for control, visibility, and security in y…”
   • [claimed-docs] “Add a provider, then use `@provider-slug/model-name` in your code.”
   • [claimed-docs] “Fine-grained control: budgets, rate limits, and model allow-lists at both org and workspace level.”

   proof ↗⚑ flag

Compatibility

1. developerPoint existing OpenAI-compatible code at the gateway by changing only the base URL and key

   weight 3 · round drawn
   OpenRouter✓fullprobed9/10

   Docs explicitly state the OpenAI SDK can be pointed at OpenRouter as a drop-in replacement, and community users corroborate using 'the same api key and same request' across models, confirming base-URL/key swap works in practice. Missing for 10: no explicit first-party code snippet showing the exact base_url/api_key change in the evidence pack, only the general drop-in claim.

   • [claimed-docs] “You can also use the OpenAI SDK pointed at OpenRouter as a drop-in replacement.”
   • [claimed-docs] “You can also use the [OpenAI SDK](#using-the-openai-sdk) pointed at OpenRouter as a drop-in replacement.”
   • [community] “Happy for the OpenRouter team... the ability to run any AI model with the same api key and same request has been great to experiment with ne…”
   • [probe] “PROBE llms.txt: HTTP 200 at https://openrouter.ai/llms.txt # OpenRouter > A unified API for every major LLM. One endpoint, hundreds of mode…”

   proof ↗⚑ flag

   Portkey✓fullclaimed9/10

   Docs explicitly state that developers using OpenAI's Python/Node SDKs can integrate Portkey by only changing the base URL and adding Portkey-specific headers, with the base URL documented as https://api.portkey.ai/v1. This is a first-party documented, purpose-built migration path. Missing for 10: independent/hands-on third-party confirmation beyond vendor docs.

   • [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey’s features into your existing setup.”
   • [claimed-docs] “The base URL for all REST API requests is `https://api.portkey.ai/v1`”
   • [claimed-docs] “Ideal for: if you’re currently utilizing OpenAI’s Python or Node.js SDKs. By changing the base URL and adding Portkey-specific headers, you …”
   • [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey's features into your existing setup.”

   proof ↗⚑ flag

One endpoint

1. developerCall many model providers through one consistent API

   weight 3 · round drawn
   OpenRouter✓fullprobed9/10

   OpenRouter's core value proposition is a single API endpoint (OpenAI-SDK compatible) that routes to hundreds of models across 80+ providers, with fallback, load-balancing, standardized tool calling, and streaming for any model — corroborated by live probes (426-model catalog, openapi.json) and community testimonials confirming 'same API key and same request' across providers with automatic fallback. Minor gaps: community notes some inconsistencies (missing sampling params, occasional reliability complaints), missing for 10: fully independent benchmark of API consistency across all providers and resolution of reported debugging/latency complaints.

   • [claimed-docs] “OpenRouter gives you access to hundreds of AI models through a single API endpoint.”
   • [claimed-docs] “You can also use the OpenAI SDK pointed at OpenRouter as a drop-in replacement.”
   • [claimed-docs] “Provide an array of model IDs in priority order. If the first model returns an error, OpenRouter will automatically try the next model in th…”
   • [claimed-docs] “OpenRouter standardizes the tool calling interface across models and providers, making it easy to integrate external tools with any supporte…”
   • [claimed-docs] “The OpenRouter API allows streaming responses from *any model*. This is useful for building chat interfaces or other applications where the …”
   • [probe] “PROBE llms.txt: HTTP 200 at https://openrouter.ai/llms.txt # OpenRouter > A unified API for every major LLM. One endpoint, hundreds of mode…”
   • [probe] “PROBE keyless-endpoint: GET https://openrouter.ai/api/v1/models returned HTTP 200 application/json with 426 models (unauthenticated) - live …”
   • [community] “Happy for the OpenRouter team... the ability to run any AI model with the same api key and same request has been great to experiment with ne…”
   • [community] “It's definitely the best way to try out new models without fiddling with each providers distinct APIs. That said, I don't understand people …”

   proof ↗⚑ flag

   Portkey✓fullclaimed9/10

   Portkey documents a unified API/gateway (single base URL, single API key) that supports 250+ providers/models via consistent SDK/REST calls, with virtual keys and model catalog abstracting provider differences. Missing for 10: independent third-party benchmark or hands-on confirmation of the claimed 250+ provider coverage beyond vendor docs.

   • [claimed-docs] “The core concept remains the same - one Portkey API key gives you access to multiple providers and models.”
   • [claimed-docs] “By changing the base URL and adding Portkey-specific headers, you can quickly integrate Portkey’s features into your existing setup.”
   • [claimed-docs] “The base URL for all REST API requests is `https://api.portkey.ai/v1`”
   • [claimed-docs] “It serves as a unified interface for interacting with over 250 AI models, offering advanced tools for control, visibility, and security in y…”
   • [claimed-docs] “Add a provider, then use `@provider-slug/model-name` in your code.”
   • [claimed-docs] “Ideal for: if you’re currently utilizing OpenAI’s Python or Node.js SDKs. By changing the base URL and adding Portkey-specific headers, you …”

   proof ↗⚑ flag