LiteLLM vs Vercel AI Gateway

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to Vercel AI Gateway
   LiteLLM✓fullprobed8/10

   A direct probe confirms LiteLLM serves a valid llms.txt file at https://docs.litellm.ai/llms.txt returning HTTP 200 with an agent-readable overview and links, exactly matching the story's request. Missing for 10: no independent/community corroboration of agents actually consuming this file, and the related docs-as-markdown endpoint (.md) returns 404, suggesting the agent-oriented docs surface is only partially built out.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.litellm.ai/llms.txt # https://docs.litellm.ai/ llms.txt - [LiteLLM Overview](https://docs.litellm.…”
   • [probe] “PROBE docs-md: HTTP 404 at https://docs.litellm.ai/docs/.md”

   proof ↗⚑ flag

   Vercel AI Gateway✓fullprobed9/10

   Vercel confirms a live llms.txt (probe-1) plus per-page .md documentation endpoints (probe-2) and an OpenAPI spec (probe-3), all machine-readable for agent consumption, and docs explicitly mention coding-agent setup guidance (docs-8, docs-24). Missing for 10: no independent/community confirmation that agents actually use these llms.txt/docs successfully in practice.

   • [probe] “PROBE llms.txt: HTTP 200 at https://vercel.com/llms.txt # Vercel > Vercel is a cloud platform for building, deploying, and scaling web appl…”
   • [probe] “PROBE docs-md: HTTP 200 at https://vercel.com/docs/ai-gateway.md --- title: AI Gateway product: vercel url: /docs/ai-gateway canonical_url: …”
   • [probe] “PROBE openapi: HTTP 200 at https://vercel.com/openapi.json — contains "openapi" key”
   • [claimed-docs] “vercel ai-gateway coding-agents setup”
   • [claimed-docs] “AI Gateway routes requests, manages fallbacks and budgets, monitors usage, and connects supported coding agents.”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to Vercel AI Gateway
   LiteLLM~partialprobed6/10

   LiteLLM is a Python SDK and proxy server with a documented CLI (worker processes, uvicorn/gunicorn/Granian) that can run headlessly in scripts or containers, and community evidence shows it running via Docker in unattended homelab/gateway setups. However there is no explicit CI/CD pipeline example, GitHub Actions integration, or automation-specific documentation in the evidence pack. Missing for 10: explicit CI pipeline examples, automation/scripting docs beyond CLI flags, evidence of non-interactive auth/config for CI environments.

   • [claimed-docs] “The number of worker processes to spin up (uvicorn, gunicorn, or Granian --workers).”
   • [claimed-docs] “The number of worker processes to spin up (uvicorn, gunicorn, or Granian `--workers`).”
   • [probe] “official CLI documented at https://docs.litellm.ai/docs/proxy/cli”
   • [community] “A user described running LiteLLM as a proxy in their homelab via the litellm/litellm docker image for local LLM gateway management, noting t…”
   • [claimed-docs] “Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI”

   proof ↗⚑ flag

   Vercel AI Gateway✓fullprobed8/10

   AI Gateway is an API/HTTP gateway with OpenAI-compatible endpoints, unauthenticated model listing, OIDC token auth for deployed apps, and a keyless models endpoint verified live — all of which support pure headless/CI/automation use without any UI dependency. missing for 10: no explicit CI pipeline example (e.g., GitHub Actions) or independent third-party report of CI usage.

   • [claimed-docs] “You can use existing OpenAI client libraries, switch to AI Gateway with a URL change, and keep your current tools and workflows without code…”
   • [claimed-docs] “For applications deployed on Vercel, OIDC tokens are automatically available as `VERCEL_OIDC_TOKEN`. No secrets to manage, no keys to rotate…”
   • [claimed-docs] “This endpoint requires no authentication and returns detailed information including model IDs, context windows, and pricing.”
   • [claimed-docs] “For applications deployed on Vercel, OIDC tokens are automatically available as VERCEL_OIDC_TOKEN. No secrets to manage, no keys to rotate. …”
   • [claimed-docs] “You can use the AI Gateway's Chat Completions API with existing tools and libraries like the OpenAI client libraries and AI SDK. Point your …”
   • [probe] “PROBE openapi: HTTP 200 at https://vercel.com/openapi.json — contains "openapi" key”
   • [probe] “PROBE keyless-endpoint: GET https://ai-gateway.vercel.sh/v1/models returned HTTP 200 application/json with an OpenAI-compatible {"object":"l…”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round to LiteLLM
   LiteLLM✓fullclaimed8/10

   LiteLLM Proxy has a documented MCP Gateway feature letting users add MCP servers via the UI and use a fixed endpoint to expose their tools, controllable by key/team — directly matching the story. missing for 10: independent/hands-on corroboration of MCP tool usage beyond first-party docs, and detail on tool-calling behavior once connected.

   • [claimed-docs] “LiteLLM Proxy provides an MCP Gateway that allows you to use a fixed endpoint for all MCP tools and control MCP access by Key, Team.”
   • [claimed-docs] “On the LiteLLM UI, Navigate to "MCP Servers" and click "Add New MCP Server".”

   proof ↗⚑ flag

   Vercel AI Gateway—none0/10

   The evidence describes Vercel AI Gateway as a model-routing/proxy layer (provider switching, fallbacks, spend monitoring, caching, coding-agent connections) but contains no mention of MCP server integration or tool-use via MCP anywhere in the docs pack.

   ⚑ flag

4. ai-native userConnect an agent via an official MCP server

   weight 3 · round to LiteLLM
   LiteLLM✓fullclaimed7/10

   LiteLLM Proxy ships an official MCP Gateway feature exposing a fixed endpoint for all configured MCP tools, with access controlled by Key/Team, and a UI flow to add MCP servers — this is exactly the 'connect an agent via an official MCP server' capability for a gateway product. Missing for 10: independent/hands-on confirmation that external agents (e.g. Claude, Cursor) successfully connect via this gateway, and deeper documentation of the MCP protocol compliance/auth flow beyond the brief doc mentions.

   • [claimed-docs] “LiteLLM Proxy provides an MCP Gateway that allows you to use a fixed endpoint for all MCP tools and control MCP access by Key, Team.”
   • [claimed-docs] “On the LiteLLM UI, Navigate to "MCP Servers" and click "Add New MCP Server".”

   proof ↗⚑ flag

   Vercel AI Gateway—none0/10

   The docs mention connecting 'supported coding agents' and generic gateway integration, but there is no evidence of an official MCP (Model Context Protocol) server or endpoint that agents can connect to. Missing for 10: explicit MCP server documentation, MCP endpoint URL, and demonstration of an agent connecting via MCP.

   • [claimed-docs] “vercel ai-gateway coding-agents setup”
   • [claimed-docs] “AI Gateway routes requests, manages fallbacks and budgets, monitors usage, and connects supported coding agents.”

   proof ↗⚑ flag

5. ai-native userUse an official CLI

   weight 2 · round to LiteLLM
   LiteLLM~partialprobed6/10

   LiteLLM ships an official CLI for its proxy (litellm docs-proxy-cli, litellm-probe-4) covering server startup and worker config, but the docs shown focus narrowly on process/worker management rather than a broad AI-native workflow CLI (e.g., chat, model testing, key/budget management from CLI). missing for 10: evidence of CLI commands beyond process startup (e.g., testing prompts, managing virtual keys/budgets, MCP servers via CLI), independent/hands-on confirmation of CLI usability.

   • [claimed-docs] “The number of worker processes to spin up (uvicorn, gunicorn, or Granian --workers).”
   • [claimed-docs] “The number of worker processes to spin up (uvicorn, gunicorn, or Granian `--workers`).”
   • [probe] “official CLI documented at https://docs.litellm.ai/docs/proxy/cli”

   proof ↗⚑ flag

   Vercel AI Gateway~partialclaimed3/10

   The only evidence of a CLI is the fragment 'vercel ai-gateway coding-agents setup', implying an official `vercel` CLI subcommand for configuring the AI Gateway with coding agents, but there is no detailed command reference, install instructions, or broader CLI documentation for AI-native workflows beyond this single mention. Missing for 10: full CLI command reference/docs, examples of CLI usage for model routing/config, independent confirmation of CLI functionality.

   • [claimed-docs] “vercel ai-gateway coding-agents setup”

   proof ↗⚑ flag

6. ai-native userDrive the product through a documented public API

   weight 3 · round to Vercel AI Gateway
   LiteLLM✓fullprobed7/10

   LiteLLM's core interface — the completion() function and the OpenAI-compatible proxy REST API — is extensively documented as a public, drop-in API that AI-native users can call directly, corroborated by community Show HN discussion confirming streaming/function-calling parity with the OpenAI SDK. Missing for 10: a discoverable machine-readable OpenAPI/swagger spec (probe found only 404s) and independent hands-on confirmation of API stability/versioning.

   • [claimed-docs] “Call any provider using the same completion() interface, with no API to re-learn for each one”
   • [claimed-docs] “Every response follows the OpenAI Chat Completions format, regardless of provider.”
   • [github] “Drop-in OpenAI compatibility — swap providers without rewriting your code”
   • [claimed-docs] “Call any provider using the same `completion()` interface, with no API to re-learn for each one”
   • [claimed-docs] “Consistent output format regardless of which provider or model you use”
   • [community] “LiteLLM maintainer confirmed proxy supports streaming and function-calling in the same way as the openai-python SDK, and added Ollama integr…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.litellm.ai/openapi.json, https://docs.litellm.ai/swagger.json, https://docs.litellm.ai/…”

   proof ↗⚑ flag

   Vercel AI Gateway✓fullprobed9/10

   AI Gateway exposes a documented, OpenAI-compatible public API (Chat Completions, models endpoint) with a verified OpenAPI spec, unauthenticated models endpoint, and probe confirming a live JSON API response, enabling programmatic/agentic control via standard SDKs or raw HTTP. missing for 10: independent third-party developer writeups explicitly praising the API's completeness/stability beyond Vercel's own docs and probes.

   • [claimed-docs] “You can use existing OpenAI client libraries, switch to AI Gateway with a URL change, and keep your current tools and workflows without code…”
   • [claimed-docs] “This endpoint requires no authentication and returns detailed information including model IDs, context windows, and pricing.”
   • [claimed-docs] “Query the models endpoint directly to get a JSON list of all available models with pricing and capabilities”
   • [claimed-docs] “You can use the AI Gateway's Chat Completions API with existing tools and libraries like the OpenAI client libraries and AI SDK. Point your …”
   • [probe] “PROBE openapi: HTTP 200 at https://vercel.com/openapi.json — contains "openapi" key”
   • [probe] “PROBE keyless-endpoint: GET https://ai-gateway.vercel.sh/v1/models returned HTTP 200 application/json with an OpenAI-compatible {"object":"l…”

   proof ↗⚑ flag

7. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round to LiteLLM
   LiteLLM✓fullcommunity8/10

   LiteLLM's proxy virtual keys let admins scope credentials by model access, budget, team, and rate limits (rpm_limit), directly enabling least-privilege API keys for agents. This is well documented across multiple first-party docs (virtual_keys, users, cost_tracking) showing budgets, per-key model restrictions, and team-based limits. Missing for 10: independent/hands-on verification of the scoping granularity in practice, and one community comment (litellm-comm-3) noted vague 'issues with creating tokens for other users' suggesting rough edges.

   • [claimed-docs] “Track Spend, and control model access via virtual keys for the proxy”
   • [claimed-docs] “Personal budgets: Create virtual keys without team_id for individual spending limits”
   • [claimed-docs] “Apply a budget across all calls on the proxy... budget_duration: 30d # (str) frequency of reset”
   • [claimed-docs] “"team_alias": "my-new-team_4", ... "rpm_limit": 99”
   • [claimed-docs] “Track spend for keys, users, and teams across 100+ LLMs.”
   • [community] “'Does anyone know a good alternate project... LiteLLM has been getting worse and trying to get me to upgrade to a paid version. I also had i…”

   proof ↗⚑ flag

   Vercel AI Gateway~partialclaimed5/10

   Docs show budget caps that can be set per API key, per project, or per team member (docs-16), and per-key usage breakdown in Requests logs (docs-12), plus short-lived OIDC tokens for Vercel-deployed apps that avoid static secrets (docs-14/22). This supports issuing distinct, budget-limited keys for an agent, but there is no documentation of fine-grained permission scoping (e.g. restricting a key to specific models/providers/actions) beyond spend limits. missing for 10: explicit least-privilege scoping controls (model/provider/action restrictions per key), role-based permission model, first-party or independent confirmation of true least-privilege agent credentials

   • [claimed-docs] “To cap how much your team, a project, an API key, or a team member can spend, set [budgets]”
   • [claimed-docs] “The Requests section displays summaries by project, API key, and a detailed log of all requests.”
   • [claimed-docs] “For applications deployed on Vercel, OIDC tokens are automatically available as `VERCEL_OIDC_TOKEN`. No secrets to manage, no keys to rotate…”
   • [claimed-docs] “For applications deployed on Vercel, OIDC tokens are automatically available as VERCEL_OIDC_TOKEN. No secrets to manage, no keys to rotate. …”

   proof ↗⚑ flag

8. ai-native userBuild against official SDKs

   weight 2 · round drawn
   LiteLLM✓fullcommunity8/10

   LiteLLM ships an official Python SDK (the `completion()` interface) that mirrors the OpenAI SDK format across 100+ providers, letting AI-native developers build without learning per-provider APIs; this is corroborated by community Show HN feedback praising it for standardizing APIs and confirming SDK parity for streaming/function-calling. Missing for 10: evidence of official SDKs in other languages (e.g., JS/TS) and independent benchmarking of SDK completeness beyond community praise.

   • [claimed-docs] “Call any provider using the same completion() interface, with no API to re-learn for each one”
   • [claimed-docs] “Every response follows the OpenAI Chat Completions format, regardless of provider.”
   • [github] “Drop-in OpenAI compatibility — swap providers without rewriting your code”
   • [claimed-docs] “Call any provider using the same `completion()` interface, with no API to re-learn for each one”
   • [claimed-docs] “Consistent output format regardless of which provider or model you use”
   • [community] “LiteLLM maintainer confirmed proxy supports streaming and function-calling in the same way as the openai-python SDK, and added Ollama integr…”
   • [community] “Show HN launch of the original LiteLLM library; a user praised it: 'This is amazing. Really needed something like this to standardize all my…”

   proof ↗⚑ flag

   Vercel AI Gateway✓fullprobed8/10

   AI Gateway is designed to be used via existing official SDKs (OpenAI client libraries, Anthropic SDK, Vercel AI SDK) with just a base URL/model-string change, documented in openai-compat and getting-started guides, and confirmed by a live OpenAI-compatible endpoint probe. It also supports agent-building primitives like ToolLoopAgent for agentic workflows. missing for 10: no independent/hands-on developer confirmation beyond vendor docs and one API probe.

   • [claimed-docs] “You can use existing OpenAI client libraries, switch to AI Gateway with a URL change, and keep your current tools and workflows without code…”
   • [claimed-docs] “Point your existing OpenAI or Anthropic SDK at AI Gateway. Same calls, no rewrites.”
   • [claimed-docs] “You can use the AI Gateway's Chat Completions API with existing tools and libraries like the OpenAI client libraries and AI SDK. Point your …”
   • [claimed-docs] “Move existing OpenAI, Anthropic, and AI SDK integrations to AI Gateway with a base URL swap.”
   • [claimed-docs] “const agent = new ToolLoopAgent({ model: 'anthropic/claude-sonnet-5', tools: { getWeather: tool({”
   • [probe] “PROBE keyless-endpoint: GET https://ai-gateway.vercel.sh/v1/models returned HTTP 200 application/json with an OpenAI-compatible {"object":"l…”

   proof ↗⚑ flag

9. ai-native userSubscribe to events via webhooks

   weight 2 · round drawn
   LiteLLM—none0/10

   No evidence in the pack mentions webhooks or event subscription mechanisms; LiteLLM's documented features cover logging integrations, cost tracking, and MCP gateway, but nothing about outbound webhook events for subscribers.

   ⚑ flag

   Vercel AI Gateway—none0/10

   No evidence anywhere in the pack of webhook subscriptions or event-driven notifications from AI Gateway; the product exposes REST APIs, logs, and observability dashboards, but nothing about webhooks for events like request completion, budget alerts, or failures.

   ⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round to Vercel AI Gateway
   LiteLLM—none0/10

   The evidence pack shows explicit probe failures for an OpenAPI/Swagger spec (litellm-probe-3) and no documented interactive API reference or runnable examples in the docs; only static markdown-style docs and code snippets are cited (litellm-docs-1/2/16/17). Since LiteLLM's proxy is an API gateway, this axis clearly applies, but there's no evidence of an interactive/try-it-out API reference.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.litellm.ai/openapi.json, https://docs.litellm.ai/swagger.json, https://docs.litellm.ai/…”
   • [probe] “PROBE docs-md: HTTP 404 at https://docs.litellm.ai/docs/.md”
   • [claimed-docs] “Call any provider using the same completion() interface, with no API to re-learn for each one”
   • [claimed-docs] “Call any provider using the same `completion()` interface, with no API to re-learn for each one”

   proof ↗⚑ flag

   Vercel AI Gateway~partialprobed5/10

   There's a documented OpenAPI spec and a live, keyless models endpoint that returns JSON model data, showing some interactive/runnable API surface, but there's no evidence of an interactive API reference UI (like Swagger/Redoc) with embedded runnable examples or a try-it-now console. missing for 10: an actual interactive API explorer/playground page, runnable code snippets embedded in docs, independent confirmation of a docs UI with live request execution.

   • [probe] “PROBE openapi: HTTP 200 at https://vercel.com/openapi.json — contains "openapi" key”
   • [probe] “PROBE keyless-endpoint: GET https://ai-gateway.vercel.sh/v1/models returned HTTP 200 application/json with an OpenAI-compatible {"object":"l…”
   • [claimed-docs] “This endpoint requires no authentication and returns detailed information including model IDs, context windows, and pricing.”
   • [claimed-docs] “Query the models endpoint directly to get a JSON list of all available models with pricing and capabilities”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round to Vercel AI Gateway
   LiteLLM—none0/10

   The evidence pack includes explicit probes for OpenAPI/swagger endpoints on LiteLLM's docs site, all returning 404, and no other citation shows a downloadable OpenAPI spec (e.g., from the proxy's FastAPI docs). Since LiteLLM's proxy is a gateway with an admin UI and CLI, this axis plausibly applies, but no evidence confirms a machine-readable spec is available.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.litellm.ai/openapi.json, https://docs.litellm.ai/swagger.json, https://docs.litellm.ai/…”
   • [probe] “PROBE docs-md: HTTP 404 at https://docs.litellm.ai/docs/.md”

   proof ↗⚑ flag

   Vercel AI Gateway✓fullprobed7/10

   A probe confirms a live OpenAPI JSON spec at vercel.com/openapi.json (contains the 'openapi' key) and the docs also expose a keyless models JSON endpoint and OpenAI-compatible API, all of which are machine-readable and programmatically consumable. Missing for 10: explicit confirmation the openapi.json spec covers AI Gateway endpoints specifically (vs. general Vercel API) and independent third-party validation of the spec's completeness.

   • [probe] “PROBE openapi: HTTP 200 at https://vercel.com/openapi.json — contains "openapi" key”
   • [probe] “PROBE keyless-endpoint: GET https://ai-gateway.vercel.sh/v1/models returned HTTP 200 application/json with an OpenAI-compatible {"object":"l…”
   • [claimed-docs] “This endpoint requires no authentication and returns detailed information including model IDs, context windows, and pricing.”
   • [claimed-docs] “Query the models endpoint directly to get a JSON list of all available models with pricing and capabilities”

   proof ↗⚑ flag

3. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   LiteLLM—none0/10

   No evidence of API versioning scheme or a documented deprecation policy anywhere in docs; OpenAPI/spec discovery probes returned 404s, suggesting no formal versioned API contract is published.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.litellm.ai/openapi.json, https://docs.litellm.ai/swagger.json, https://docs.litellm.ai/…”
   • [probe] “PROBE docs-md: HTTP 404 at https://docs.litellm.ai/docs/.md”

   proof ↗⚑ flag

   Vercel AI Gateway—none0/10

   Evidence shows an OpenAI-compatible endpoint with a 'v1' path and an openapi.json spec, but nowhere does the pack document an explicit API versioning scheme or a deprecation policy for endpoints/models. Missing for 10: explicit versioning strategy documentation, deprecation/sunset timelines, migration guidance for breaking changes.

   • [probe] “PROBE openapi: HTTP 200 at https://vercel.com/openapi.json — contains "openapi" key”
   • [probe] “PROBE keyless-endpoint: GET https://ai-gateway.vercel.sh/v1/models returned HTTP 200 application/json with an OpenAI-compatible {"object":"l…”
   • [claimed-docs] “Query the models endpoint directly to get a JSON list of all available models with pricing and capabilities”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round drawn
   LiteLLM—none0/10

   The evidence pack describes LiteLLM's unified completion interface, routing, fallbacks, cost tracking, and virtual keys, but contains no mention of batch/bulk operations (e.g., batch completions across many prompts, bulk key/user management, or bulk import/export) that would let a user act on many items at once. Since LiteLLM manages many models, keys, and users, this axis is plausible for the product, but no supporting evidence exists.

   ⚑ flag

   Vercel AI Gateway—none0/10

   The evidence pack covers routing, failover, BYOK, observability, and embeddings, but there is no mention of any batch/bulk request API or capability to process many items in a single operation. Since a gateway product could plausibly offer batch endpoints (e.g., batch inference or bulk embeddings), the axis applies, but no evidence supports it being delivered.

   ⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to LiteLLM
   LiteLLM~partialclaimed5/10

   LiteLLM ships several built-in event-triggered automations (automatic fallback to another model/provider on call failure or content-policy violation, budget resets on a schedule, rate limits per key/team) but these are fixed, pre-defined behaviors rather than a general user-definable rule/trigger-action engine (e.g. no webhook/if-this-then-that automation builder is evidenced). Missing for 10: a general-purpose custom rule/condition-action automation system, evidence of user-defined triggers beyond fallback/budget/rate-limit, and any webhook or external-action firing mechanism.

   • [claimed-docs] “If a call fails after num_retries, LiteLLM falls back to another model group, so a failing model or provider automatically fails over to a h…”
   • [claimed-docs] “content_policy_fallbacks: For litellm.ContentPolicyViolationError - LiteLLM maps content policy violation errors across providers”
   • [claimed-docs] “so a failing model or provider automatically fails over to a healthy backup”
   • [claimed-docs] “The request to `model="zephyr-beta"` will fail... litellm proxy will loop through all the model_groups specified in `fallbacks=["gpt-3.5-tur…”
   • [claimed-docs] “Apply a budget across all calls on the proxy... budget_duration: 30d # (str) frequency of reset”
   • [claimed-docs] “"team_alias": "my-new-team_4", ... "rpm_limit": 99”

   proof ↗⚑ flag

   Vercel AI Gateway~partialclaimed4/10

   The Gateway supports narrow built-in automation triggers — failover/retry on provider failure, per-provider timeouts triggering fast failover, and budget caps that block spend once a threshold is hit — which are rule-like actions triggered by events (docs-3, docs-10, docs-13, docs-16). However, there's no general-purpose, user-defined rules/event system (e.g., custom conditions, webhooks, arbitrary trigger-action pairs) for AI-native users to configure; the 'rules' are limited to routing, caching, and spend-cap logic. Missing for 10: a configurable rules engine for arbitrary events/actions, webhook or external-trigger support, and evidence of user-defined conditional automation beyond routing/budget defaults.

   • [claimed-docs] “High reliability. Automatically retries requests to other providers if one fails”
   • [claimed-docs] “You can set per-provider timeouts to trigger fast failover when a provider is slow to respond.”
   • [claimed-docs] “If a query using your credentials fails, AI Gateway will retry the query with its system credentials to improve service availability.”
   • [claimed-docs] “To cap how much your team, a project, an API key, or a team member can spend, set [budgets]”
   • [claimed-docs] “You can use order, only, and sort in providerOptions.gateway to control which providers handle your requests, in what order, and how they ar…”

   proof ↗⚑ flag

Caching

1. developerCache responses at the gateway to cut cost and latency on repeated requests

   weight 2 · round to LiteLLM
   LiteLLM✓fullclaimed8/10

   LiteLLM proxy has a documented caching system that explicitly stores and reuses LLM responses to save costs and reduce latency, directly matching the story. Missing for 10: independent/hands-on benchmarks quantifying latency/cost savings, and detail on cache backend configuration options in the evidence pack.

   • [claimed-docs] “LiteLLM's caching system stores and reuses LLM responses to save costs and reduce latency.”

   proof ↗⚑ flag

   Vercel AI Gateway~partialcommunity6/10

   Docs explicitly describe a `caching: 'auto'` option in providerOptions.gateway that lets the gateway apply provider-appropriate caching automatically, directly supporting cost/latency reduction on repeated requests, and community pricing discussion confirms cache-read pricing exists for at least one model. However, details on cache hit-rate guarantees, TTL, scope, or independent benchmarks of latency/cost savings are absent. Missing for 10: independent hands-on benchmarks showing latency/cost reduction, documentation on cache TTL/invalidation/scope, confirmation caching works uniformly across all providers rather than being provider-dependent.

   • [claimed-docs] “You can use `caching: 'auto'` in `providerOptions.gateway` to let AI Gateway automatically apply the appropriate caching strategy based on t…”
   • [claimed-docs] “You can use caching: 'auto' in providerOptions.gateway to let AI Gateway automatically apply the appropriate caching strategy based on the p…”
   • [community] “Vercel cache read pricing for DeepSeek-v4: $0.01/M for flash, $0.14/M for pro — a 64% discount vs DeepSeek/OpenRouter on flash caching, prom…”

   proof ↗⚑ flag

Latency

1. platform engineerRun traffic through gateway infrastructure that adds minimal latency overhead to provider calls

   weight 2 · round drawn
   LiteLLM~partialcommunity4/10

   LiteLLM docs describe features that reduce or mitigate latency (response caching, configurable worker processes for uvicorn/gunicorn/Granian) and fast failover, but there is no explicit benchmark or vendor claim quantifying the proxy's own added latency overhead versus direct provider calls. Community evidence includes a report of the proxy being 'incredible flaky and causing outages,' which is a reliability complaint rather than a direct latency measurement, so it doesn't rise to a concrete dispute of a latency claim. missing for 10: explicit latency-overhead benchmarks/numbers, independent performance testing of proxy overhead, direct vendor claim about added latency being minimal.

   • [claimed-docs] “LiteLLM's caching system stores and reuses LLM responses to save costs and reduce latency.”
   • [claimed-docs] “The number of worker processes to spin up (uvicorn, gunicorn, or Granian --workers).”
   • [claimed-docs] “The number of worker processes to spin up (uvicorn, gunicorn, or Granian `--workers`).”
   • [claimed-docs] “If a call fails after num_retries, LiteLLM falls back to another model group, so a failing model or provider automatically fails over to a h…”
   • [community] “'Now I feel lucky that I switched to just using OpenRouter a year ago because LiteLLM was incredible flaky and kept causing outages.'”

   proof ↗⚑ flag

   Vercel AI Gateway~partialclaimed4/10

   Docs show performance-related controls (per-provider timeouts for fast failover, sort-by-performance metric, automatic caching) that imply attention to latency, but there is no direct benchmark or explicit claim quantifying the gateway's own added latency overhead. Missing for 10: published latency benchmarks or first-party/independent measurements showing the gateway adds minimal overhead versus direct provider calls, and no community corroboration of low added latency.

   • [claimed-docs] “You can set per-provider timeouts to trigger fast failover when a provider is slow to respond.”
   • [claimed-docs] “You can also use `sort` to rank providers by a performance or cost metric. The gateway sorts providers by the chosen metric and tries them i…”
   • [claimed-docs] “You can use `caching: 'auto'` in `providerOptions.gateway` to let AI Gateway automatically apply the appropriate caching strategy based on t…”
   • [claimed-docs] “You can use caching: 'auto' in providerOptions.gateway to let AI Gateway automatically apply the appropriate caching strategy based on the p…”

   proof ↗⚑ flag

Agent guardrails

1. ai-native userGive an autonomous agent its own key with budget and rate guardrails so it cannot run away on spend

   weight 2 · round to LiteLLM
   LiteLLM✓fullclaimed8/10

   LiteLLM's virtual key system explicitly supports per-key budgets (budget_duration, max_budget), rate limits (rpm_limit), and personal/team-scoped keys, directly enabling an autonomous agent to get its own key with spend and rate guardrails. Docs also show spend tracking by key/user/tag to monitor and enforce these limits. Missing for 10: independent/hands-on verification that budget caps hard-stop overspend in practice, and more detail on rate-limit enforcement behavior under load.

   • [claimed-docs] “Personal budgets: Create virtual keys without team_id for individual spending limits”
   • [claimed-docs] “Track Spend, and control model access via virtual keys for the proxy”
   • [claimed-docs] “Apply a budget across all calls on the proxy... budget_duration: 30d # (str) frequency of reset”
   • [claimed-docs] “"team_alias": "my-new-team_4", ... "rpm_limit": 99”
   • [claimed-docs] “Track spend for keys, users, and teams across 100+ LLMs.”
   • [claimed-docs] “pass tags to track spend by tags”
   • [claimed-docs] “Put your full AI stack behind one key. See who is driving spend, cap it before it runs, and send each request to the model that should handl…”

   proof ↗⚑ flag

   Vercel AI Gateway~partialcommunity6/10

   Docs confirm you can create per-API-key budgets and view spend/usage broken out by API key (docs-16, docs-12, docs-5), which supports giving an agent its own key with a spend cap. However, there is no explicit documentation of rate-limit guardrails (only spend budgets), and community evidence shows real friction around balance-based enforcement even with BYOK keys, so the 'cannot run away on spend' guarantee is only partially evidenced. Missing for 10: explicit per-key rate-limiting controls, clear docs on key-level rate guardrails (not just budget caps), and independent confirmation that budget enforcement reliably halts an autonomous agent before overspend.

   • [claimed-docs] “To cap how much your team, a project, an API key, or a team member can spend, set [budgets]”
   • [claimed-docs] “The Requests section displays summaries by project, API key, and a detailed log of all requests.”
   • [claimed-docs] “Spend monitoring. Monitor your spending across different providers”
   • [community] “If I bring my own Anthropic/OpenAI key and have usage available, why does Vercel's account balance determine whether my request executes? (V…”

   proof ↗⚑ flag

Budgets

1. platform engineerSet hard budgets and spend limits per key, team, or user

   weight 3 · round to LiteLLM
   LiteLLM✓fullclaimed9/10

   Docs explicitly cover per-key, per-team, and per-user budgets/spend tracking (virtual keys with cost tracking, personal budgets, team rpm_limit, budget_duration for resets, tag-based spend tracking) plus marketing copy on capping spend before it runs. Missing for 10: independent hands-on verification that hard limits actually block calls once exceeded (only docs, no community confirmation of enforcement behavior).

   • [claimed-docs] “Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI”
   • [claimed-docs] “Track spend for keys, users, and teams across 100+ LLMs.”
   • [claimed-docs] “Personal budgets: Create virtual keys without team_id for individual spending limits”
   • [claimed-docs] “Track Spend, and control model access via virtual keys for the proxy”
   • [claimed-docs] “Put your full AI stack behind one key. See who is driving spend, cap it before it runs, and send each request to the model that should handl…”
   • [claimed-docs] “pass tags to track spend by tags”
   • [claimed-docs] “Apply a budget across all calls on the proxy... budget_duration: 30d # (str) frequency of reset”
   • [claimed-docs] “"team_alias": "my-new-team_4", ... "rpm_limit": 99”

   proof ↗⚑ flag

   Vercel AI Gateway✓fullclaimed8/10

   Docs explicitly state budgets can be set to cap spend for a team, project, API key, or team member, directly matching the story, and spend monitoring/usage dashboards support enforcement. missing for 10: independent/hands-on confirmation that hard budgets actually block spend at the limit (community evidence only discusses BYOK balance blocking, not per-key/team/user budget enforcement specifics) and more detail on granularity/configuration UI.

   • [claimed-docs] “To cap how much your team, a project, an API key, or a team member can spend, set [budgets]”
   • [claimed-docs] “Spend monitoring. Monitor your spending across different providers”
   • [claimed-docs] “The Requests section displays summaries by project, API key, and a detailed log of all requests.”
   • [claimed-docs] “The Usage section displays four metrics to help you monitor your AI Gateway activity.”

   proof ↗⚑ flag

Spend tracking

1. platform engineerTrack spend per model, key, team, or user across all providers in one place

   weight 3 · round to LiteLLM
   LiteLLM✓fullclaimed9/10

   LiteLLM's docs directly document tracking spend per key, user, team, and by tags across 100+ providers via the proxy's cost tracking feature, plus budgets, virtual keys, and admin UI to view spend — matching the story closely. Missing for 10: independent hands-on verification of the cost dashboard's accuracy/UX and explicit per-model spend breakdown evidence beyond the general 'across 100+ LLMs' claim.

   • [claimed-docs] “Track spend for keys, users, and teams across 100+ LLMs.”
   • [claimed-docs] “Track Spend, and control model access via virtual keys for the proxy”
   • [claimed-docs] “pass tags to track spend by tags”
   • [claimed-docs] “Apply a budget across all calls on the proxy... budget_duration: 30d # (str) frequency of reset”
   • [claimed-docs] “Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI”
   • [claimed-docs] “Put your full AI stack behind one key. See who is driving spend, cap it before it runs, and send each request to the model that should handl…”

   proof ↗⚑ flag

   Vercel AI Gateway✓fullclaimed8/10

   Docs show request/usage summaries broken down by project and API key, budgets settable per team/project/API key/team member, and detailed logs filterable by model/provider/status with CSV/JSON export, directly matching the story's cost-tracking dimensions. Missing for 10: explicit per-user (not just per-team-member/API key) breakdowns and independent third-party confirmation of the dashboard's granularity in practice.

   • [claimed-docs] “Spend monitoring. Monitor your spending across different providers”
   • [claimed-docs] “The Requests section displays summaries by project, API key, and a detailed log of all requests.”
   • [claimed-docs] “To cap how much your team, a project, an API key, or a team member can spend, set [budgets]”
   • [claimed-docs] “open the Logs page, where you can search by request ID, filter by model, provider, or status code, follow requests live, and open one reques…”
   • [claimed-docs] “The Usage section displays four metrics to help you monitor your AI Gateway activity.”

   proof ↗⚑ flag

Byok

1. developerBring my own provider API keys and have the gateway use them for my traffic

   weight 2 · round to LiteLLM
   LiteLLM~partialclaimed6/10

   LiteLLM's core design lets developers call any provider through one interface and issue virtual keys that map to underlying provider credentials, with per-key spend/model access control ('Put your full AI stack behind one key', virtual key docs) — implying BYOK usage. However, the evidence pack lacks an explicit walkthrough of adding a personal provider API key to the proxy config and having it used for a developer's own traffic, and there is no independent confirmation of this specific flow. Missing for 10: explicit BYOK config example, independent/hands-on confirmation of per-developer key routing.

   • [claimed-docs] “Call any provider using the same completion() interface, with no API to re-learn for each one”
   • [claimed-docs] “Track Spend, and control model access via virtual keys for the proxy”
   • [claimed-docs] “Put your full AI stack behind one key. See who is driving spend, cap it before it runs, and send each request to the model that should handl…”
   • [claimed-docs] “Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI”

   proof ↗⚑ flag

   Vercel AI Gateway!disputedcontradicted5/10

   Vercel's docs clearly describe BYOK support — pass provider credentials per-request via providerOptions.gateway.byok, with no markup on tokens, and even fallback to system credentials on failure — suggesting bring-your-own-key traffic should run on the developer's own credentials (docs-6, docs-13, docs-19, docs-27). However, a concrete community report describes BYOK requests being blocked by Vercel's account balance reaching $0 even when the user's own provider key has available usage, directly contradicting the expectation that BYOK traffic is unaffected by platform account balance (comm-1). missing for 10: independent confirmation resolving the balance-dependency complaint, clearer documentation on how billing/balance interacts with BYOK requests.

   • [claimed-docs] “No markup on tokens. Tokens cost the same as they would from the provider directly, with zero markup, including with [Bring Your Own Key (BY…”
   • [claimed-docs] “If a query using your credentials fails, AI Gateway will retry the query with its system credentials to improve service availability.”
   • [claimed-docs] “No markup on tokens. Tokens cost the same as they would from the provider directly, with zero markup, including with Bring Your Own Key (BYO…”
   • [claimed-docs] “you can pass provider credentials on a per-request basis using the byok option in providerOptions.gateway. This is useful when you need to u…”
   • [community] “If I bring my own Anthropic/OpenAI key and have usage available, why does Vercel's account balance determine whether my request executes? (V…”

   proof ↗⚑ flag

Programmatic admin

1. ai-native userProvision gateways, keys, and budgets programmatically through an admin API

   weight 3 · round to LiteLLM
   LiteLLM!disputedcontradicted5/10

   Docs describe programmatic key/team/budget management (virtual keys, personal and team budgets, rpm_limit, team_alias via API/config) which supports the story's core claim, but no OpenAPI/swagger spec was found (probe shows 404s) and a community user reported concrete hands-on issues 'creating tokens for other users' via the proxy admin functions, directly contradicting the smoothness implied by the docs. Missing for 10: verified OpenAPI/admin-API reference, evidence of 'gateway' provisioning (multiple proxy instances) via API, and independent confirmation that key/budget creation works reliably at scale.

   • [claimed-docs] “Personal budgets: Create virtual keys without team_id for individual spending limits”
   • [claimed-docs] “Track Spend, and control model access via virtual keys for the proxy”
   • [claimed-docs] “Apply a budget across all calls on the proxy... budget_duration: 30d # (str) frequency of reset”
   • [claimed-docs] “"team_alias": "my-new-team_4", ... "rpm_limit": 99”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.litellm.ai/openapi.json, https://docs.litellm.ai/swagger.json, https://docs.litellm.ai/…”
   • [community] “'Does anyone know a good alternate project... LiteLLM has been getting worse and trying to get me to upgrade to a paid version. I also had i…”

   proof ↗⚑ flag

   Vercel AI Gateway—none0/10

   Docs describe setting budgets, API keys, and monitoring spend through the Vercel dashboard/CLI, and a general Vercel OpenAPI spec exists, but there is no evidence of a documented admin API specifically for programmatically provisioning AI Gateway resources, keys, or budgets. Missing for 10: explicit admin/API endpoints for creating gateways, generating keys, or setting budgets programmatically, and any docs or examples showing this workflow.

   • [claimed-docs] “To cap how much your team, a project, an API key, or a team member can spend, set [budgets]”
   • [claimed-docs] “The Requests section displays summaries by project, API key, and a detailed log of all requests.”
   • [probe] “PROBE openapi: HTTP 200 at https://vercel.com/openapi.json — contains "openapi" key”

   proof ↗⚑ flag

Virtual keys

1. platform engineerMint gateway-managed keys for teams and apps without exposing raw provider keys

   weight 3 · round to Vercel AI Gateway
   LiteLLM~partialcommunity7/10

   LiteLLM's proxy docs explicitly support minting scoped 'virtual keys' for teams/users/apps with budget and rate limits (litellm-docs-5, litellm-docs-6, litellm-docs-7, litellm-docs-8, litellm-docs-26), which is exactly the gateway-managed-key model that hides raw provider keys behind the proxy. However, one community report describes hands-on 'issues with creating tokens for other users' (litellm-comm-3), and there is no independent corroboration of the key-isolation guarantees or a walkthrough of the actual key-generation API beyond first-party docs. Missing for 10: independent/hands-on confirmation that key scoping reliably prevents provider-key leakage, and resolution of the reported token-creation friction.

   • [claimed-docs] “Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI”
   • [claimed-docs] “Track spend for keys, users, and teams across 100+ LLMs.”
   • [claimed-docs] “Personal budgets: Create virtual keys without team_id for individual spending limits”
   • [claimed-docs] “Track Spend, and control model access via virtual keys for the proxy”
   • [claimed-docs] “"team_alias": "my-new-team_4", ... "rpm_limit": 99”
   • [community] “'Does anyone know a good alternate project... LiteLLM has been getting worse and trying to get me to upgrade to a paid version. I also had i…”

   proof ↗⚑ flag

   Vercel AI Gateway✓fullclaimed7/10

   Docs show a single Gateway-managed API key (or OIDC token) replaces direct provider keys, with per-API-key usage tracking and budget caps that imply teams/apps can have distinct managed keys, and BYOK/system-credential fallback keeps raw provider secrets out of app code. Missing for 10: explicit dashboard walkthrough for creating/rotating multiple named keys per team or app, and documentation of per-key role/permission scoping.

   • [claimed-docs] “One key, hundreds of models. Access models from multiple providers with a single API key”
   • [claimed-docs] “The Requests section displays summaries by project, API key, and a detailed log of all requests.”
   • [claimed-docs] “To cap how much your team, a project, an API key, or a team member can spend, set [budgets]”
   • [claimed-docs] “If a query using your credentials fails, AI Gateway will retry the query with its system credentials to improve service availability.”
   • [claimed-docs] “For applications deployed on Vercel, OIDC tokens are automatically available as `VERCEL_OIDC_TOKEN`. No secrets to manage, no keys to rotate…”
   • [claimed-docs] “you can pass provider credentials on a per-request basis using the byok option in providerOptions.gateway. This is useful when you need to u…”

   proof ↗⚑ flag

Integrations

1. developerExport gateway logs and traces to my own observability stack

   weight 1 · round to LiteLLM
   LiteLLM✓fullclaimed8/10

   Docs explicitly confirm proxy logging integrations to external observability stacks (Langfuse, OpenTelemetry, GCS, S3, Azure Blob), plus a unique call_id for tracing requests across systems, directly matching the story of exporting logs/traces to one's own stack. Missing for 10: independent hands-on verification of these integrations working reliably, and more detail on trace format/OpenTelemetry span structure.

   • [claimed-docs] “Log Proxy input, output, and exceptions using: Langfuse, OpenTelemetry, GCS, s3, Azure (Blob) Buckets...”
   • [claimed-docs] “LiteLLM generates a unique call_id for each request. This call_id can be used to track the request across the system.”
   • [claimed-docs] “LiteLLM generates a unique `call_id` for each request. This `call_id` can be used to track the request across the system.”
   • [claimed-docs] “This will prevent the messages and responses from being logged to your logging provider, but request metadata - e.g. spend, will still be tr…”

   proof ↗⚑ flag

   Vercel AI Gateway~partialclaimed5/10

   The gateway provides a Logs UI with filtering and CSV/JSON export of request logs, which supports manual export of logs, but there's no evidence of native integration with external observability stacks (e.g., OpenTelemetry export, webhook/streaming to Datadog/Grafana, or trace export in standard formats like OTLP). missing for 10: OpenTelemetry/OTLP trace export, native integrations with third-party observability platforms, automated/streaming log export (vs manual CSV/JSON download), independent confirmation of export usability.

   • [claimed-docs] “open the Logs page, where you can search by request ID, filter by model, provider, or status code, follow requests live, and open one reques…”
   • [claimed-docs] “The Requests section displays summaries by project, API key, and a detailed log of all requests.”
   • [claimed-docs] “The Usage section displays four metrics to help you monitor your AI Gateway activity.”

   proof ↗⚑ flag

Logs

1. platform engineerInspect logged requests and responses with latency, token counts, and cost attached

   weight 3 · round to Vercel AI Gateway
   LiteLLM~partialclaimed6/10

   Docs show LiteLLM logs each request with a unique call_id, tracks spend/cost by key/user/team, and integrates with Langfuse, OpenTelemetry, S3, GCS, etc. for logging proxy input/output/exceptions, all viewable via the admin UI — a reasonable basis for inspecting requests with cost attached. However, the evidence never explicitly confirms that latency or token counts are captured/displayed alongside these logs, and there's no independent/hands-on confirmation of the inspection UI itself. Missing for 10: explicit documentation of per-request latency metrics, explicit token-count logging, and independent verification of the UI/log inspection experience.

   • [claimed-docs] “Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI”
   • [claimed-docs] “Track spend for keys, users, and teams across 100+ LLMs.”
   • [claimed-docs] “Log Proxy input, output, and exceptions using: Langfuse, OpenTelemetry, GCS, s3, Azure (Blob) Buckets...”
   • [claimed-docs] “LiteLLM generates a unique call_id for each request. This call_id can be used to track the request across the system.”
   • [claimed-docs] “pass tags to track spend by tags”

   proof ↗⚑ flag

   Vercel AI Gateway✓fullclaimed8/10

   Docs describe a Logs page with per-request detail, search by request ID, filtering by model/provider/status, and export as CSV/JSON, plus a Requests section with summaries by project/API key and a Usage section with metrics — covering latency, tokens, cost, and routing detail. Missing for 10: independent/hands-on confirmation that latency and cost are explicitly visible per logged request (only vendor docs cited), and no third-party corroboration of the observability UI.

   • [claimed-docs] “The Requests section displays summaries by project, API key, and a detailed log of all requests.”
   • [claimed-docs] “open the Logs page, where you can search by request ID, filter by model, provider, or status code, follow requests live, and open one reques…”
   • [claimed-docs] “The Usage section displays four metrics to help you monitor your AI Gateway activity.”
   • [claimed-docs] “Spend monitoring. Monitor your spending across different providers”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round drawn
   LiteLLM~partialprobed6/10

   LiteLLM's proxy exposes virtual keys, budgets, spend tracking, MCP server management, and routing/fallback config via REST API endpoints and CLI, and the admin UI is largely a wrapper over these same proxy APIs, suggesting strong API/UI parity. However, no OpenAPI spec was found at expected locations (404s), and some UI-specific flows like adding MCP servers are only documented via UI clicks rather than confirmed equivalent API calls, and no independent confirmation exists that every UI action has a documented API equivalent. Missing for 10: a published OpenAPI/swagger spec confirming full endpoint coverage, explicit API equivalents for every UI workflow (e.g., MCP server add via API), and independent/hands-on verification that API and UI are truly at parity.

   • [claimed-docs] “Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI”
   • [claimed-docs] “Track Spend, and control model access via virtual keys for the proxy”
   • [claimed-docs] “LiteLLM Proxy provides an MCP Gateway that allows you to use a fixed endpoint for all MCP tools and control MCP access by Key, Team.”
   • [claimed-docs] “On the LiteLLM UI, Navigate to "MCP Servers" and click "Add New MCP Server".”
   • [claimed-docs] “The number of worker processes to spin up (uvicorn, gunicorn, or Granian --workers).”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.litellm.ai/openapi.json, https://docs.litellm.ai/swagger.json, https://docs.litellm.ai/…”
   • [probe] “official CLI documented at https://docs.litellm.ai/docs/proxy/cli”

   proof ↗⚑ flag

   Vercel AI Gateway~partialprobed6/10

   The gateway is largely API-first: model access, provider routing, caching, BYOK, and even a keyless public models endpoint are confirmed to work programmatically (docs-1,2,9,15,25,27; probe-4). However, budgets, spend caps, and the detailed Requests/Logs dashboard are described only as UI features (docs-5,12,16,23,30) without documented API endpoints to set budgets or pull logs programmatically. Missing for 10: budgets/spend-cap management API, log/usage export API parity, team/key management API parity.

   • [claimed-docs] “One key, hundreds of models. Access models from multiple providers with a single API key”
   • [claimed-docs] “Unified API. Switch between providers and models with minimal code changes”
   • [claimed-docs] “You can use `order`, `only`, and `sort` in `providerOptions.gateway` to control which providers handle your requests, in what order, and how…”
   • [claimed-docs] “This endpoint requires no authentication and returns detailed information including model IDs, context windows, and pricing.”
   • [claimed-docs] “Query the models endpoint directly to get a JSON list of all available models with pricing and capabilities”
   • [claimed-docs] “you can pass provider credentials on a per-request basis using the byok option in providerOptions.gateway. This is useful when you need to u…”
   • [claimed-docs] “Spend monitoring. Monitor your spending across different providers”
   • [claimed-docs] “To cap how much your team, a project, an API key, or a team member can spend, set [budgets]”
   • [claimed-docs] “open the Logs page, where you can search by request ID, filter by model, provider, or status code, follow requests live, and open one reques…”
   • [probe] “PROBE keyless-endpoint: GET https://ai-gateway.vercel.sh/v1/models returned HTTP 200 application/json with an OpenAI-compatible {"object":"l…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round to Vercel AI Gateway
   LiteLLM—none0/10

   LiteLLM stores spend, logs, virtual-key and config data (self-hosted proxy with admin UI), so an explicit 'export all data in open formats' capability is a fair ask, but no evidence pack item documents any export/download feature, open data format, or data-portability tooling — only logging integrations to third-party sinks (Langfuse, S3, GCS) are mentioned, which is not the same as user-initiated full data export. missing for 10: documented export/download feature, open-format data export (CSV/JSON) of spend/logs/keys, any data-portability or 'leave the platform' guidance.

   • [claimed-docs] “Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI”
   • [claimed-docs] “Track spend for keys, users, and teams across 100+ LLMs.”
   • [claimed-docs] “Log Proxy input, output, and exceptions using: Langfuse, OpenTelemetry, GCS, s3, Azure (Blob) Buckets...”

   proof ↗⚑ flag

   Vercel AI Gateway~partialcommunity5/10

   Vercel documents that the Logs page lets users export request/usage data as CSV or JSON, and its OpenAI-compatible API means you can point clients elsewhere with minimal rewrite, reducing lock-in. However, there's no evidence of a comprehensive account/data export (billing history, configs, keys) beyond request logs, and BYOK behavior tied to Vercel balance (comm-1) suggests some platform dependency remains. Missing for 10: full account/config data export, evidence of exporting all historical usage/billing data, and confirmation that BYOK/API keys are fully portable without gateway dependency.

   • [claimed-docs] “open the Logs page, where you can search by request ID, filter by model, provider, or status code, follow requests live, and open one reques…”
   • [claimed-docs] “You can use existing OpenAI client libraries, switch to AI Gateway with a URL change, and keep your current tools and workflows without code…”
   • [claimed-docs] “Point your existing OpenAI or Anthropic SDK at AI Gateway. Same calls, no rewrites.”
   • [claimed-docs] “You can use the AI Gateway's Chat Completions API with existing tools and libraries like the OpenAI client libraries and AI SDK. Point your …”
   • [community] “If I bring my own Anthropic/OpenAI key and have usage available, why does Vercel's account balance determine whether my request executes? (V…”

   proof ↗⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round to LiteLLM
   LiteLLM~partialcommunity4/10

   The product's source is publicly hosted on GitHub (BerriAI/litellm), confirmed by community references to 'public repos' being vandalized, showing the codebase is openly readable. However, the evidence pack never states or cites an actual license (e.g., MIT/Apache) for the repository, so the 'open license' claim itself is unconfirmed. Missing for 10: explicit license documentation/citation, first-party statement of licensing terms.

   • [github] “Drop-in OpenAI compatibility — swap providers without rewriting your code”
   • [community] “The GitHub account of LiteLLM's Founder/CTO (krrishdholakia) appears to have been fully compromised, with public repos vandalized to say 'te…”

   proof ↗⚑ flag

   Vercel AI Gateway—none0/10

   Vercel AI Gateway is a closed, proprietary hosted service; no evidence in the pack points to any open-source repository or open license for its source code — all evidence is documentation and API endpoints, not source availability.

   ⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round to LiteLLM
   LiteLLM✓fullcommunity9/10

   LiteLLM explicitly ships a self-hosted LLM Gateway/Proxy (docs-5, docs-23) with configurable worker processes and CLI (docs-13/20), and this is corroborated by hands-on community use running it via Docker in a homelab (litellm-comm-6) and widespread adoption as a self-hosted gateway (litellm-comm-7). missing for 10: no independent benchmark or enterprise-scale self-hosting case study beyond a single homelab anecdote.

   • [claimed-docs] “Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI”
   • [claimed-docs] “Self-hosted [LLM Gateway (Proxy)](/docs/simple_proxy) with virtual keys, cost tracking, and an admin UI”
   • [claimed-docs] “The number of worker processes to spin up (uvicorn, gunicorn, or Granian --workers).”
   • [claimed-docs] “The number of worker processes to spin up (uvicorn, gunicorn, or Granian `--workers`).”
   • [community] “A user described running LiteLLM as a proxy in their homelab via the litellm/litellm docker image for local LLM gateway management, noting t…”
   • [community] “'That's a bad supply-chain attack, many folks use litellm as main gateway' — reflecting how widely used LiteLLM is as an LLM gateway.”

   proof ↗⚑ flag

   Vercel AI Gateway—none0/10

   Vercel AI Gateway is a hosted cloud service with no evidence of a self-hostable core product or open-source release; nothing in the docs suggests an on-prem or self-managed deployment option.

   ⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round drawn
   LiteLLM—none0/10

   LiteLLM is self-hostable (which lets users control physical data location by choosing their own deployment region), but no evidence pack item documents any explicit data residency/region-selection feature, region-pinning configuration, or compliance-region controls offered by the product itself.

   ⚑ flag

   Vercel AI Gateway—none0/10

   No evidence of region/data-residency controls or storage location choices in the AI Gateway docs; the pack covers routing, failover, pricing, and observability but nothing about where data is stored or processed. Missing for 10: any mention of regional data residency options, storage location controls, or compliance certifications addressing data locality.

   ⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   LiteLLM—none0/10

   The evidence shows LiteLLM can disable logging of prompts/responses to its own logging providers (litellm-docs-12), but nothing indicates it offers a mechanism to opt out of model-training use by the underlying LLM providers (e.g., passing zero-retention/no-train flags to OpenAI/Anthropic/etc.). As a gateway, this is a plausible capability, but no evidence supports it.

   • [claimed-docs] “This will prevent the messages and responses from being logged to your logging provider, but request metadata - e.g. spend, will still be tr…”

   proof ↗⚑ flag

   Vercel AI Gateway—none0/10

   No evidence in the pack addresses data-training opt-out policies, zero-retention agreements, or any explicit statement that prompts/outputs are excluded from model training; the docs focus on routing, pricing, and observability instead. Missing for 10: any privacy policy or training-opt-out statement, zero-data-retention terms, provider-level data-use guarantees.

   ⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round to LiteLLM
   LiteLLM~partialclaimed5/10

   LiteLLM lets admins disable logging of request/response content to reduce data retention while still tracking metadata (litellm-docs-12), and being self-hosted gives users infrastructure-level control over where data lives. However, there is no documented explicit data-deletion API, retention-period settings, or data lifecycle/export controls in the evidence pack. Missing for 10: explicit deletion/purge API, configurable retention windows, documented data lifecycle policy, independent confirmation these logging controls work as described.

   • [claimed-docs] “This will prevent the messages and responses from being logged to your logging provider, but request metadata - e.g. spend, will still be tr…”
   • [claimed-docs] “Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI”

   proof ↗⚑ flag

   Vercel AI Gateway—none0/10

   No evidence in the pack addresses data retention policies, log/data deletion controls, or retention configuration for requests/prompts processed through AI Gateway; observability docs mention logging/exporting requests but nothing about retention duration or deletion mechanisms.

   ⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   LiteLLM—none0/10

   The evidence covers logging integrations (Langfuse, OTel, S3, etc.) and a way to disable message/response logging to those *external* logging providers (litellm-docs-12), but nothing in the pack addresses opting out of LiteLLM's own product telemetry/usage tracking sent back to the vendor. As a self-hosted gateway this axis plausibly applies, but there is no documented telemetry opt-out flag or setting in the evidence pack. Missing for 10: any mention of a telemetry/usage-tracking toggle (e.g., an env var or config flag disabling anonymized usage reporting to the vendor), independent confirmation that no phone-home occurs.

   • [claimed-docs] “This will prevent the messages and responses from being logged to your logging provider, but request metadata - e.g. spend, will still be tr…”
   • [claimed-docs] “Log Proxy input, output, and exceptions using: Langfuse, OpenTelemetry, GCS, s3, Azure (Blob) Buckets...”

   proof ↗⚑ flag

   Vercel AI Gateway—none0/10

   No evidence of any telemetry opt-out or privacy/data-collection controls; the gateway's observability features are about logging usage for the account owner, not about disabling tracking. Missing for 10: any documented telemetry opt-out setting, privacy policy on data retention/training use, or way to disable request logging.

   ⚑ flag

Fallbacks

1. platform engineerConfigure automatic fallback to another model or provider when one fails

   weight 3 · round drawn
   LiteLLM✓fullcommunity9/10

   LiteLLM's proxy reliability docs explicitly describe configuring fallbacks so a failing model/provider automatically fails over to a healthy backup, including model-group fallback chains and content-policy-specific fallback handling, corroborated by a maintainer's stated motivation for reliable model switching across providers. Missing for 10: independent hands-on validation of fallback behavior in production beyond docs/maintainer statements.

   • [claimed-docs] “If a call fails after num_retries, LiteLLM falls back to another model group, so a failing model or provider automatically fails over to a h…”
   • [claimed-docs] “so a failing model or provider automatically fails over to a healthy backup”
   • [claimed-docs] “The request to `model="zephyr-beta"` will fail... litellm proxy will loop through all the model_groups specified in `fallbacks=["gpt-3.5-tur…”
   • [claimed-docs] “content_policy_fallbacks: For litellm.ContentPolicyViolationError - LiteLLM maps content policy violation errors across providers”
   • [community] “Show HN launch of LiteLLM Proxy Server: praised as 'super compelling' with maintainers noting motivation was avoiding messy 100-line if/else…”

   proof ↗⚑ flag

   Vercel AI Gateway✓fullclaimed9/10

   Docs explicitly describe automatic retries to other providers on failure, plus fine-grained control via `order`, `only`, and `sort` in providerOptions.gateway, per-provider timeouts for fast failover, and even BYOK-to-system-credential fallback — directly matching the fallback/routing-resilience story. This is well-documented first-party capability with concrete configuration options, though missing for 10: independent hands-on verification that fallback behaves correctly in production scenarios.

   • [claimed-docs] “High reliability. Automatically retries requests to other providers if one fails”
   • [claimed-docs] “You can use `order`, `only`, and `sort` in `providerOptions.gateway` to control which providers handle your requests, in what order, and how…”
   • [claimed-docs] “You can set per-provider timeouts to trigger fast failover when a provider is slow to respond.”
   • [claimed-docs] “If a query using your credentials fails, AI Gateway will retry the query with its system credentials to improve service availability.”
   • [claimed-docs] “You can use order, only, and sort in providerOptions.gateway to control which providers handle your requests, in what order, and how they ar…”
   • [claimed-docs] “AI Gateway routes requests, manages fallbacks and budgets, monitors usage, and connects supported coding agents.”

   proof ↗⚑ flag

Load balancing

1. platform engineerLoad-balance traffic across providers, deployments, or keys by weight, latency, or cost

   weight 2 · round to Vercel AI Gateway
   LiteLLM~partialclaimed5/10

   Docs confirm load-balancing across multiple deployments and automatic failover/fallback to healthy backups, which covers reliability-based routing, but the evidence pack never cites explicit weight-based, latency-based, or cost-based routing strategies by name (LiteLLM's known routing options aren't documented here). missing for 10: explicit documentation of weight-based routing strategy, latency-based routing strategy, and cost-based routing strategy configuration.

   • [claimed-docs] “Load-balance across multiple deployments (e.g. Azure/OpenAI)”
   • [claimed-docs] “If a call fails after num_retries, LiteLLM falls back to another model group, so a failing model or provider automatically fails over to a h…”
   • [claimed-docs] “so a failing model or provider automatically fails over to a healthy backup”
   • [claimed-docs] “The request to `model="zephyr-beta"` will fail... litellm proxy will loop through all the model_groups specified in `fallbacks=["gpt-3.5-tur…”
   • [claimed-docs] “content_policy_fallbacks: For litellm.ContentPolicyViolationError - LiteLLM maps content policy violation errors across providers”

   proof ↗⚑ flag

   Vercel AI Gateway~partialclaimed6/10

   Docs confirm ordering/sorting by performance or cost metric (`order`, `only`, `sort`), per-provider timeouts for fast failover, and automatic retries/fallbacks to other providers — covering latency- and cost-based routing and failover. However, explicit weight-based load balancing across providers/deployments/keys is not documented, and there's no evidence of key-level load balancing beyond BYOK fallback to system credentials. missing for 10: explicit weighted load-balancing configuration, per-key routing/balancing control, independent verification of routing behavior under load.

   • [claimed-docs] “You can use `order`, `only`, and `sort` in `providerOptions.gateway` to control which providers handle your requests, in what order, and how…”
   • [claimed-docs] “You can set per-provider timeouts to trigger fast failover when a provider is slow to respond.”
   • [claimed-docs] “You can also use `sort` to rank providers by a performance or cost metric. The gateway sorts providers by the chosen metric and tries them i…”
   • [claimed-docs] “High reliability. Automatically retries requests to other providers if one fails”
   • [claimed-docs] “If a query using your credentials fails, AI Gateway will retry the query with its system credentials to improve service availability.”

   proof ↗⚑ flag

Policy routing

1. ai-native userMy agent can switch models mid-task by policy — cost, capability, or availability — through gateway routing rules

   weight 3 · round to LiteLLM
   LiteLLM✓fullcommunity7/10

   LiteLLM's proxy documents concrete routing policies matching the story: load balancing across deployments, automatic fallback to a healthy backup model/provider on failure, and content-policy-specific fallbacks mapped across providers — all confirmed by community commentary on its origin as a fix for manual multi-provider if/else logic. Missing for 10: explicit cost-triggered routing (switching to cheaper model based on budget threshold) and any hands-on demonstration of live mid-task switching rather than static failover/fallback configuration.

   • [claimed-docs] “Load-balance across multiple deployments (e.g. Azure/OpenAI)”
   • [claimed-docs] “If a call fails after num_retries, LiteLLM falls back to another model group, so a failing model or provider automatically fails over to a h…”
   • [claimed-docs] “so a failing model or provider automatically fails over to a healthy backup”
   • [claimed-docs] “content_policy_fallbacks: For litellm.ContentPolicyViolationError - LiteLLM maps content policy violation errors across providers”
   • [claimed-docs] “The request to `model="zephyr-beta"` will fail... litellm proxy will loop through all the model_groups specified in `fallbacks=["gpt-3.5-tur…”
   • [community] “Show HN launch of LiteLLM Proxy Server: praised as 'super compelling' with maintainers noting motivation was avoiding messy 100-line if/else…”

   proof ↗⚑ flag

   Vercel AI Gateway~partialcommunity7/10

   Docs show routing rules via providerOptions.gateway (order, only, sort, timeouts) that let requests fail over across providers by availability/latency, plus cost/spend visibility and budgets, which supports policy-driven switching. However, there's no explicit evidence of dynamic mid-task capability-based switching or agent-driven runtime policy changes beyond static config, and community evidence flags BYOK requests being blocked once account balance hits $0, undermining pure availability-based routing in some cases. missing for 10: explicit support for capability-based (not just cost/availability) routing rules, evidence of an agent dynamically changing models mid-task at runtime, and independent confirmation that failover/ordering works reliably in production.

   • [claimed-docs] “You can use `order`, `only`, and `sort` in `providerOptions.gateway` to control which providers handle your requests, in what order, and how…”
   • [claimed-docs] “You can set per-provider timeouts to trigger fast failover when a provider is slow to respond.”
   • [claimed-docs] “You can also use `sort` to rank providers by a performance or cost metric. The gateway sorts providers by the chosen metric and tries them i…”
   • [claimed-docs] “High reliability. Automatically retries requests to other providers if one fails”
   • [claimed-docs] “To cap how much your team, a project, an API key, or a team member can spend, set [budgets]”
   • [community] “If I bring my own Anthropic/OpenAI key and have usage available, why does Vercel's account balance determine whether my request executes? (V…”

   proof ↗⚑ flag

Rate limits

1. platform engineerSmooth provider rate limits by spreading traffic across keys and queuing or throttling requests

   weight 2 · round to LiteLLM
   LiteLLM✓fullclaimed7/10

   LiteLLM's Router explicitly supports load-balancing across multiple deployments/keys (e.g., multiple Azure/OpenAI keys) and virtual keys carry per-key rpm_limit fields, directly enabling spreading traffic across provider keys to smooth rate limits, with retries/fallbacks for failures. Missing for 10: explicit documentation of a request queuing/throttling mechanism beyond rpm_limit and load balancing, and independent hands-on confirmation that this smooths real provider rate-limit errors under load.

   • [claimed-docs] “Load-balance across multiple deployments (e.g. Azure/OpenAI)”
   • [claimed-docs] “"team_alias": "my-new-team_4", ... "rpm_limit": 99”
   • [claimed-docs] “If a call fails after num_retries, LiteLLM falls back to another model group, so a failing model or provider automatically fails over to a h…”
   • [claimed-docs] “so a failing model or provider automatically fails over to a healthy backup”
   • [claimed-docs] “The request to `model="zephyr-beta"` will fail... litellm proxy will loop through all the model_groups specified in `fallbacks=["gpt-3.5-tur…”

   proof ↗⚑ flag

   Vercel AI Gateway~partialclaimed4/10

   The gateway supports failover/retry across providers, per-provider timeouts for fast failover, and ordering/sorting of providers (docs-3, docs-9, docs-10, docs-13, docs-20, docs-29), which can mitigate a single provider's rate limits by shifting traffic elsewhere. However, there's no documented mechanism for spreading load across multiple API keys for the same provider, nor any explicit request queuing/throttling feature to smooth bursts against rate limits. Missing for 10: multi-key load balancing, explicit request queuing/throttling controls, and any rate-limit-specific documentation or independent verification.

   • [claimed-docs] “High reliability. Automatically retries requests to other providers if one fails”
   • [claimed-docs] “You can use `order`, `only`, and `sort` in `providerOptions.gateway` to control which providers handle your requests, in what order, and how…”
   • [claimed-docs] “You can set per-provider timeouts to trigger fast failover when a provider is slow to respond.”
   • [claimed-docs] “If a query using your credentials fails, AI Gateway will retry the query with its system credentials to improve service availability.”
   • [claimed-docs] “You can also use `sort` to rank providers by a performance or cost metric. The gateway sorts providers by the chosen metric and tries them i…”

   proof ↗⚑ flag

Retries

1. platform engineerSet automatic retry policies for transient provider errors

   weight 2 · round to Vercel AI Gateway
   LiteLLM✓fullclaimed7/10

   Docs explicitly describe a `num_retries` setting on the proxy, after which LiteLLM falls back to another model/group, directly supporting configurable retry policies for transient failures (docs-4, docs-21, docs-22). Missing for 10: detailed backoff/jitter configuration, granular retry-per-error-type mapping (only content policy fallback is separately documented), and independent hands-on verification of retry behavior under real transient provider errors.

   • [claimed-docs] “If a call fails after num_retries, LiteLLM falls back to another model group, so a failing model or provider automatically fails over to a h…”
   • [claimed-docs] “so a failing model or provider automatically fails over to a healthy backup”
   • [claimed-docs] “The request to `model="zephyr-beta"` will fail... litellm proxy will loop through all the model_groups specified in `fallbacks=["gpt-3.5-tur…”
   • [claimed-docs] “content_policy_fallbacks: For litellm.ContentPolicyViolationError - LiteLLM maps content policy violation errors across providers”

   proof ↗⚑ flag

   Vercel AI Gateway✓fullclaimed8/10

   Docs explicitly describe automatic failover/retry to other providers on failure (docs-3), configurable provider ordering/fallback with `order`, `only`, `sort` (docs-9, docs-29), per-provider timeouts to trigger fast failover (docs-10), and even automatic retry with system credentials when BYOK fails (docs-13). This directly covers automatic retry policies for transient provider errors. Missing for 10: no independent/hands-on confirmation of retry behavior in production, and no explicit documentation of configurable retry counts/backoff intervals distinct from provider ordering.

   • [claimed-docs] “High reliability. Automatically retries requests to other providers if one fails”
   • [claimed-docs] “You can use `order`, `only`, and `sort` in `providerOptions.gateway` to control which providers handle your requests, in what order, and how…”
   • [claimed-docs] “You can set per-provider timeouts to trigger fast failover when a provider is slow to respond.”
   • [claimed-docs] “If a query using your credentials fails, AI Gateway will retry the query with its system credentials to improve service availability.”
   • [claimed-docs] “You can also use `sort` to rank providers by a performance or cost metric. The gateway sorts providers by the chosen metric and tries them i…”

   proof ↗⚑ flag

Streaming

1. developerStream token-by-token responses through the gateway from any provider

   weight 3 · round to LiteLLM
   LiteLLM✓fullcommunity7/10

   LiteLLM's unified completion() interface and consistent output format across providers is well documented, and a maintainer explicitly confirmed the proxy 'supports streaming and function-calling in the same way as the openai-python SDK' across 50+ providers, matching the streaming-tools story directly. Missing for 10: a dedicated first-party streaming docs page/code snippet (e.g., stream=True) and independent hands-on confirmation beyond the single maintainer HN comment.

   • [community] “LiteLLM maintainer confirmed proxy supports streaming and function-calling in the same way as the openai-python SDK, and added Ollama integr…”
   • [claimed-docs] “Call any provider using the same completion() interface, with no API to re-learn for each one”
   • [claimed-docs] “Every response follows the OpenAI Chat Completions format, regardless of provider.”
   • [claimed-docs] “Call any provider using the same `completion()` interface, with no API to re-learn for each one”
   • [claimed-docs] “Consistent output format regardless of which provider or model you use”
   • [community] “Show HN launch of LiteLLM Proxy Server: praised as 'super compelling' with maintainers noting motivation was avoiding messy 100-line if/else…”

   proof ↗⚑ flag

   Vercel AI Gateway~partialclaimed5/10

   The gateway advertises full OpenAI/Anthropic SDK compatibility and AI SDK integration (docs-7, docs-18, docs-28, docs-31), and since streaming is a standard capability of those client libraries, it is implicitly supported when pointing existing clients at the gateway's base URL. However, no evidence explicitly documents streaming/token-by-token responses, SSE support, or shows this working across all providers. Missing for 10: explicit docs on streaming endpoints/parameters, hands-on confirmation of token-by-token streaming across multiple providers, and any independent verification.

   • [claimed-docs] “You can use existing OpenAI client libraries, switch to AI Gateway with a URL change, and keep your current tools and workflows without code…”
   • [claimed-docs] “Point your existing OpenAI or Anthropic SDK at AI Gateway. Same calls, no rewrites.”
   • [claimed-docs] “You can use the AI Gateway's Chat Completions API with existing tools and libraries like the OpenAI client libraries and AI SDK. Point your …”
   • [claimed-docs] “Move existing OpenAI, Anthropic, and AI SDK integrations to AI Gateway with a base URL swap.”

   proof ↗⚑ flag

Tool calling

1. developerRequest structured JSON-schema outputs across providers

   weight 1 · round drawn
   LiteLLM—none0/10

   The evidence pack covers LiteLLM's unified completion() interface, consistent OpenAI-format responses, retries/fallbacks, and proxy features, but nowhere mentions structured/JSON-schema output support (e.g. response_format, json_schema mode) across providers. This is a fair axis for a multi-provider LLM gateway, but no citation substantiates the specific capability.

   ⚑ flag

   Vercel AI Gateway—none0/10

   The evidence describes unified APIs, OpenAI-compatibility, provider routing, embeddings, and observability, but nowhere mentions structured JSON-schema/response_format output support or how it is normalized across different providers. Missing for 10: any documentation of JSON-schema/structured-output support, cross-provider schema normalization, or examples using response_format/json_schema.

   ⚑ flag

2. developerMake tool and function calls across different providers with a consistent schema

   weight 3 · round drawn
   LiteLLM✓fullcommunity8/10

   LiteLLM documents a unified completion() interface that normalizes tool/function calls and outputs to the OpenAI Chat Completions schema across 100+ providers, and a maintainer confirms function-calling parity with the openai-python SDK. missing for 10: independent hands-on verification of tool-call schema consistency across many specific providers, and no dedicated docs excerpt specifically on function-calling parameter translation edge cases.

   • [claimed-docs] “Call any provider using the same completion() interface, with no API to re-learn for each one”
   • [claimed-docs] “Every response follows the OpenAI Chat Completions format, regardless of provider.”
   • [claimed-docs] “Call any provider using the same `completion()` interface, with no API to re-learn for each one”
   • [claimed-docs] “Consistent output format regardless of which provider or model you use”
   • [github] “Drop-in OpenAI compatibility — swap providers without rewriting your code”
   • [community] “LiteLLM maintainer confirmed proxy supports streaming and function-calling in the same way as the openai-python SDK, and added Ollama integr…”

   proof ↗⚑ flag

   Vercel AI Gateway✓fullprobed8/10

   Vercel AI Gateway supports a unified API/schema (OpenAI-compatible and AI SDK) that lets developers call tools/functions across many providers with consistent code, including an explicit ToolLoopAgent example with tool calling, provider routing/fallback, and no-rewrite compatibility with existing OpenAI/Anthropic clients. missing for 10: independent hands-on verification of cross-provider tool-call schema consistency (e.g., confirming tool-calling behaves identically across Anthropic/OpenAI/other models) and any documented edge cases or limitations in function-calling parity across providers.

   • [claimed-docs] “Unified API. Switch between providers and models with minimal code changes”
   • [claimed-docs] “You can use existing OpenAI client libraries, switch to AI Gateway with a URL change, and keep your current tools and workflows without code…”
   • [claimed-docs] “Point your existing OpenAI or Anthropic SDK at AI Gateway. Same calls, no rewrites.”
   • [claimed-docs] “const agent = new ToolLoopAgent({ model: 'anthropic/claude-sonnet-5', tools: { getWeather: tool({”
   • [claimed-docs] “You can use the AI Gateway's Chat Completions API with existing tools and libraries like the OpenAI client libraries and AI SDK. Point your …”
   • [probe] “PROBE keyless-endpoint: GET https://ai-gateway.vercel.sh/v1/models returned HTTP 200 application/json with an OpenAI-compatible {"object":"l…”

   proof ↗⚑ flag

Catalog

1. developerBrowse or query a catalog of available models with pricing and context-window metadata

   weight 2 · round to Vercel AI Gateway
   LiteLLM—none0/10

   While LiteLLM claims support for 100+ LLMs and tracks spend/cost, the evidence pack contains no mention of a browsable/queryable catalog listing models with pricing and context-window metadata (e.g., no model_cost table, /model/info endpoint, or docs page referencing context window sizes). Cost-tracking docs (litellm-docs-6) cover spend tracking, not a metadata catalog.

   ⚑ flag

   Vercel AI Gateway✓fullprobed9/10

   Docs explicitly describe a models endpoint returning model IDs, context windows, and pricing with no authentication required, and a probe independently confirms the live unauthenticated /v1/models endpoint returns a full JSON model catalog. This directly satisfies browsing/querying a model catalog with pricing and context-window metadata. Missing for 10: no evidence of a dedicated visual browsing UI/dashboard for the catalog beyond the API and docs page.

   • [claimed-docs] “This endpoint requires no authentication and returns detailed information including model IDs, context windows, and pricing.”
   • [claimed-docs] “Query the models endpoint directly to get a JSON list of all available models with pricing and capabilities”
   • [probe] “PROBE keyless-endpoint: GET https://ai-gateway.vercel.sh/v1/models returned HTTP 200 application/json with an OpenAI-compatible {"object":"l…”

   proof ↗⚑ flag

Compatibility

1. developerPoint existing OpenAI-compatible code at the gateway by changing only the base URL and key

   weight 3 · round to Vercel AI Gateway
   LiteLLM✓fullcommunity8/10

   LiteLLM's docs and GitHub explicitly advertise drop-in OpenAI compatibility with consistent Chat Completions format across providers, and the self-hosted proxy exposes a single gateway endpoint with virtual keys, matching the exact story of only swapping base URL and key. Community evidence (Show HN, HN threads) corroborates real-world use as a drop-in gateway for existing OpenAI-style code. Missing for 10: independent hands-on verification of literal 'change only base_url+key' migration and no explicit code diff example in the pack.

   • [claimed-docs] “Call any provider using the same completion() interface, with no API to re-learn for each one”
   • [claimed-docs] “Every response follows the OpenAI Chat Completions format, regardless of provider.”
   • [github] “Drop-in OpenAI compatibility — swap providers without rewriting your code”
   • [claimed-docs] “Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI”
   • [claimed-docs] “Self-hosted [LLM Gateway (Proxy)](/docs/simple_proxy) with virtual keys, cost tracking, and an admin UI”
   • [community] “Show HN launch of LiteLLM Proxy Server: praised as 'super compelling' with maintainers noting motivation was avoiding messy 100-line if/else…”
   • [community] “LiteLLM maintainer confirmed proxy supports streaming and function-calling in the same way as the openai-python SDK, and added Ollama integr…”

   proof ↗⚑ flag

   Vercel AI Gateway✓fullprobed9/10

   Vercel explicitly documents OpenAI-compatible Chat Completions API where developers just swap the base URL and key with existing OpenAI SDKs, with no code rewrites, and a probe confirms the gateway serves an OpenAI-compatible /v1/models JSON response. Community evidence around BYOK balance behavior is unrelated to base-URL compatibility, so it doesn't dispute this. Missing for 10: independent third-party hands-on confirmation of a full migration beyond Vercel's own docs/probes.

   • [claimed-docs] “You can use existing OpenAI client libraries, switch to AI Gateway with a URL change, and keep your current tools and workflows without code…”
   • [claimed-docs] “Point your existing OpenAI or Anthropic SDK at AI Gateway. Same calls, no rewrites.”
   • [claimed-docs] “You can use the AI Gateway's Chat Completions API with existing tools and libraries like the OpenAI client libraries and AI SDK. Point your …”
   • [claimed-docs] “Move existing OpenAI, Anthropic, and AI SDK integrations to AI Gateway with a base URL swap.”
   • [probe] “PROBE keyless-endpoint: GET https://ai-gateway.vercel.sh/v1/models returned HTTP 200 application/json with an OpenAI-compatible {"object":"l…”

   proof ↗⚑ flag

One endpoint

1. developerCall many model providers through one consistent API

   weight 3 · round to Vercel AI Gateway
   LiteLLM✓fullcommunity8/10

   LiteLLM's core value proposition—calling 100+ providers through one OpenAI-compatible completion() interface with consistent output format—is well documented (litellm-docs-1, litellm-docs-2, litellm-docs-16, litellm-docs-17, litellm-gh-1) and corroborated by independent Show HN community feedback praising it as a way to standardize disparate AI APIs and quickly add new providers (litellm-comm-12, litellm-comm-13, litellm-comm-14, litellm-comm-15). Some community sentiment flags proxy flakiness/outages, but that concerns operational reliability, not whether the unified-API abstraction itself works. Missing for 10: no independent benchmark verifying identical output format across all supported providers, and no public OpenAPI spec was found for the proxy (litellm-probe-3).

   • [claimed-docs] “Call any provider using the same completion() interface, with no API to re-learn for each one”
   • [claimed-docs] “Every response follows the OpenAI Chat Completions format, regardless of provider.”
   • [claimed-docs] “Call any provider using the same `completion()` interface, with no API to re-learn for each one”
   • [claimed-docs] “Consistent output format regardless of which provider or model you use”
   • [github] “Drop-in OpenAI compatibility — swap providers without rewriting your code”
   • [community] “Show HN launch of LiteLLM Proxy Server: praised as 'super compelling' with maintainers noting motivation was avoiding messy 100-line if/else…”
   • [community] “LiteLLM maintainer confirmed proxy supports streaming and function-calling in the same way as the openai-python SDK, and added Ollama integr…”
   • [community] “Show HN launch of the original LiteLLM library; a user praised it: 'This is amazing. Really needed something like this to standardize all my…”
   • [community] “Maintainer distinguished LiteLLM from a similar library (llm-client-sdk) by stating 'while your library looks really powerful my goal with L…”

   proof ↗⚑ flag

   Vercel AI Gateway✓fullprobed9/10

   Extensive first-party docs describe a single unified API/key across hundreds of providers, OpenAI-compatible client support, and provider routing/fallback controls, and a live probe confirms the OpenAI-compatible /v1/models endpoint works unauthenticated. There is a minor community-reported caveat about BYOK requests being blocked when Vercel balance hits $0, which slightly limits the 'bring your own key' flow but doesn't contradict the core unified-API story. Missing for 10: independent hands-on developer confirmation of multi-provider switching in production beyond docs and one probe.

   • [claimed-docs] “One key, hundreds of models. Access models from multiple providers with a single API key”
   • [claimed-docs] “Unified API. Switch between providers and models with minimal code changes”
   • [claimed-docs] “You can use existing OpenAI client libraries, switch to AI Gateway with a URL change, and keep your current tools and workflows without code…”
   • [claimed-docs] “Point your existing OpenAI or Anthropic SDK at AI Gateway. Same calls, no rewrites.”
   • [claimed-docs] “You can use the AI Gateway's Chat Completions API with existing tools and libraries like the OpenAI client libraries and AI SDK. Point your …”
   • [probe] “PROBE keyless-endpoint: GET https://ai-gateway.vercel.sh/v1/models returned HTTP 200 application/json with an OpenAI-compatible {"object":"l…”
   • [community] “If I bring my own Anthropic/OpenAI key and have usage available, why does Vercel's account balance determine whether my request executes? (V…”

   proof ↗⚑ flag