LiteLLM vs Kong AI Gateway

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round drawn
   LiteLLM✓fullprobed8/10

   A direct probe confirms LiteLLM serves a valid llms.txt file at https://docs.litellm.ai/llms.txt returning HTTP 200 with an agent-readable overview and links, exactly matching the story's request. Missing for 10: no independent/community corroboration of agents actually consuming this file, and the related docs-as-markdown endpoint (.md) returns 404, suggesting the agent-oriented docs surface is only partially built out.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.litellm.ai/llms.txt # https://docs.litellm.ai/ llms.txt - [LiteLLM Overview](https://docs.litellm.…”
   • [probe] “PROBE docs-md: HTTP 404 at https://docs.litellm.ai/docs/.md”

   proof ↗⚑ flag

   Kong AI Gateway✓fullprobed8/10

   A probe confirms developer.konghq.com/llms.txt returns HTTP 200 with a valid llms.txt manifest listing Kong's docs (including AI Gateway), and individual doc pages are also served as .md files (e.g. ai-providers.md, a2a.md), making the docs directly agent-consumable. missing for 10: no explicit vendor statement instructing agents to use llms.txt, and the generic /.md probe for the ai-gateway index returned 404, showing coverage is per-page rather than fully uniform.

   • [probe] “PROBE llms.txt: HTTP 200 at https://developer.konghq.com/llms.txt # developer.konghq.com > Kong's developer documentation for Kong Gateway,…”
   • [claimed-docs] “The core of AI Gateway is the ability to serve AI Models from various AI Model Providers via a provider-agnostic API.”
   • [probe] “PROBE docs-md: HTTP 404 at https://developer.konghq.com/ai-gateway/.md”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round drawn
   LiteLLM~partialprobed6/10

   LiteLLM is a Python SDK and proxy server with a documented CLI (worker processes, uvicorn/gunicorn/Granian) that can run headlessly in scripts or containers, and community evidence shows it running via Docker in unattended homelab/gateway setups. However there is no explicit CI/CD pipeline example, GitHub Actions integration, or automation-specific documentation in the evidence pack. Missing for 10: explicit CI pipeline examples, automation/scripting docs beyond CLI flags, evidence of non-interactive auth/config for CI environments.

   • [claimed-docs] “The number of worker processes to spin up (uvicorn, gunicorn, or Granian --workers).”
   • [claimed-docs] “The number of worker processes to spin up (uvicorn, gunicorn, or Granian `--workers`).”
   • [probe] “official CLI documented at https://docs.litellm.ai/docs/proxy/cli”
   • [community] “A user described running LiteLLM as a proxy in their homelab via the litellm/litellm docker image for local LLM gateway management, noting t…”
   • [claimed-docs] “Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI”

   proof ↗⚑ flag

   Kong AI Gateway~partialprobed6/10

   Kong AI Gateway exposes a CLI (kongctl) and Konnect APIs (list-ai-gateways, config conversion) plus a quickstart script, implying it can be driven headlessly and scripted rather than only via UI. However there is no explicit CI or CD pipeline example, declarative config-as-code workflow, or GitOps documentation demonstrating automated headless operation in CI. Missing for 10: explicit CI or CD pipeline documentation, dedicated headless-mode guide, independent CI usage report.

   • [claimed-docs] “list-ai-gateways summary: List AI Gateways description: Returns a list of AI Gateways in the organization.”
   • [claimed-docs] “walks you through migrating an existing configuration using the kongctl AI Gateway conversion extension”
   • [claimed-docs] “You can use the quickstart script to get a demo instance of AI Gateway running almost instantly.”
   • [probe] “official CLI documented at https://developer.konghq.com/kongctl/”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round to LiteLLM
   LiteLLM✓fullclaimed8/10

   LiteLLM Proxy has a documented MCP Gateway feature letting users add MCP servers via the UI and use a fixed endpoint to expose their tools, controllable by key/team — directly matching the story. missing for 10: independent/hands-on corroboration of MCP tool usage beyond first-party docs, and detail on tool-calling behavior once connected.

   • [claimed-docs] “LiteLLM Proxy provides an MCP Gateway that allows you to use a fixed endpoint for all MCP tools and control MCP access by Key, Team.”
   • [claimed-docs] “On the LiteLLM UI, Navigate to "MCP Servers" and click "Add New MCP Server".”

   proof ↗⚑ flag

   Kong AI Gateway~partialclaimed6/10

   Kong AI Gateway documents an 'AI MCP Proxy' capability that explicitly can front upstream MCP servers (in addition to converting REST APIs into MCP tools), meaning it can plug in external MCP servers so the traffic/tools flow through the gateway. However, the evidence is a single brief concept mention with no detailed configuration walkthrough, no example of registering a specific upstream MCP server, and no independent/hands-on corroboration. Missing for 10: a full docs page on connecting/registering upstream MCP servers, concrete config examples, and third-party verification.

   • [claimed-docs] “AI MCP Proxy bridges Kong-managed Services to the Model Context Protocol, converting REST APIs into MCP tools or fronting upstream MCP serve…”
   • [claimed-docs] “Turn any API into an MCP server using the AI MCP Server entity. This approach does not require an LLM and provides full control over product…”
   • [claimed-docs] “Govern LLM, MCP, and agent-to-agent (A2A) traffic with the same Kong AI Gateway.”

   proof ↗⚑ flag

4. ai-native userConnect an agent via an official MCP server

   weight 3 · round to Kong AI Gateway
   LiteLLM✓fullclaimed7/10

   LiteLLM Proxy ships an official MCP Gateway feature exposing a fixed endpoint for all configured MCP tools, with access controlled by Key/Team, and a UI flow to add MCP servers — this is exactly the 'connect an agent via an official MCP server' capability for a gateway product. Missing for 10: independent/hands-on confirmation that external agents (e.g. Claude, Cursor) successfully connect via this gateway, and deeper documentation of the MCP protocol compliance/auth flow beyond the brief doc mentions.

   • [claimed-docs] “LiteLLM Proxy provides an MCP Gateway that allows you to use a fixed endpoint for all MCP tools and control MCP access by Key, Team.”
   • [claimed-docs] “On the LiteLLM UI, Navigate to "MCP Servers" and click "Add New MCP Server".”

   proof ↗⚑ flag

   Kong AI Gateway✓fullclaimed8/10

   Kong AI Gateway ships an official 'AI MCP Server' entity and 'AI MCP Proxy' that converts REST APIs into MCP tools or fronts upstream MCP servers, letting agents connect via a first-party MCP interface, plus governance of MCP traffic (auth, OTel metrics, rate limiting) alongside it. Missing for 10: independent/hands-on verification of an agent successfully connecting through the MCP server and no example configuration walkthrough shown in the pack.

   • [claimed-docs] “AI MCP Proxy bridges Kong-managed Services to the Model Context Protocol, converting REST APIs into MCP tools or fronting upstream MCP serve…”
   • [claimed-docs] “Turn any API into an MCP server using the AI MCP Server entity. This approach does not require an LLM and provides full control over product…”
   • [claimed-docs] “Govern LLM, MCP, and agent-to-agent (A2A) traffic with the same Kong AI Gateway.”
   • [claimed-docs] “AI Gateway can export OpenTelemetry (OTLP) metrics for generative AI, MCP, and A2A traffic through an OpenTelemetry AI Policy.”

   proof ↗⚑ flag

5. ai-native userUse an official CLI

   weight 2 · round to Kong AI Gateway
   LiteLLM~partialprobed6/10

   LiteLLM ships an official CLI for its proxy (litellm docs-proxy-cli, litellm-probe-4) covering server startup and worker config, but the docs shown focus narrowly on process/worker management rather than a broad AI-native workflow CLI (e.g., chat, model testing, key/budget management from CLI). missing for 10: evidence of CLI commands beyond process startup (e.g., testing prompts, managing virtual keys/budgets, MCP servers via CLI), independent/hands-on confirmation of CLI usability.

   • [claimed-docs] “The number of worker processes to spin up (uvicorn, gunicorn, or Granian --workers).”
   • [claimed-docs] “The number of worker processes to spin up (uvicorn, gunicorn, or Granian `--workers`).”
   • [probe] “official CLI documented at https://docs.litellm.ai/docs/proxy/cli”

   proof ↗⚑ flag

   Kong AI Gateway✓fullprobed7/10

   Kong ships an official CLI, kongctl, documented at a dedicated docs URL and explicitly used for AI Gateway tasks (e.g., the 'kongctl AI Gateway conversion extension' for migrating configurations). This confirms a real, first-party CLI exists for AI Gateway workflows. Missing for 10: deeper documentation of kongctl's full command surface for AI-native tasks (agents, MCP, A2A) and independent/hands-on corroboration beyond the docs probe.

   • [claimed-docs] “walks you through migrating an existing configuration using the kongctl AI Gateway conversion extension”
   • [probe] “official CLI documented at https://developer.konghq.com/kongctl/”

   proof ↗⚑ flag

6. ai-native userDrive the product through a documented public API

   weight 3 · round to LiteLLM
   LiteLLM✓fullprobed7/10

   LiteLLM's core interface — the completion() function and the OpenAI-compatible proxy REST API — is extensively documented as a public, drop-in API that AI-native users can call directly, corroborated by community Show HN discussion confirming streaming/function-calling parity with the OpenAI SDK. Missing for 10: a discoverable machine-readable OpenAPI/swagger spec (probe found only 404s) and independent hands-on confirmation of API stability/versioning.

   • [claimed-docs] “Call any provider using the same completion() interface, with no API to re-learn for each one”
   • [claimed-docs] “Every response follows the OpenAI Chat Completions format, regardless of provider.”
   • [github] “Drop-in OpenAI compatibility — swap providers without rewriting your code”
   • [claimed-docs] “Call any provider using the same `completion()` interface, with no API to re-learn for each one”
   • [claimed-docs] “Consistent output format regardless of which provider or model you use”
   • [community] “LiteLLM maintainer confirmed proxy supports streaming and function-calling in the same way as the openai-python SDK, and added Ollama integr…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.litellm.ai/openapi.json, https://docs.litellm.ai/swagger.json, https://docs.litellm.ai/…”

   proof ↗⚑ flag

   Kong AI Gateway~partialprobed6/10

   Kong documents a Konnect API for managing AI Gateway resources (e.g., list-ai-gateways endpoint) and a dedicated kongctl CLI, showing programmatic control exists and is documented. However, probes for a discoverable OpenAPI/swagger spec at standard locations all returned 404, so the breadth/openness of the API surface is not independently confirmed. Missing for 10: a publicly discoverable OpenAPI/swagger spec, broader endpoint coverage beyond the single example, and independent/hands-on confirmation of API usage.

   • [claimed-docs] “list-ai-gateways summary: List AI Gateways description: Returns a list of AI Gateways in the organization.”
   • [probe] “official CLI documented at https://developer.konghq.com/kongctl/”
   • [probe] “PROBE openapi: all candidate paths 404 (https://developer.konghq.com/openapi.json, https://developer.konghq.com/swagger.json, https://develo…”

   proof ↗⚑ flag

7. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round to LiteLLM
   LiteLLM✓fullcommunity8/10

   LiteLLM's proxy virtual keys let admins scope credentials by model access, budget, team, and rate limits (rpm_limit), directly enabling least-privilege API keys for agents. This is well documented across multiple first-party docs (virtual_keys, users, cost_tracking) showing budgets, per-key model restrictions, and team-based limits. Missing for 10: independent/hands-on verification of the scoping granularity in practice, and one community comment (litellm-comm-3) noted vague 'issues with creating tokens for other users' suggesting rough edges.

   • [claimed-docs] “Track Spend, and control model access via virtual keys for the proxy”
   • [claimed-docs] “Personal budgets: Create virtual keys without team_id for individual spending limits”
   • [claimed-docs] “Apply a budget across all calls on the proxy... budget_duration: 30d # (str) frequency of reset”
   • [claimed-docs] “"team_alias": "my-new-team_4", ... "rpm_limit": 99”
   • [claimed-docs] “Track spend for keys, users, and teams across 100+ LLMs.”
   • [community] “'Does anyone know a good alternate project... LiteLLM has been getting worse and trying to get me to upgrade to a paid version. I also had i…”

   proof ↗⚑ flag

   Kong AI Gateway~partialclaimed5/10

   Kong AI Gateway provides building blocks relevant to credential control for agents — AI Auth Strategy entities can authenticate via enterprise IdPs (Okta/Azure AD/OIDC) 'without managing keys manually,' and AI Vault lets model providers, auth strategies, agents, and MCP servers reference secrets from external backends instead of storing literal keys, plus Azure Managed Identity support for scoped cloud authentication. However, none of this evidence explicitly describes issuing scoped/least-privilege credentials per agent (e.g., fine-grained permission scopes, per-agent token minting) — it's centralized secret referencing and IdP auth rather than a dedicated least-privilege credential-issuance mechanism. missing for 10: explicit documentation of scoped/least-privilege credential issuance per agent, granular permission/scope controls tied to individual agent identities, and independent/hands-on verification of this behavior.

   • [claimed-docs] “An AI Vault entity lets you register an external secret backend (AWS Secrets Manager, HashiCorp Vault, environment variables, or others) so …”
   • [claimed-docs] “Authenticate enterprise users through an existing identity provider (Okta, Azure AD, Google, or any OIDC-compliant IdP) without managing key…”
   • [claimed-docs] “Yes, if AI Gateway is running on Azure, you can configure an AI Model Provider to detect the designated Managed Identity or User-Assigned Id…”

   proof ↗⚑ flag

8. ai-native userBuild against official SDKs

   weight 2 · round to LiteLLM
   LiteLLM✓fullcommunity8/10

   LiteLLM ships an official Python SDK (the `completion()` interface) that mirrors the OpenAI SDK format across 100+ providers, letting AI-native developers build without learning per-provider APIs; this is corroborated by community Show HN feedback praising it for standardizing APIs and confirming SDK parity for streaming/function-calling. Missing for 10: evidence of official SDKs in other languages (e.g., JS/TS) and independent benchmarking of SDK completeness beyond community praise.

   • [claimed-docs] “Call any provider using the same completion() interface, with no API to re-learn for each one”
   • [claimed-docs] “Every response follows the OpenAI Chat Completions format, regardless of provider.”
   • [github] “Drop-in OpenAI compatibility — swap providers without rewriting your code”
   • [claimed-docs] “Call any provider using the same `completion()` interface, with no API to re-learn for each one”
   • [claimed-docs] “Consistent output format regardless of which provider or model you use”
   • [community] “LiteLLM maintainer confirmed proxy supports streaming and function-calling in the same way as the openai-python SDK, and added Ollama integr…”
   • [community] “Show HN launch of the original LiteLLM library; a user praised it: 'This is amazing. Really needed something like this to standardize all my…”

   proof ↗⚑ flag

   Kong AI Gateway—none0/10

   The evidence pack contains no mention of official SDKs for building against Kong AI Gateway (only a CLI 'kongctl' and REST API references), and probes explicitly show no OpenAPI/SDK artifacts (404s for openapi.json, swagger.json, etc.). While an API gateway could plausibly offer official client SDKs, none are documented here.

   • [probe] “PROBE openapi: all candidate paths 404 (https://developer.konghq.com/openapi.json, https://developer.konghq.com/swagger.json, https://develo…”
   • [probe] “official CLI documented at https://developer.konghq.com/kongctl/”
   • [claimed-docs] “list-ai-gateways summary: List AI Gateways description: Returns a list of AI Gateways in the organization.”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   LiteLLM—none0/10

   The evidence pack shows explicit probe failures for an OpenAPI/Swagger spec (litellm-probe-3) and no documented interactive API reference or runnable examples in the docs; only static markdown-style docs and code snippets are cited (litellm-docs-1/2/16/17). Since LiteLLM's proxy is an API gateway, this axis clearly applies, but there's no evidence of an interactive/try-it-out API reference.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.litellm.ai/openapi.json, https://docs.litellm.ai/swagger.json, https://docs.litellm.ai/…”
   • [probe] “PROBE docs-md: HTTP 404 at https://docs.litellm.ai/docs/.md”
   • [claimed-docs] “Call any provider using the same completion() interface, with no API to re-learn for each one”
   • [claimed-docs] “Call any provider using the same `completion()` interface, with no API to re-learn for each one”

   proof ↗⚑ flag

   Kong AI Gateway—none0/10

   Evidence shows Kong publishes API reference content (e.g., list-ai-gateways endpoint docs) but there is no indication of an interactive reference with runnable/try-it-out examples; probes for OpenAPI/swagger specs on the docs site all returned 404, suggesting no such interactive tooling is exposed.

   • [claimed-docs] “list-ai-gateways summary: List AI Gateways description: Returns a list of AI Gateways in the organization.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://developer.konghq.com/openapi.json, https://developer.konghq.com/swagger.json, https://develo…”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   LiteLLM—none0/10

   The evidence pack includes explicit probes for OpenAPI/swagger endpoints on LiteLLM's docs site, all returning 404, and no other citation shows a downloadable OpenAPI spec (e.g., from the proxy's FastAPI docs). Since LiteLLM's proxy is a gateway with an admin UI and CLI, this axis plausibly applies, but no evidence confirms a machine-readable spec is available.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.litellm.ai/openapi.json, https://docs.litellm.ai/swagger.json, https://docs.litellm.ai/…”
   • [probe] “PROBE docs-md: HTTP 404 at https://docs.litellm.ai/docs/.md”

   proof ↗⚑ flag

   Kong AI Gateway—none0/10

   Evidence includes API reference pages (e.g. Konnect AI Gateway API docs) but no downloadable OpenAPI/Swagger spec file is evidenced; a direct probe for standard OpenAPI/swagger.json locations returned 404 at all candidate paths, indicating no machine-readable spec is exposed there.

   • [probe] “PROBE openapi: all candidate paths 404 (https://developer.konghq.com/openapi.json, https://developer.konghq.com/swagger.json, https://develo…”
   • [claimed-docs] “list-ai-gateways summary: List AI Gateways description: Returns a list of AI Gateways in the organization.”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round drawn
   LiteLLM—none0/10

   No evidence in the pack of a dedicated sandbox/test mode, mock-response capability, or staging environment for validating integrations without touching production data — the docs cover routing, budgets, virtual keys, logging, and reliability, but nothing about isolated test environments. missing for 10: sandbox/test mode, mock LLM responses, staging deployment guidance, docs mentioning safe non-production testing.

   ⚑ flag

   Kong AI Gateway—none0/10

   The evidence pack only shows a quickstart script for spinning up a demo instance and general AI Gateway features (proxying, observability, policies) but no dedicated sandbox/staging environment or test-mode capability that isolates from production data. No mention of environment separation, mock upstreams, or a documented sandbox mode.

   ⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   LiteLLM—none0/10

   No evidence of API versioning scheme or a documented deprecation policy anywhere in docs; OpenAPI/spec discovery probes returned 404s, suggesting no formal versioned API contract is published.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.litellm.ai/openapi.json, https://docs.litellm.ai/swagger.json, https://docs.litellm.ai/…”
   • [probe] “PROBE docs-md: HTTP 404 at https://docs.litellm.ai/docs/.md”

   proof ↗⚑ flag

   Kong AI Gateway—none0/10

   The evidence shows an API reference exists (e.g., 'v1' Konnect AI Gateway API) but there is no documentation of a versioning scheme or deprecation policy for the AI Gateway APIs, and OpenAPI spec probes returned 404s.

   • [claimed-docs] “list-ai-gateways summary: List AI Gateways description: Returns a list of AI Gateways in the organization.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://developer.konghq.com/openapi.json, https://developer.konghq.com/swagger.json, https://develo…”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round drawn
   LiteLLM—none0/10

   The evidence pack describes LiteLLM's unified completion interface, routing, fallbacks, cost tracking, and virtual keys, but contains no mention of batch/bulk operations (e.g., batch completions across many prompts, bulk key/user management, or bulk import/export) that would let a user act on many items at once. Since LiteLLM manages many models, keys, and users, this axis is plausible for the product, but no supporting evidence exists.

   ⚑ flag

   Kong AI Gateway—none0/10

   The evidence covers proxying, observability, security, and MCP/A2A support but never describes any bulk or batch operation capability (e.g., batch creating/updating routes, plugins, or AI entities across many items at once). The kongctl CLI and Konnect API are mentioned but only for single-entity or config migration tasks, not bulk item processing.

   • [claimed-docs] “list-ai-gateways summary: List AI Gateways description: Returns a list of AI Gateways in the organization.”
   • [claimed-docs] “walks you through migrating an existing configuration using the kongctl AI Gateway conversion extension”
   • [probe] “official CLI documented at https://developer.konghq.com/kongctl/”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round drawn
   LiteLLM~partialclaimed5/10

   LiteLLM ships several built-in event-triggered automations (automatic fallback to another model/provider on call failure or content-policy violation, budget resets on a schedule, rate limits per key/team) but these are fixed, pre-defined behaviors rather than a general user-definable rule/trigger-action engine (e.g. no webhook/if-this-then-that automation builder is evidenced). Missing for 10: a general-purpose custom rule/condition-action automation system, evidence of user-defined triggers beyond fallback/budget/rate-limit, and any webhook or external-action firing mechanism.

   • [claimed-docs] “If a call fails after num_retries, LiteLLM falls back to another model group, so a failing model or provider automatically fails over to a h…”
   • [claimed-docs] “content_policy_fallbacks: For litellm.ContentPolicyViolationError - LiteLLM maps content policy violation errors across providers”
   • [claimed-docs] “so a failing model or provider automatically fails over to a healthy backup”
   • [claimed-docs] “The request to `model="zephyr-beta"` will fail... litellm proxy will loop through all the model_groups specified in `fallbacks=["gpt-3.5-tur…”
   • [claimed-docs] “Apply a budget across all calls on the proxy... budget_duration: 30d # (str) frequency of reset”
   • [claimed-docs] “"team_alias": "my-new-team_4", ... "rpm_limit": 99”

   proof ↗⚑ flag

   Kong AI Gateway~partialclaimed5/10

   Kong AI Gateway supports policy-based automatic actions triggered by request/response events—e.g., Prompt Guards and content-safety policies acting on chat/completion requests, AI Rate Limiting Advanced enforcing limits automatically, AWS Guardrails validating requests/responses before forwarding, and semantic caching/routing acting on similarity triggers—which functions as rule-based automation on traffic events. However, this is scoped to gateway traffic policies rather than a general-purpose event/trigger automation framework for arbitrary conditions and actions. Missing for 10: evidence of a generic conditional rule engine (if-this-then-that style), user-defined custom triggers beyond built-in policies, and any workflow/orchestration automation beyond request-level policy enforcement.

   • [claimed-docs] “AI Gateway supports content safety features across providers and also includes our Prompt Guards that act on any llm/v1/chat or llm/v1/compl…”
   • [claimed-docs] “AI Rate Limiting Advanced - Plugin | Kong Docs”
   • [claimed-docs] “Apply safety and DLP policies to prevent toxic content and remove personally identifiable information.”
   • [claimed-docs] “AI Gateway can perform intelligent request routing, caching, and content filtering using semantic similarity queries.”
   • [claimed-docs] “Use AWS Guardrails to validate requests and/or responses before forwarding them between clients and upstream LLMs.”

   proof ↗⚑ flag

Caching

1. developerCache responses at the gateway to cut cost and latency on repeated requests

   weight 2 · round drawn
   LiteLLM✓fullclaimed8/10

   LiteLLM proxy has a documented caching system that explicitly stores and reuses LLM responses to save costs and reduce latency, directly matching the story. Missing for 10: independent/hands-on benchmarks quantifying latency/cost savings, and detail on cache backend configuration options in the evidence pack.

   • [claimed-docs] “LiteLLM's caching system stores and reuses LLM responses to save costs and reduce latency.”

   proof ↗⚑ flag

   Kong AI Gateway✓fullclaimed8/10

   Kong ships a dedicated AI Semantic Cache plugin explicitly designed to cache LLM responses at the gateway, and separate docs confirm gateway-level semantic caching to reduce redundant calls and improve latency/cost. This is first-party documentation with a named product feature, though no independent benchmark or hands-on cost/latency measurement is provided. Missing for 10: independent/hands-on validation of actual cost/latency savings, and exact-match (non-semantic) cache configuration details.

   • [claimed-docs] “AI Semantic Cache - Plugin | Kong Docs”
   • [claimed-docs] “AI Gateway can perform intelligent request routing, caching, and content filtering using semantic similarity queries.”

   proof ↗⚑ flag

Latency

1. platform engineerRun traffic through gateway infrastructure that adds minimal latency overhead to provider calls

   weight 2 · round to Kong AI Gateway
   LiteLLM~partialcommunity4/10

   LiteLLM docs describe features that reduce or mitigate latency (response caching, configurable worker processes for uvicorn/gunicorn/Granian) and fast failover, but there is no explicit benchmark or vendor claim quantifying the proxy's own added latency overhead versus direct provider calls. Community evidence includes a report of the proxy being 'incredible flaky and causing outages,' which is a reliability complaint rather than a direct latency measurement, so it doesn't rise to a concrete dispute of a latency claim. missing for 10: explicit latency-overhead benchmarks/numbers, independent performance testing of proxy overhead, direct vendor claim about added latency being minimal.

   • [claimed-docs] “LiteLLM's caching system stores and reuses LLM responses to save costs and reduce latency.”
   • [claimed-docs] “The number of worker processes to spin up (uvicorn, gunicorn, or Granian --workers).”
   • [claimed-docs] “The number of worker processes to spin up (uvicorn, gunicorn, or Granian `--workers`).”
   • [claimed-docs] “If a call fails after num_retries, LiteLLM falls back to another model group, so a failing model or provider automatically fails over to a h…”
   • [community] “'Now I feel lucky that I switched to just using OpenRouter a year ago because LiteLLM was incredible flaky and kept causing outages.'”

   proof ↗⚑ flag

   Kong AI Gateway~partialclaimed5/10

   Kong documents latency-related tooling (usage analytics and OTel metrics that track LLM request latency and upstream processing time, semantic caching, load balancing across models) which implies attention to performance, but there is no explicit claim or benchmark quantifying the gateway's own added latency overhead as 'minimal'. missing for 10: quantified latency-overhead benchmarks, independent performance tests, explicit vendor claim of low proxy overhead.

   • [claimed-docs] “You can use these metrics to: Track LLM request latency and upstream provider processing time * Monitor token consumption ... * Calculate AI…”
   • [claimed-docs] “AI Semantic Cache - Plugin | Kong Docs”
   • [claimed-docs] “AI Gateway provides load balancing capabilities to distribute requests across multiple LLM models.”
   • [claimed-docs] “See usage analytics for requests, tokens, errors, and latency”

   proof ↗⚑ flag

Agent guardrails

1. ai-native userGive an autonomous agent its own key with budget and rate guardrails so it cannot run away on spend

   weight 2 · round to LiteLLM
   LiteLLM✓fullclaimed8/10

   LiteLLM's virtual key system explicitly supports per-key budgets (budget_duration, max_budget), rate limits (rpm_limit), and personal/team-scoped keys, directly enabling an autonomous agent to get its own key with spend and rate guardrails. Docs also show spend tracking by key/user/tag to monitor and enforce these limits. Missing for 10: independent/hands-on verification that budget caps hard-stop overspend in practice, and more detail on rate-limit enforcement behavior under load.

   • [claimed-docs] “Personal budgets: Create virtual keys without team_id for individual spending limits”
   • [claimed-docs] “Track Spend, and control model access via virtual keys for the proxy”
   • [claimed-docs] “Apply a budget across all calls on the proxy... budget_duration: 30d # (str) frequency of reset”
   • [claimed-docs] “"team_alias": "my-new-team_4", ... "rpm_limit": 99”
   • [claimed-docs] “Track spend for keys, users, and teams across 100+ LLMs.”
   • [claimed-docs] “pass tags to track spend by tags”
   • [claimed-docs] “Put your full AI stack behind one key. See who is driving spend, cap it before it runs, and send each request to the model that should handl…”

   proof ↗⚑ flag

   Kong AI Gateway~partialclaimed6/10

   Kong AI Gateway supports per-consumer/agent authentication (AI Auth Strategy, AI Vault for keys) plus AI Rate Limiting Advanced and model cost calculation/tracking that matches provider billing, which together let an operator scope an agent's key and cap its request rate and see accrued spend. However, evidence shows cost *calculation and observability* (docs-9, docs-10, docs-18) rather than an explicit hard budget cap/kill-switch that automatically cuts off an agent once a spend threshold is reached, so the 'cannot run away on spend' guarantee is only partially evidenced. Missing for 10: explicit documented budget-threshold enforcement/auto-cutoff tied to a specific agent key, and a worked example of issuing a scoped key to an autonomous agent with combined rate+spend limits.

   • [claimed-docs] “AI Rate Limiting Advanced - Plugin | Kong Docs”
   • [claimed-docs] “AI Gateway calculates that figure for every request to a large language model (LLM) provider, matching what the provider actually bills.”
   • [claimed-docs] “An AI Vault entity lets you register an external secret backend (AWS Secrets Manager, HashiCorp Vault, environment variables, or others) so …”
   • [claimed-docs] “Authenticate enterprise users through an existing identity provider (Okta, Azure AD, Google, or any OIDC-compliant IdP) without managing key…”
   • [claimed-docs] “You can use these metrics to: Track LLM request latency and upstream provider processing time * Monitor token consumption ... * Calculate AI…”

   proof ↗⚑ flag

Budgets

1. platform engineerSet hard budgets and spend limits per key, team, or user

   weight 3 · round to LiteLLM
   LiteLLM✓fullclaimed9/10

   Docs explicitly cover per-key, per-team, and per-user budgets/spend tracking (virtual keys with cost tracking, personal budgets, team rpm_limit, budget_duration for resets, tag-based spend tracking) plus marketing copy on capping spend before it runs. Missing for 10: independent hands-on verification that hard limits actually block calls once exceeded (only docs, no community confirmation of enforcement behavior).

   • [claimed-docs] “Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI”
   • [claimed-docs] “Track spend for keys, users, and teams across 100+ LLMs.”
   • [claimed-docs] “Personal budgets: Create virtual keys without team_id for individual spending limits”
   • [claimed-docs] “Track Spend, and control model access via virtual keys for the proxy”
   • [claimed-docs] “Put your full AI stack behind one key. See who is driving spend, cap it before it runs, and send each request to the model that should handl…”
   • [claimed-docs] “pass tags to track spend by tags”
   • [claimed-docs] “Apply a budget across all calls on the proxy... budget_duration: 30d # (str) frequency of reset”
   • [claimed-docs] “"team_alias": "my-new-team_4", ... "rpm_limit": 99”

   proof ↗⚑ flag

   Kong AI Gateway~partialclaimed5/10

   Kong AI Gateway provides cost calculation, tracking across teams, and rate limiting plugins (AI Rate Limiting Advanced) which can enforce request/token limits per consumer, but evidence shows rate limiting and cost tracking as separate capabilities rather than a unified hard 'budget/spend limit' enforcement tied directly to cost thresholds per key/team/user. missing for 10: explicit documentation of setting a dollar-based spend cap or budget that auto-blocks requests once exceeded, per-key/team/user budget configuration UI or API, and independent confirmation of budget enforcement in production.

   • [claimed-docs] “AI Rate Limiting Advanced - Plugin | Kong Docs”
   • [claimed-docs] “AI Gateway calculates that figure for every request to a large language model (LLM) provider, matching what the provider actually bills.”
   • [claimed-docs] “AI Gateway can proxy requests from AI command-line tools to LLM providers. This gives you centralized control over AI traffic: log all reque…”
   • [claimed-docs] “You can use these metrics to: Track LLM request latency and upstream provider processing time * Monitor token consumption ... * Calculate AI…”

   proof ↗⚑ flag

Spend tracking

1. platform engineerTrack spend per model, key, team, or user across all providers in one place

   weight 3 · round to LiteLLM
   LiteLLM✓fullclaimed9/10

   LiteLLM's docs directly document tracking spend per key, user, team, and by tags across 100+ providers via the proxy's cost tracking feature, plus budgets, virtual keys, and admin UI to view spend — matching the story closely. Missing for 10: independent hands-on verification of the cost dashboard's accuracy/UX and explicit per-model spend breakdown evidence beyond the general 'across 100+ LLMs' claim.

   • [claimed-docs] “Track spend for keys, users, and teams across 100+ LLMs.”
   • [claimed-docs] “Track Spend, and control model access via virtual keys for the proxy”
   • [claimed-docs] “pass tags to track spend by tags”
   • [claimed-docs] “Apply a budget across all calls on the proxy... budget_duration: 30d # (str) frequency of reset”
   • [claimed-docs] “Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI”
   • [claimed-docs] “Put your full AI stack behind one key. See who is driving spend, cap it before it runs, and send each request to the model that should handl…”

   proof ↗⚑ flag

   Kong AI Gateway~partialclaimed7/10

   Kong AI Gateway computes exact per-request LLM costs matching provider billing, logs cost metrics with token/model details, exports cost-related OTel metrics, and documentation explicitly states cost tracking 'across teams'; since all providers route through the same gateway, aggregation across providers is plausible. However, evidence never shows a consolidated dashboard or breakdown specifically by model, API key, or individual user — only 'teams' is explicitly named as a tracked dimension. Missing for 10: explicit per-user and per-API-key cost attribution/dashboard, and a documented single-pane cross-provider spend view (vs. raw logs/metrics that would need external aggregation).

   • [claimed-docs] “AI Gateway calculates that figure for every request to a large language model (LLM) provider, matching what the provider actually bills.”
   • [claimed-docs] “AI Gateway logs capture detailed information about the request and response payloads, token usage, model details, latency, and cost metrics.”
   • [claimed-docs] “You can use these metrics to: Track LLM request latency and upstream provider processing time * Monitor token consumption ... * Calculate AI…”
   • [claimed-docs] “AI Gateway can proxy requests from AI command-line tools to LLM providers. This gives you centralized control over AI traffic: log all reque…”
   • [claimed-docs] “AI Gateway can export OpenTelemetry (OTLP) metrics for generative AI, MCP, and A2A traffic through an OpenTelemetry AI Policy.”

   proof ↗⚑ flag

Byok

1. developerBring my own provider API keys and have the gateway use them for my traffic

   weight 2 · round drawn
   LiteLLM~partialclaimed6/10

   LiteLLM's core design lets developers call any provider through one interface and issue virtual keys that map to underlying provider credentials, with per-key spend/model access control ('Put your full AI stack behind one key', virtual key docs) — implying BYOK usage. However, the evidence pack lacks an explicit walkthrough of adding a personal provider API key to the proxy config and having it used for a developer's own traffic, and there is no independent confirmation of this specific flow. Missing for 10: explicit BYOK config example, independent/hands-on confirmation of per-developer key routing.

   • [claimed-docs] “Call any provider using the same completion() interface, with no API to re-learn for each one”
   • [claimed-docs] “Track Spend, and control model access via virtual keys for the proxy”
   • [claimed-docs] “Put your full AI stack behind one key. See who is driving spend, cap it before it runs, and send each request to the model that should handl…”
   • [claimed-docs] “Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI”

   proof ↗⚑ flag

   Kong AI Gateway~partialclaimed6/10

   Docs show AI Model Providers are configured with credentials, and an AI Vault entity lets AI Model Providers reference externally-stored secrets (AWS Secrets Manager, HashiCorp Vault, env vars) rather than storing keys as literal values, and Azure-specific managed identity auth is also supported — implying developers can supply and manage their own provider keys/secrets. However, no explicit doc snippet shows the exact 'paste your own API key per provider' config example or per-request key override behavior. Missing for 10: an explicit example of configuring a provider with a literal/bring-your-own API key, and confirmation of per-consumer or per-request key overrides.

   • [claimed-docs] “The core of AI Gateway is the ability to serve AI Models from various AI Model Providers via a provider-agnostic API.”
   • [claimed-docs] “Yes, if AI Gateway is running on Azure, you can configure an AI Model Provider to detect the designated Managed Identity or User-Assigned Id…”
   • [claimed-docs] “An AI Vault entity lets you register an external secret backend (AWS Secrets Manager, HashiCorp Vault, environment variables, or others) so …”
   • [claimed-docs] “Authenticate enterprise users through an existing identity provider (Okta, Azure AD, Google, or any OIDC-compliant IdP) without managing key…”

   proof ↗⚑ flag

Programmatic admin

1. ai-native userProvision gateways, keys, and budgets programmatically through an admin API

   weight 3 · round to Kong AI Gateway
   LiteLLM!disputedcontradicted5/10

   Docs describe programmatic key/team/budget management (virtual keys, personal and team budgets, rpm_limit, team_alias via API/config) which supports the story's core claim, but no OpenAPI/swagger spec was found (probe shows 404s) and a community user reported concrete hands-on issues 'creating tokens for other users' via the proxy admin functions, directly contradicting the smoothness implied by the docs. Missing for 10: verified OpenAPI/admin-API reference, evidence of 'gateway' provisioning (multiple proxy instances) via API, and independent confirmation that key/budget creation works reliably at scale.

   • [claimed-docs] “Personal budgets: Create virtual keys without team_id for individual spending limits”
   • [claimed-docs] “Track Spend, and control model access via virtual keys for the proxy”
   • [claimed-docs] “Apply a budget across all calls on the proxy... budget_duration: 30d # (str) frequency of reset”
   • [claimed-docs] “"team_alias": "my-new-team_4", ... "rpm_limit": 99”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.litellm.ai/openapi.json, https://docs.litellm.ai/swagger.json, https://docs.litellm.ai/…”
   • [community] “'Does anyone know a good alternate project... LiteLLM has been getting worse and trying to get me to upgrade to a paid version. I also had i…”

   proof ↗⚑ flag

   Kong AI Gateway~partialclaimed5/10

   Kong exposes a Konnect API for listing/managing AI Gateways (list-ai-gateways) and entity-based configuration (AI Auth Strategy, AI Vault) that implies programmatic provisioning of gateways and credentials, plus a kongctl CLI for gateway configuration/migration. However, there is no explicit documentation of an API or CLI for creating/managing spend budgets or quotas, and no end-to-end example of provisioning gateway+keys+budget via API. Missing for 10: explicit budget/quota provisioning API, a full API reference for creating AI Auth Strategies/keys programmatically, and independent confirmation of the admin API's completeness for this workflow.

   • [claimed-docs] “list-ai-gateways summary: List AI Gateways description: Returns a list of AI Gateways in the organization.”
   • [claimed-docs] “An AI Vault entity lets you register an external secret backend (AWS Secrets Manager, HashiCorp Vault, environment variables, or others) so …”
   • [claimed-docs] “Authenticate enterprise users through an existing identity provider (Okta, Azure AD, Google, or any OIDC-compliant IdP) without managing key…”
   • [claimed-docs] “walks you through migrating an existing configuration using the kongctl AI Gateway conversion extension”
   • [claimed-docs] “AI Gateway calculates that figure for every request to a large language model (LLM) provider, matching what the provider actually bills.”

   proof ↗⚑ flag

Virtual keys

1. platform engineerMint gateway-managed keys for teams and apps without exposing raw provider keys

   weight 3 · round to LiteLLM
   LiteLLM~partialcommunity7/10

   LiteLLM's proxy docs explicitly support minting scoped 'virtual keys' for teams/users/apps with budget and rate limits (litellm-docs-5, litellm-docs-6, litellm-docs-7, litellm-docs-8, litellm-docs-26), which is exactly the gateway-managed-key model that hides raw provider keys behind the proxy. However, one community report describes hands-on 'issues with creating tokens for other users' (litellm-comm-3), and there is no independent corroboration of the key-isolation guarantees or a walkthrough of the actual key-generation API beyond first-party docs. Missing for 10: independent/hands-on confirmation that key scoping reliably prevents provider-key leakage, and resolution of the reported token-creation friction.

   • [claimed-docs] “Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI”
   • [claimed-docs] “Track spend for keys, users, and teams across 100+ LLMs.”
   • [claimed-docs] “Personal budgets: Create virtual keys without team_id for individual spending limits”
   • [claimed-docs] “Track Spend, and control model access via virtual keys for the proxy”
   • [claimed-docs] “"team_alias": "my-new-team_4", ... "rpm_limit": 99”
   • [community] “'Does anyone know a good alternate project... LiteLLM has been getting worse and trying to get me to upgrade to a paid version. I also had i…”

   proof ↗⚑ flag

   Kong AI Gateway~partialclaimed6/10

   Kong AI Gateway's AI Vault lets provider keys be stored in external secret backends and referenced by AI Model Providers/Agents instead of literal values, and AI Auth Strategy lets client apps authenticate via an IdP or Kong-managed auth without handling provider keys directly — both point toward the platform-engineer being able to shield raw provider keys from consumers. However, the evidence never explicitly describes a workflow for minting distinct gateway-managed keys per team/app, rotating them, or a consumer/key-auth flow tied to AI traffic. Missing for 10: explicit documentation of issuing/rotating per-team or per-app gateway-managed API keys, and any admin UI/API walkthrough for that key-minting process.

   • [claimed-docs] “An AI Vault entity lets you register an external secret backend (AWS Secrets Manager, HashiCorp Vault, environment variables, or others) so …”
   • [claimed-docs] “Authenticate enterprise users through an existing identity provider (Okta, Azure AD, Google, or any OIDC-compliant IdP) without managing key…”
   • [claimed-docs] “Reference an AI Auth Strategy on your AI Agent entities to authenticate A2A clients, and attach the Request Size Limiting Policy to limit re…”

   proof ↗⚑ flag

Integrations

1. developerExport gateway logs and traces to my own observability stack

   weight 1 · round drawn
   LiteLLM✓fullclaimed8/10

   Docs explicitly confirm proxy logging integrations to external observability stacks (Langfuse, OpenTelemetry, GCS, S3, Azure Blob), plus a unique call_id for tracing requests across systems, directly matching the story of exporting logs/traces to one's own stack. Missing for 10: independent hands-on verification of these integrations working reliably, and more detail on trace format/OpenTelemetry span structure.

   • [claimed-docs] “Log Proxy input, output, and exceptions using: Langfuse, OpenTelemetry, GCS, s3, Azure (Blob) Buckets...”
   • [claimed-docs] “LiteLLM generates a unique call_id for each request. This call_id can be used to track the request across the system.”
   • [claimed-docs] “LiteLLM generates a unique `call_id` for each request. This `call_id` can be used to track the request across the system.”
   • [claimed-docs] “This will prevent the messages and responses from being logged to your logging provider, but request metadata - e.g. spend, will still be tr…”

   proof ↗⚑ flag

   Kong AI Gateway✓fullclaimed8/10

   Kong AI Gateway documents exporting OpenTelemetry (OTLP) metrics for generative AI, MCP, and A2A traffic via an OpenTelemetry AI Policy, plus detailed audit logs capturing request/response payloads, token usage, latency, and cost — enabling integration with a developer's own observability stack. Missing for 10: explicit trace export (spans/tracing) documentation and independent/hands-on confirmation of exporting to third-party backends like Datadog or Grafana.

   • [claimed-docs] “AI Gateway can export OpenTelemetry (OTLP) metrics for generative AI, MCP, and A2A traffic through an OpenTelemetry AI Policy.”
   • [claimed-docs] “You can use these metrics to: Track LLM request latency and upstream provider processing time * Monitor token consumption ... * Calculate AI…”
   • [claimed-docs] “AI Gateway logs capture detailed information about the request and response payloads, token usage, model details, latency, and cost metrics.”

   proof ↗⚑ flag

Logs

1. platform engineerInspect logged requests and responses with latency, token counts, and cost attached

   weight 3 · round to Kong AI Gateway
   LiteLLM~partialclaimed6/10

   Docs show LiteLLM logs each request with a unique call_id, tracks spend/cost by key/user/team, and integrates with Langfuse, OpenTelemetry, S3, GCS, etc. for logging proxy input/output/exceptions, all viewable via the admin UI — a reasonable basis for inspecting requests with cost attached. However, the evidence never explicitly confirms that latency or token counts are captured/displayed alongside these logs, and there's no independent/hands-on confirmation of the inspection UI itself. Missing for 10: explicit documentation of per-request latency metrics, explicit token-count logging, and independent verification of the UI/log inspection experience.

   • [claimed-docs] “Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI”
   • [claimed-docs] “Track spend for keys, users, and teams across 100+ LLMs.”
   • [claimed-docs] “Log Proxy input, output, and exceptions using: Langfuse, OpenTelemetry, GCS, s3, Azure (Blob) Buckets...”
   • [claimed-docs] “LiteLLM generates a unique call_id for each request. This call_id can be used to track the request across the system.”
   • [claimed-docs] “pass tags to track spend by tags”

   proof ↗⚑ flag

   Kong AI Gateway✓fullclaimed8/10

   Kong AI Gateway docs explicitly describe audit logs capturing request/response payloads, token usage, model details, latency, and cost metrics, plus usage analytics dashboards and OTel metrics for latency, tokens, and cost. missing for 10: independent/hands-on corroboration of the logging UI in practice and detail on log retention/query interface.

   • [claimed-docs] “See usage analytics for requests, tokens, errors, and latency”
   • [claimed-docs] “You can use these metrics to: Track LLM request latency and upstream provider processing time * Monitor token consumption ... * Calculate AI…”
   • [claimed-docs] “AI Gateway logs capture detailed information about the request and response payloads, token usage, model details, latency, and cost metrics.”
   • [claimed-docs] “AI Gateway calculates that figure for every request to a large language model (LLM) provider, matching what the provider actually bills.”
   • [claimed-docs] “AI Gateway can export OpenTelemetry (OTLP) metrics for generative AI, MCP, and A2A traffic through an OpenTelemetry AI Policy.”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to LiteLLM
   LiteLLM~partialprobed6/10

   LiteLLM's proxy exposes virtual keys, budgets, spend tracking, MCP server management, and routing/fallback config via REST API endpoints and CLI, and the admin UI is largely a wrapper over these same proxy APIs, suggesting strong API/UI parity. However, no OpenAPI spec was found at expected locations (404s), and some UI-specific flows like adding MCP servers are only documented via UI clicks rather than confirmed equivalent API calls, and no independent confirmation exists that every UI action has a documented API equivalent. Missing for 10: a published OpenAPI/swagger spec confirming full endpoint coverage, explicit API equivalents for every UI workflow (e.g., MCP server add via API), and independent/hands-on verification that API and UI are truly at parity.

   • [claimed-docs] “Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI”
   • [claimed-docs] “Track Spend, and control model access via virtual keys for the proxy”
   • [claimed-docs] “LiteLLM Proxy provides an MCP Gateway that allows you to use a fixed endpoint for all MCP tools and control MCP access by Key, Team.”
   • [claimed-docs] “On the LiteLLM UI, Navigate to "MCP Servers" and click "Add New MCP Server".”
   • [claimed-docs] “The number of worker processes to spin up (uvicorn, gunicorn, or Granian --workers).”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.litellm.ai/openapi.json, https://docs.litellm.ai/swagger.json, https://docs.litellm.ai/…”
   • [probe] “official CLI documented at https://docs.litellm.ai/docs/proxy/cli”

   proof ↗⚑ flag

   Kong AI Gateway~partialprobed5/10

   Kong exposes AI Gateway configuration via the Konnect API (e.g. list-ai-gateways endpoint) and the kongctl CLI, and docs explicitly note that 'the same capabilities' configurable via plugins can be set on Services/Routes rather than only the UI, implying strong API-driven configuration. However there is no explicit statement of full UI/API parity, and probes for a discoverable OpenAPI spec returned 404s, leaving completeness unverified. Missing for 10: an explicit parity statement or comprehensive OpenAPI spec confirming every UI action (e.g. dashboards, analytics views) is also API-accessible, and independent confirmation of no UI-only features.

   • [claimed-docs] “list-ai-gateways summary: List AI Gateways description: Returns a list of AI Gateways in the organization.”
   • [claimed-docs] “This page shows you how to configure the same capabilities with AI plugins on Services and Routes.”
   • [probe] “official CLI documented at https://developer.konghq.com/kongctl/”
   • [probe] “PROBE openapi: all candidate paths 404 (https://developer.konghq.com/openapi.json, https://developer.konghq.com/swagger.json, https://develo…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round drawn
   LiteLLM—none0/10

   LiteLLM stores spend, logs, virtual-key and config data (self-hosted proxy with admin UI), so an explicit 'export all data in open formats' capability is a fair ask, but no evidence pack item documents any export/download feature, open data format, or data-portability tooling — only logging integrations to third-party sinks (Langfuse, S3, GCS) are mentioned, which is not the same as user-initiated full data export. missing for 10: documented export/download feature, open-format data export (CSV/JSON) of spend/logs/keys, any data-portability or 'leave the platform' guidance.

   • [claimed-docs] “Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI”
   • [claimed-docs] “Track spend for keys, users, and teams across 100+ LLMs.”
   • [claimed-docs] “Log Proxy input, output, and exceptions using: Langfuse, OpenTelemetry, GCS, s3, Azure (Blob) Buckets...”

   proof ↗⚑ flag

   Kong AI Gateway—none0/10

   Kong AI Gateway/Konnect is a SaaS-hosted control plane with configuration, logs, and analytics data, so data portability/export-and-leave is a fair axis to ask, but the evidence pack contains no mention of a bulk data export feature, open-format export of configs/logs/analytics, or a documented migration-out path in open standards. Evidence only covers observability (metrics, logs) and a CLI conversion extension for migrating configs between versions, not exporting all user data to leave the platform.

   • [claimed-docs] “walks you through migrating an existing configuration using the kongctl AI Gateway conversion extension”
   • [claimed-docs] “You can use these metrics to: Track LLM request latency and upstream provider processing time * Monitor token consumption ... * Calculate AI…”
   • [claimed-docs] “AI Gateway logs capture detailed information about the request and response payloads, token usage, model details, latency, and cost metrics.”

   proof ↗⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round to LiteLLM
   LiteLLM~partialcommunity4/10

   The product's source is publicly hosted on GitHub (BerriAI/litellm), confirmed by community references to 'public repos' being vandalized, showing the codebase is openly readable. However, the evidence pack never states or cites an actual license (e.g., MIT/Apache) for the repository, so the 'open license' claim itself is unconfirmed. Missing for 10: explicit license documentation/citation, first-party statement of licensing terms.

   • [github] “Drop-in OpenAI compatibility — swap providers without rewriting your code”
   • [community] “The GitHub account of LiteLLM's Founder/CTO (krrishdholakia) appears to have been fully compromised, with public repos vandalized to say 'te…”

   proof ↗⚑ flag

   Kong AI Gateway—none0/10

   Kong AI Gateway is a commercial/enterprise product built on Kong Gateway; the evidence pack contains only product documentation, API references, and probes for docs/CLI—no mention of source code availability or an open-source license for AI Gateway itself. Kong Gateway OSS is a separate product; nothing here confirms AI Gateway's source is published under an open license.

   ⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round to LiteLLM
   LiteLLM✓fullcommunity9/10

   LiteLLM explicitly ships a self-hosted LLM Gateway/Proxy (docs-5, docs-23) with configurable worker processes and CLI (docs-13/20), and this is corroborated by hands-on community use running it via Docker in a homelab (litellm-comm-6) and widespread adoption as a self-hosted gateway (litellm-comm-7). missing for 10: no independent benchmark or enterprise-scale self-hosting case study beyond a single homelab anecdote.

   • [claimed-docs] “Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI”
   • [claimed-docs] “Self-hosted [LLM Gateway (Proxy)](/docs/simple_proxy) with virtual keys, cost tracking, and an admin UI”
   • [claimed-docs] “The number of worker processes to spin up (uvicorn, gunicorn, or Granian --workers).”
   • [claimed-docs] “The number of worker processes to spin up (uvicorn, gunicorn, or Granian `--workers`).”
   • [community] “A user described running LiteLLM as a proxy in their homelab via the litellm/litellm docker image for local LLM gateway management, noting t…”
   • [community] “'That's a bad supply-chain attack, many folks use litellm as main gateway' — reflecting how widely used LiteLLM is as an LLM gateway.”

   proof ↗⚑ flag

   Kong AI Gateway~partialclaimed5/10

   Docs reference configuring AI Gateway capabilities 'on-prem' via plugins on Services and Routes, and a quickstart script to spin up a demo instance, implying self-hosted deployment is possible. However, there's no detailed self-hosting guide, licensing/open-source clarification, or infrastructure requirements documented in the evidence pack. Missing for 10: explicit self-hosting/installation documentation, open-source license details, infrastructure/deployment guides beyond a demo quickstart.

   • [claimed-docs] “This page shows you how to configure the same capabilities with AI plugins on Services and Routes.”
   • [claimed-docs] “You can use the quickstart script to get a demo instance of AI Gateway running almost instantly.”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round to Kong AI Gateway
   LiteLLM—none0/10

   LiteLLM is self-hostable (which lets users control physical data location by choosing their own deployment region), but no evidence pack item documents any explicit data residency/region-selection feature, region-pinning configuration, or compliance-region controls offered by the product itself.

   ⚑ flag

   Kong AI Gateway~partialclaimed3/10

   Kong AI Gateway documentation mentions an on-prem/self-hosted configuration option (configure-on-prem.md), which implies some ability to control where the gateway (and thus data in transit) runs, but there is no explicit documentation of region selection, data residency guarantees, or storage location controls for logs/analytics/cost data. missing for 10: explicit region/residency configuration options, documented data storage location controls, compliance statements about data residency.

   • [claimed-docs] “This page shows you how to configure the same capabilities with AI plugins on Services and Routes.”
   • [claimed-docs] “AI Gateway logs capture detailed information about the request and response payloads, token usage, model details, latency, and cost metrics.”

   proof ↗⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   LiteLLM—none0/10

   The evidence shows LiteLLM can disable logging of prompts/responses to its own logging providers (litellm-docs-12), but nothing indicates it offers a mechanism to opt out of model-training use by the underlying LLM providers (e.g., passing zero-retention/no-train flags to OpenAI/Anthropic/etc.). As a gateway, this is a plausible capability, but no evidence supports it.

   • [claimed-docs] “This will prevent the messages and responses from being logged to your logging provider, but request metadata - e.g. spend, will still be tr…”

   proof ↗⚑ flag

   Kong AI Gateway—none0/10

   Kong AI Gateway is an enterprise routing/governance layer for LLM traffic, so control over how provider data is used is a fair question, but the evidence pack contains no mention of training-data opt-out, zero-retention guarantees, or contractual terms preventing model providers from using proxied data for training — only general DLP/PII redaction and content-safety features are documented, which don't address this specific claim.

   ⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round to LiteLLM
   LiteLLM~partialclaimed5/10

   LiteLLM lets admins disable logging of request/response content to reduce data retention while still tracking metadata (litellm-docs-12), and being self-hosted gives users infrastructure-level control over where data lives. However, there is no documented explicit data-deletion API, retention-period settings, or data lifecycle/export controls in the evidence pack. Missing for 10: explicit deletion/purge API, configurable retention windows, documented data lifecycle policy, independent confirmation these logging controls work as described.

   • [claimed-docs] “This will prevent the messages and responses from being logged to your logging provider, but request metadata - e.g. spend, will still be tr…”
   • [claimed-docs] “Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI”

   proof ↗⚑ flag

   Kong AI Gateway—none0/10

   Evidence shows Kong AI Gateway logs request/response payloads, token usage, and costs (docs-10), and offers DLP/PII removal (docs-20) and semantic caching (docs-11), but none of the evidence describes configurable data retention periods or deletion/purge controls for logs, caches, or audit trails. Missing for 10: explicit retention policy settings, log/cache deletion or expiry controls, data subject deletion workflow.

   • [claimed-docs] “AI Gateway logs capture detailed information about the request and response payloads, token usage, model details, latency, and cost metrics.”
   • [claimed-docs] “Apply safety and DLP policies to prevent toxic content and remove personally identifiable information.”
   • [claimed-docs] “AI Semantic Cache - Plugin | Kong Docs”

   proof ↗⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   LiteLLM—none0/10

   The evidence covers logging integrations (Langfuse, OTel, S3, etc.) and a way to disable message/response logging to those *external* logging providers (litellm-docs-12), but nothing in the pack addresses opting out of LiteLLM's own product telemetry/usage tracking sent back to the vendor. As a self-hosted gateway this axis plausibly applies, but there is no documented telemetry opt-out flag or setting in the evidence pack. Missing for 10: any mention of a telemetry/usage-tracking toggle (e.g., an env var or config flag disabling anonymized usage reporting to the vendor), independent confirmation that no phone-home occurs.

   • [claimed-docs] “This will prevent the messages and responses from being logged to your logging provider, but request metadata - e.g. spend, will still be tr…”
   • [claimed-docs] “Log Proxy input, output, and exceptions using: Langfuse, OpenTelemetry, GCS, s3, Azure (Blob) Buckets...”

   proof ↗⚑ flag

   Kong AI Gateway—none0/10

   No evidence pack items mention any telemetry opt-out, privacy settings, or usage-tracking controls for Kong AI Gateway itself; all evidence concerns the product's own traffic logging/analytics features rather than user opt-out of vendor telemetry.

   ⚑ flag

Fallbacks

1. platform engineerConfigure automatic fallback to another model or provider when one fails

   weight 3 · round to LiteLLM
   LiteLLM✓fullcommunity9/10

   LiteLLM's proxy reliability docs explicitly describe configuring fallbacks so a failing model/provider automatically fails over to a healthy backup, including model-group fallback chains and content-policy-specific fallback handling, corroborated by a maintainer's stated motivation for reliable model switching across providers. Missing for 10: independent hands-on validation of fallback behavior in production beyond docs/maintainer statements.

   • [claimed-docs] “If a call fails after num_retries, LiteLLM falls back to another model group, so a failing model or provider automatically fails over to a h…”
   • [claimed-docs] “so a failing model or provider automatically fails over to a healthy backup”
   • [claimed-docs] “The request to `model="zephyr-beta"` will fail... litellm proxy will loop through all the model_groups specified in `fallbacks=["gpt-3.5-tur…”
   • [claimed-docs] “content_policy_fallbacks: For litellm.ContentPolicyViolationError - LiteLLM maps content policy violation errors across providers”
   • [community] “Show HN launch of LiteLLM Proxy Server: praised as 'super compelling' with maintainers noting motivation was avoiding messy 100-line if/else…”

   proof ↗⚑ flag

   Kong AI Gateway~partialclaimed5/10

   Kong AI Gateway documents load balancing across multiple LLM models/providers, which implies routing resilience, but the evidence never explicitly describes an automatic fallback mechanism triggered by provider failure (e.g., retry-on-error, circuit breaking, or failover behavior). Missing for 10: explicit fallback-on-failure documentation, failure detection/health-check mechanism, and any hands-on confirmation that fallback actually occurs when a provider errors out.

   • [claimed-docs] “AI Gateway provides load balancing capabilities to distribute requests across multiple LLM models.”
   • [claimed-docs] “The core of AI Gateway is the ability to serve AI Models from various AI Model Providers via a provider-agnostic API.”

   proof ↗⚑ flag

Load balancing

1. platform engineerLoad-balance traffic across providers, deployments, or keys by weight, latency, or cost

   weight 2 · round to Kong AI Gateway
   LiteLLM~partialclaimed5/10

   Docs confirm load-balancing across multiple deployments and automatic failover/fallback to healthy backups, which covers reliability-based routing, but the evidence pack never cites explicit weight-based, latency-based, or cost-based routing strategies by name (LiteLLM's known routing options aren't documented here). missing for 10: explicit documentation of weight-based routing strategy, latency-based routing strategy, and cost-based routing strategy configuration.

   • [claimed-docs] “Load-balance across multiple deployments (e.g. Azure/OpenAI)”
   • [claimed-docs] “If a call fails after num_retries, LiteLLM falls back to another model group, so a failing model or provider automatically fails over to a h…”
   • [claimed-docs] “so a failing model or provider automatically fails over to a healthy backup”
   • [claimed-docs] “The request to `model="zephyr-beta"` will fail... litellm proxy will loop through all the model_groups specified in `fallbacks=["gpt-3.5-tur…”
   • [claimed-docs] “content_policy_fallbacks: For litellm.ContentPolicyViolationError - LiteLLM maps content policy violation errors across providers”

   proof ↗⚑ flag

   Kong AI Gateway~partialclaimed6/10

   Kong AI Gateway explicitly documents load-balancing across multiple LLM models/providers (docs-17), and it also tracks and calculates per-request LLM costs (docs-18) and latency metrics (docs-9), which are the raw signals needed for cost- or latency-aware routing. However, the evidence pack never confirms the specific load-balancing algorithms (e.g., weighted, latency-based, cost-based) are selectable options, so it's unclear whether these criteria can actually be used as balancing strategies. Missing for 10: explicit documentation of weight-based, latency-based, and cost-based load-balancing algorithms/configuration options, and independent confirmation these modes work as described.

   • [claimed-docs] “AI Gateway provides load balancing capabilities to distribute requests across multiple LLM models.”
   • [claimed-docs] “AI Gateway calculates that figure for every request to a large language model (LLM) provider, matching what the provider actually bills.”
   • [claimed-docs] “You can use these metrics to: Track LLM request latency and upstream provider processing time * Monitor token consumption ... * Calculate AI…”

   proof ↗⚑ flag

Policy routing

1. ai-native userMy agent can switch models mid-task by policy — cost, capability, or availability — through gateway routing rules

   weight 3 · round to LiteLLM
   LiteLLM✓fullcommunity7/10

   LiteLLM's proxy documents concrete routing policies matching the story: load balancing across deployments, automatic fallback to a healthy backup model/provider on failure, and content-policy-specific fallbacks mapped across providers — all confirmed by community commentary on its origin as a fix for manual multi-provider if/else logic. Missing for 10: explicit cost-triggered routing (switching to cheaper model based on budget threshold) and any hands-on demonstration of live mid-task switching rather than static failover/fallback configuration.

   • [claimed-docs] “Load-balance across multiple deployments (e.g. Azure/OpenAI)”
   • [claimed-docs] “If a call fails after num_retries, LiteLLM falls back to another model group, so a failing model or provider automatically fails over to a h…”
   • [claimed-docs] “so a failing model or provider automatically fails over to a healthy backup”
   • [claimed-docs] “content_policy_fallbacks: For litellm.ContentPolicyViolationError - LiteLLM maps content policy violation errors across providers”
   • [claimed-docs] “The request to `model="zephyr-beta"` will fail... litellm proxy will loop through all the model_groups specified in `fallbacks=["gpt-3.5-tur…”
   • [community] “Show HN launch of LiteLLM Proxy Server: praised as 'super compelling' with maintainers noting motivation was avoiding messy 100-line if/else…”

   proof ↗⚑ flag

   Kong AI Gateway~partialclaimed5/10

   Kong AI Gateway documents load balancing across multiple LLM models (covering availability-based routing) and per-request cost calculation, which together imply some policy-driven model selection, but there is no explicit documentation of switching models based on capability or triggering mid-task, dynamic policy switches tied to cost or availability thresholds. Missing for 10: explicit cost/capability-based routing policy documentation, mid-task dynamic model switching evidence, and independent/hands-on corroboration of this routing-resilience behavior.

   • [claimed-docs] “AI Gateway provides load balancing capabilities to distribute requests across multiple LLM models.”
   • [claimed-docs] “AI Gateway calculates that figure for every request to a large language model (LLM) provider, matching what the provider actually bills.”
   • [claimed-docs] “AI Gateway can perform intelligent request routing, caching, and content filtering using semantic similarity queries.”

   proof ↗⚑ flag

Rate limits

1. platform engineerSmooth provider rate limits by spreading traffic across keys and queuing or throttling requests

   weight 2 · round to LiteLLM
   LiteLLM✓fullclaimed7/10

   LiteLLM's Router explicitly supports load-balancing across multiple deployments/keys (e.g., multiple Azure/OpenAI keys) and virtual keys carry per-key rpm_limit fields, directly enabling spreading traffic across provider keys to smooth rate limits, with retries/fallbacks for failures. Missing for 10: explicit documentation of a request queuing/throttling mechanism beyond rpm_limit and load balancing, and independent hands-on confirmation that this smooths real provider rate-limit errors under load.

   • [claimed-docs] “Load-balance across multiple deployments (e.g. Azure/OpenAI)”
   • [claimed-docs] “"team_alias": "my-new-team_4", ... "rpm_limit": 99”
   • [claimed-docs] “If a call fails after num_retries, LiteLLM falls back to another model group, so a failing model or provider automatically fails over to a h…”
   • [claimed-docs] “so a failing model or provider automatically fails over to a healthy backup”
   • [claimed-docs] “The request to `model="zephyr-beta"` will fail... litellm proxy will loop through all the model_groups specified in `fallbacks=["gpt-3.5-tur…”

   proof ↗⚑ flag

   Kong AI Gateway~partialclaimed6/10

   Kong AI Gateway documents load balancing across multiple LLM models/providers and a dedicated 'AI Rate Limiting Advanced' plugin for enforcing rate limits, which together support spreading traffic and throttling. However, there is no explicit documentation of spreading traffic across multiple API keys for the same provider or of request queuing behavior when limits are hit. Missing for 10: explicit key-rotation/multi-key load balancing, request queuing mechanics, and any hands-on/independent verification of these features working together.

   • [claimed-docs] “AI Gateway provides load balancing capabilities to distribute requests across multiple LLM models.”
   • [claimed-docs] “AI Rate Limiting Advanced - Plugin | Kong Docs”
   • [claimed-docs] “AI Gateway calculates that figure for every request to a large language model (LLM) provider, matching what the provider actually bills.”

   proof ↗⚑ flag

Retries

1. platform engineerSet automatic retry policies for transient provider errors

   weight 2 · round to LiteLLM
   LiteLLM✓fullclaimed7/10

   Docs explicitly describe a `num_retries` setting on the proxy, after which LiteLLM falls back to another model/group, directly supporting configurable retry policies for transient failures (docs-4, docs-21, docs-22). Missing for 10: detailed backoff/jitter configuration, granular retry-per-error-type mapping (only content policy fallback is separately documented), and independent hands-on verification of retry behavior under real transient provider errors.

   • [claimed-docs] “If a call fails after num_retries, LiteLLM falls back to another model group, so a failing model or provider automatically fails over to a h…”
   • [claimed-docs] “so a failing model or provider automatically fails over to a healthy backup”
   • [claimed-docs] “The request to `model="zephyr-beta"` will fail... litellm proxy will loop through all the model_groups specified in `fallbacks=["gpt-3.5-tur…”
   • [claimed-docs] “content_policy_fallbacks: For litellm.ContentPolicyViolationError - LiteLLM maps content policy violation errors across providers”

   proof ↗⚑ flag

   Kong AI Gateway~partialclaimed4/10

   Kong AI Gateway documents load balancing across multiple LLM models/providers (docs-17), which implies some failover capability, but the evidence never explicitly describes configurable automatic retry policies for transient provider errors (e.g., retry counts, backoff, error-code triggers). Missing for 10: explicit retry-policy documentation, retry configuration options, and evidence of behavior on transient errors specifically.

   • [claimed-docs] “AI Gateway provides load balancing capabilities to distribute requests across multiple LLM models.”
   • [claimed-docs] “The core of AI Gateway is the ability to serve AI Models from various AI Model Providers via a provider-agnostic API.”

   proof ↗⚑ flag

Streaming

1. developerStream token-by-token responses through the gateway from any provider

   weight 3 · round to Kong AI Gateway
   LiteLLM✓fullcommunity7/10

   LiteLLM's unified completion() interface and consistent output format across providers is well documented, and a maintainer explicitly confirmed the proxy 'supports streaming and function-calling in the same way as the openai-python SDK' across 50+ providers, matching the streaming-tools story directly. Missing for 10: a dedicated first-party streaming docs page/code snippet (e.g., stream=True) and independent hands-on confirmation beyond the single maintainer HN comment.

   • [community] “LiteLLM maintainer confirmed proxy supports streaming and function-calling in the same way as the openai-python SDK, and added Ollama integr…”
   • [claimed-docs] “Call any provider using the same completion() interface, with no API to re-learn for each one”
   • [claimed-docs] “Every response follows the OpenAI Chat Completions format, regardless of provider.”
   • [claimed-docs] “Call any provider using the same `completion()` interface, with no API to re-learn for each one”
   • [claimed-docs] “Consistent output format regardless of which provider or model you use”
   • [community] “Show HN launch of LiteLLM Proxy Server: praised as 'super compelling' with maintainers noting motivation was avoiding messy 100-line if/else…”

   proof ↗⚑ flag

   Kong AI Gateway✓fullclaimed8/10

   Kong AI Gateway has a dedicated streaming.md doc confirming token-by-token streaming from models is supported and rendered in real time, layered on top of its provider-agnostic API supporting multiple LLM providers. Missing for 10: independent/hands-on confirmation of streaming behavior across every provider and any details on streaming with proxy-added policies (e.g., guardrails) applied mid-stream.

   • [claimed-docs] “most models can stream each word (or sets of words and tokens) back to the client. This allows the chat response to be rendered in real time…”
   • [claimed-docs] “The core of AI Gateway is the ability to serve AI Models from various AI Model Providers via a provider-agnostic API.”

   proof ↗⚑ flag

Tool calling

1. developerRequest structured JSON-schema outputs across providers

   weight 1 · round drawn
   LiteLLM—none0/10

   The evidence pack covers LiteLLM's unified completion() interface, consistent OpenAI-format responses, retries/fallbacks, and proxy features, but nowhere mentions structured/JSON-schema output support (e.g. response_format, json_schema mode) across providers. This is a fair axis for a multi-provider LLM gateway, but no citation substantiates the specific capability.

   ⚑ flag

   Kong AI Gateway—none0/10

   The evidence pack covers provider-agnostic proxying, streaming, load balancing, cost tracking, and safety policies, but no document mentions structured/JSON-schema output enforcement or normalization across providers. This is a fair capability to expect from an AI gateway, but no evidence supports it.

   ⚑ flag

2. developerMake tool and function calls across different providers with a consistent schema

   weight 3 · round to LiteLLM
   LiteLLM✓fullcommunity8/10

   LiteLLM documents a unified completion() interface that normalizes tool/function calls and outputs to the OpenAI Chat Completions schema across 100+ providers, and a maintainer confirms function-calling parity with the openai-python SDK. missing for 10: independent hands-on verification of tool-call schema consistency across many specific providers, and no dedicated docs excerpt specifically on function-calling parameter translation edge cases.

   • [claimed-docs] “Call any provider using the same completion() interface, with no API to re-learn for each one”
   • [claimed-docs] “Every response follows the OpenAI Chat Completions format, regardless of provider.”
   • [claimed-docs] “Call any provider using the same `completion()` interface, with no API to re-learn for each one”
   • [claimed-docs] “Consistent output format regardless of which provider or model you use”
   • [github] “Drop-in OpenAI compatibility — swap providers without rewriting your code”
   • [community] “LiteLLM maintainer confirmed proxy supports streaming and function-calling in the same way as the openai-python SDK, and added Ollama integr…”

   proof ↗⚑ flag

   Kong AI Gateway~partialclaimed6/10

   Kong AI Gateway documents a provider-agnostic API for LLM traffic and an AI MCP Proxy/Server that converts REST APIs into MCP tools, suggesting a consistent tool-calling interface across providers and models. However, there is no explicit documentation showing normalization of function/tool-call schemas (e.g., translating OpenAI-style function calls to Anthropic tool-use format) for the same request across different providers. Missing for 10: explicit cross-provider function-call schema translation examples, hands-on/independent verification of consistent tool-call behavior across providers.

   • [claimed-docs] “The core of AI Gateway is the ability to serve AI Models from various AI Model Providers via a provider-agnostic API.”
   • [claimed-docs] “AI MCP Proxy bridges Kong-managed Services to the Model Context Protocol, converting REST APIs into MCP tools or fronting upstream MCP serve…”
   • [claimed-docs] “Turn any API into an MCP server using the AI MCP Server entity. This approach does not require an LLM and provides full control over product…”
   • [claimed-docs] “Govern LLM, MCP, and agent-to-agent (A2A) traffic with the same Kong AI Gateway.”

   proof ↗⚑ flag

Catalog

1. developerBrowse or query a catalog of available models with pricing and context-window metadata

   weight 2 · round drawn
   LiteLLM—none0/10

   While LiteLLM claims support for 100+ LLMs and tracks spend/cost, the evidence pack contains no mention of a browsable/queryable catalog listing models with pricing and context-window metadata (e.g., no model_cost table, /model/info endpoint, or docs page referencing context window sizes). Cost-tracking docs (litellm-docs-6) cover spend tracking, not a metadata catalog.

   ⚑ flag

   Kong AI Gateway—none0/10

   Evidence shows Kong AI Gateway supports multi-provider routing and calculates per-request costs, but there is no documented catalog/endpoint for browsing or querying available models with pricing and context-window metadata — the closest evidence (model-cost-management, list-ai-gateways API) covers cost calculation per request and listing gateway instances, not a model metadata catalog.

   • [claimed-docs] “AI Gateway calculates that figure for every request to a large language model (LLM) provider, matching what the provider actually bills.”
   • [claimed-docs] “list-ai-gateways summary: List AI Gateways description: Returns a list of AI Gateways in the organization.”
   • [claimed-docs] “The core of AI Gateway is the ability to serve AI Models from various AI Model Providers via a provider-agnostic API.”

   proof ↗⚑ flag

Compatibility

1. developerPoint existing OpenAI-compatible code at the gateway by changing only the base URL and key

   weight 3 · round to LiteLLM
   LiteLLM✓fullcommunity8/10

   LiteLLM's docs and GitHub explicitly advertise drop-in OpenAI compatibility with consistent Chat Completions format across providers, and the self-hosted proxy exposes a single gateway endpoint with virtual keys, matching the exact story of only swapping base URL and key. Community evidence (Show HN, HN threads) corroborates real-world use as a drop-in gateway for existing OpenAI-style code. Missing for 10: independent hands-on verification of literal 'change only base_url+key' migration and no explicit code diff example in the pack.

   • [claimed-docs] “Call any provider using the same completion() interface, with no API to re-learn for each one”
   • [claimed-docs] “Every response follows the OpenAI Chat Completions format, regardless of provider.”
   • [github] “Drop-in OpenAI compatibility — swap providers without rewriting your code”
   • [claimed-docs] “Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI”
   • [claimed-docs] “Self-hosted [LLM Gateway (Proxy)](/docs/simple_proxy) with virtual keys, cost tracking, and an admin UI”
   • [community] “Show HN launch of LiteLLM Proxy Server: praised as 'super compelling' with maintainers noting motivation was avoiding messy 100-line if/else…”
   • [community] “LiteLLM maintainer confirmed proxy supports streaming and function-calling in the same way as the openai-python SDK, and added Ollama integr…”

   proof ↗⚑ flag

   Kong AI Gateway~partialclaimed6/10

   Kong AI Gateway is documented as a provider-agnostic API layer that proxies LLM requests (chat/completions endpoints) with streaming support, which implies OpenAI-style compatibility, but no evidence explicitly confirms that existing OpenAI SDK code works unmodified by only swapping base URL and API key. missing for 10: explicit doc/example showing an OpenAI SDK client redirected via base_url with no code changes, and confirmation that request/response schemas exactly mirror OpenAI's format.

   • [claimed-docs] “The core of AI Gateway is the ability to serve AI Models from various AI Model Providers via a provider-agnostic API.”
   • [claimed-docs] “most models can stream each word (or sets of words and tokens) back to the client. This allows the chat response to be rendered in real time…”
   • [claimed-docs] “This page shows you how to configure the same capabilities with AI plugins on Services and Routes.”

   proof ↗⚑ flag

One endpoint

1. developerCall many model providers through one consistent API

   weight 3 · round drawn
   LiteLLM✓fullcommunity8/10

   LiteLLM's core value proposition—calling 100+ providers through one OpenAI-compatible completion() interface with consistent output format—is well documented (litellm-docs-1, litellm-docs-2, litellm-docs-16, litellm-docs-17, litellm-gh-1) and corroborated by independent Show HN community feedback praising it as a way to standardize disparate AI APIs and quickly add new providers (litellm-comm-12, litellm-comm-13, litellm-comm-14, litellm-comm-15). Some community sentiment flags proxy flakiness/outages, but that concerns operational reliability, not whether the unified-API abstraction itself works. Missing for 10: no independent benchmark verifying identical output format across all supported providers, and no public OpenAPI spec was found for the proxy (litellm-probe-3).

   • [claimed-docs] “Call any provider using the same completion() interface, with no API to re-learn for each one”
   • [claimed-docs] “Every response follows the OpenAI Chat Completions format, regardless of provider.”
   • [claimed-docs] “Call any provider using the same `completion()` interface, with no API to re-learn for each one”
   • [claimed-docs] “Consistent output format regardless of which provider or model you use”
   • [github] “Drop-in OpenAI compatibility — swap providers without rewriting your code”
   • [community] “Show HN launch of LiteLLM Proxy Server: praised as 'super compelling' with maintainers noting motivation was avoiding messy 100-line if/else…”
   • [community] “LiteLLM maintainer confirmed proxy supports streaming and function-calling in the same way as the openai-python SDK, and added Ollama integr…”
   • [community] “Show HN launch of the original LiteLLM library; a user praised it: 'This is amazing. Really needed something like this to standardize all my…”
   • [community] “Maintainer distinguished LiteLLM from a similar library (llm-client-sdk) by stating 'while your library looks really powerful my goal with L…”

   proof ↗⚑ flag

   Kong AI Gateway✓fullclaimed8/10

   Kong AI Gateway explicitly provides a provider-agnostic API to route to many LLM providers, with load balancing and unified logging/observability across providers, matching the core unified-API story. Missing for 10: independent/hands-on developer corroboration of consistent request/response schema across providers, and no explicit list of exact supported providers in evidence.

   • [claimed-docs] “The core of AI Gateway is the ability to serve AI Models from various AI Model Providers via a provider-agnostic API.”
   • [claimed-docs] “AI Gateway provides load balancing capabilities to distribute requests across multiple LLM models.”
   • [claimed-docs] “You can use these metrics to: Track LLM request latency and upstream provider processing time * Monitor token consumption ... * Calculate AI…”
   • [claimed-docs] “AI Gateway calculates that figure for every request to a large language model (LLM) provider, matching what the provider actually bills.”

   proof ↗⚑ flag