How LiteLLM’s scores are calculated

Point an agent at llms.txt or agent-oriented docsweight 2

2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max

• [probe] https://docs.litellm.ai/llms.txt“PROBE llms.txt: HTTP 200 at https://docs.litellm.ai/llms.txt # https://docs.litellm.ai/ llms.txt - [LiteLLM Overview](https://docs.litellm.ai/): Access and manage 100+ LLMs with Li”
• [probe] https://docs.litellm.ai/docs/.md“PROBE docs-md: HTTP 404 at https://docs.litellm.ai/docs/.md”

Run the product headlessly / in CI for automationweight 2

2 (weight) × 6 (quality) × 0.6 (partial) = 7.2 of 20 max

• [claimed-docs] https://docs.litellm.ai/docs/proxy/cli“The number of worker processes to spin up (uvicorn, gunicorn, or Granian --workers).”
• [claimed-docs] https://docs.litellm.ai/docs/proxy/cli“The number of worker processes to spin up (uvicorn, gunicorn, or Granian `--workers`).”
• [probe] https://docs.litellm.ai/docs/proxy/cli“official CLI documented at https://docs.litellm.ai/docs/proxy/cli”
• [community] https://news.ycombinator.com/item?id=47501426“A user described running LiteLLM as a proxy in their homelab via the litellm/litellm docker image for local LLM gateway management, noting they were on an unaffected older version (1.77).”
• [claimed-docs] https://docs.litellm.ai/docs/“Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI”

Plug MCP servers into this product so it can use their toolsweight 3

3 (weight) × 8 (quality) × 1.0 (full) = 24.0 of 30 max

• [claimed-docs] https://docs.litellm.ai/docs/mcp“LiteLLM Proxy provides an MCP Gateway that allows you to use a fixed endpoint for all MCP tools and control MCP access by Key, Team.”
• [claimed-docs] https://docs.litellm.ai/docs/mcp“On the LiteLLM UI, Navigate to "MCP Servers" and click "Add New MCP Server".”

Connect an agent via an official MCP serverweight 3

3 (weight) × 7 (quality) × 1.0 (full) = 21.0 of 30 max

• [claimed-docs] https://docs.litellm.ai/docs/mcp“LiteLLM Proxy provides an MCP Gateway that allows you to use a fixed endpoint for all MCP tools and control MCP access by Key, Team.”
• [claimed-docs] https://docs.litellm.ai/docs/mcp“On the LiteLLM UI, Navigate to "MCP Servers" and click "Add New MCP Server".”

Use an official CLIweight 2

2 (weight) × 6 (quality) × 0.6 (partial) = 7.2 of 20 max

• [claimed-docs] https://docs.litellm.ai/docs/proxy/cli“The number of worker processes to spin up (uvicorn, gunicorn, or Granian --workers).”
• [claimed-docs] https://docs.litellm.ai/docs/proxy/cli“The number of worker processes to spin up (uvicorn, gunicorn, or Granian `--workers`).”
• [probe] https://docs.litellm.ai/docs/proxy/cli“official CLI documented at https://docs.litellm.ai/docs/proxy/cli”

Drive the product through a documented public APIweight 3

3 (weight) × 7 (quality) × 1.0 (full) = 21.0 of 30 max

• [claimed-docs] https://docs.litellm.ai/docs/“Call any provider using the same completion() interface, with no API to re-learn for each one”
• [claimed-docs] https://docs.litellm.ai/docs/“Every response follows the OpenAI Chat Completions format, regardless of provider.”
• [github] https://github.com/BerriAI/litellm“Drop-in OpenAI compatibility — swap providers without rewriting your code”
• [claimed-docs] https://docs.litellm.ai/docs/“Call any provider using the same `completion()` interface, with no API to re-learn for each one”
• [claimed-docs] https://docs.litellm.ai/docs/“Consistent output format regardless of which provider or model you use”
• [community] https://hn.algolia.com/api/v1/items/37095542“LiteLLM maintainer confirmed proxy supports streaming and function-calling in the same way as the openai-python SDK, and added Ollama integration within hours of user requests for local LLM support.”
• [probe] https://docs.litellm.ai/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.litellm.ai/openapi.json, https://docs.litellm.ai/swagger.json, https://docs.litellm.ai/api/openapi.json, https://docs.litellm.ai/.well-known/openapi.json)”

Issue scoped/least-privilege API credentials for an agentweight 2

2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max

• [claimed-docs] https://docs.litellm.ai/docs/proxy/virtual_keys“Track Spend, and control model access via virtual keys for the proxy”
• [claimed-docs] https://docs.litellm.ai/docs/proxy/users“Personal budgets: Create virtual keys without team_id for individual spending limits”
• [claimed-docs] https://docs.litellm.ai/docs/proxy/users“Apply a budget across all calls on the proxy... budget_duration: 30d # (str) frequency of reset”
• [claimed-docs] https://docs.litellm.ai/docs/proxy/users“"team_alias": "my-new-team_4", ... "rpm_limit": 99”
• [claimed-docs] https://docs.litellm.ai/docs/proxy/cost_tracking“Track spend for keys, users, and teams across 100+ LLMs.”
• [community] https://news.ycombinator.com/item?id=47501426“'Does anyone know a good alternate project... LiteLLM has been getting worse and trying to get me to upgrade to a paid version. I also had issues with creating tokens for other users etc.'”

Build against official SDKsweight 2

2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max

• [claimed-docs] https://docs.litellm.ai/docs/“Call any provider using the same completion() interface, with no API to re-learn for each one”
• [claimed-docs] https://docs.litellm.ai/docs/“Every response follows the OpenAI Chat Completions format, regardless of provider.”
• [github] https://github.com/BerriAI/litellm“Drop-in OpenAI compatibility — swap providers without rewriting your code”
• [claimed-docs] https://docs.litellm.ai/docs/“Call any provider using the same `completion()` interface, with no API to re-learn for each one”
• [claimed-docs] https://docs.litellm.ai/docs/“Consistent output format regardless of which provider or model you use”
• [community] https://hn.algolia.com/api/v1/items/37095542“LiteLLM maintainer confirmed proxy supports streaming and function-calling in the same way as the openai-python SDK, and added Ollama integration within hours of user requests for local LLM support.”
• [community] https://hn.algolia.com/api/v1/items/36887711“Show HN launch of the original LiteLLM library; a user praised it: 'This is amazing. Really needed something like this to standardize all my different AI APIs! ... I love how quickly your team is shipping!'”

Subscribe to events via webhooksweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Explore an interactive API reference with runnable examplesweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

• [probe] https://docs.litellm.ai/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.litellm.ai/openapi.json, https://docs.litellm.ai/swagger.json, https://docs.litellm.ai/api/openapi.json, https://docs.litellm.ai/.well-known/openapi.json)”
• [probe] https://docs.litellm.ai/docs/.md“PROBE docs-md: HTTP 404 at https://docs.litellm.ai/docs/.md”
• [claimed-docs] https://docs.litellm.ai/docs/“Call any provider using the same completion() interface, with no API to re-learn for each one”
• [claimed-docs] https://docs.litellm.ai/docs/“Call any provider using the same `completion()` interface, with no API to re-learn for each one”

Download a machine-readable API spec (OpenAPI or equivalent)weight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

• [probe] https://docs.litellm.ai/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.litellm.ai/openapi.json, https://docs.litellm.ai/swagger.json, https://docs.litellm.ai/api/openapi.json, https://docs.litellm.ai/.well-known/openapi.json)”
• [probe] https://docs.litellm.ai/docs/.md“PROBE docs-md: HTTP 404 at https://docs.litellm.ai/docs/.md”

Test against a sandbox environment without touching production dataweight 1

1 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 10 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Rely on versioned APIs with a documented deprecation policyweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

• [probe] https://docs.litellm.ai/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.litellm.ai/openapi.json, https://docs.litellm.ai/swagger.json, https://docs.litellm.ai/api/openapi.json, https://docs.litellm.ai/.well-known/openapi.json)”
• [probe] https://docs.litellm.ai/docs/.md“PROBE docs-md: HTTP 404 at https://docs.litellm.ai/docs/.md”

Do everything through the API that I can do in the UIweight 2

2 (weight) × 6 (quality) × 0.6 (partial) = 7.2 of 20 max

• [claimed-docs] https://docs.litellm.ai/docs/“Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI”
• [claimed-docs] https://docs.litellm.ai/docs/proxy/virtual_keys“Track Spend, and control model access via virtual keys for the proxy”
• [claimed-docs] https://docs.litellm.ai/docs/mcp“LiteLLM Proxy provides an MCP Gateway that allows you to use a fixed endpoint for all MCP tools and control MCP access by Key, Team.”
• [claimed-docs] https://docs.litellm.ai/docs/mcp“On the LiteLLM UI, Navigate to "MCP Servers" and click "Add New MCP Server".”
• [claimed-docs] https://docs.litellm.ai/docs/proxy/cli“The number of worker processes to spin up (uvicorn, gunicorn, or Granian --workers).”
• [probe] https://docs.litellm.ai/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.litellm.ai/openapi.json, https://docs.litellm.ai/swagger.json, https://docs.litellm.ai/api/openapi.json, https://docs.litellm.ai/.well-known/openapi.json)”
• [probe] https://docs.litellm.ai/docs/proxy/cli“official CLI documented at https://docs.litellm.ai/docs/proxy/cli”

Export all of my data in open formats and leaveweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

• [claimed-docs] https://docs.litellm.ai/docs/“Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI”
• [claimed-docs] https://docs.litellm.ai/docs/proxy/cost_tracking“Track spend for keys, users, and teams across 100+ LLMs.”
• [claimed-docs] https://docs.litellm.ai/docs/proxy/logging“Log Proxy input, output, and exceptions using: Langfuse, OpenTelemetry, GCS, s3, Azure (Blob) Buckets...”

Read the product's source under an open licenseweight 2

2 (weight) × 4 (quality) × 0.6 (partial) = 4.8 of 20 max

• [github] https://github.com/BerriAI/litellm“Drop-in OpenAI compatibility — swap providers without rewriting your code”
• [community] https://news.ycombinator.com/item?id=47501426“The GitHub account of LiteLLM's Founder/CTO (krrishdholakia) appears to have been fully compromised, with public repos vandalized to say 'teampcp owns BerriAI.'”

Self-host the core productweight 3

3 (weight) × 9 (quality) × 1.0 (full) = 27.0 of 30 max

• [claimed-docs] https://docs.litellm.ai/docs/“Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI”
• [claimed-docs] https://docs.litellm.ai/docs/“Self-hosted [LLM Gateway (Proxy)](/docs/simple_proxy) with virtual keys, cost tracking, and an admin UI”
• [claimed-docs] https://docs.litellm.ai/docs/proxy/cli“The number of worker processes to spin up (uvicorn, gunicorn, or Granian --workers).”
• [claimed-docs] https://docs.litellm.ai/docs/proxy/cli“The number of worker processes to spin up (uvicorn, gunicorn, or Granian `--workers`).”
• [community] https://news.ycombinator.com/item?id=47501426“A user described running LiteLLM as a proxy in their homelab via the litellm/litellm docker image for local LLM gateway management, noting they were on an unaffected older version (1.77).”
• [community] https://news.ycombinator.com/item?id=47501426“'That's a bad supply-chain attack, many folks use litellm as main gateway' — reflecting how widely used LiteLLM is as an LLM gateway.”

Get AI-generated insights and suggestions from my data inside the productweight 2

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Set up automations that run autonomously in the backgroundweight 2

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Delegate tasks to a built-in AI assistant inside the productweight 3

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Operate the product with natural-language commandsweight 2

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Perform bulk operations across many items at onceweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Define rules that trigger actions automatically on eventsweight 3

3 (weight) × 5 (quality) × 0.6 (partial) = 9.0 of 30 max

• [claimed-docs] https://docs.litellm.ai/docs/proxy/reliability“If a call fails after num_retries, LiteLLM falls back to another model group, so a failing model or provider automatically fails over to a healthy backup.”
• [claimed-docs] https://docs.litellm.ai/docs/proxy/reliability“content_policy_fallbacks: For litellm.ContentPolicyViolationError - LiteLLM maps content policy violation errors across providers”
• [claimed-docs] https://docs.litellm.ai/docs/proxy/reliability“so a failing model or provider automatically fails over to a healthy backup”
• [claimed-docs] https://docs.litellm.ai/docs/proxy/reliability“The request to `model="zephyr-beta"` will fail... litellm proxy will loop through all the model_groups specified in `fallbacks=["gpt-3.5-turbo"]`”
• [claimed-docs] https://docs.litellm.ai/docs/proxy/users“Apply a budget across all calls on the proxy... budget_duration: 30d # (str) frequency of reset”
• [claimed-docs] https://docs.litellm.ai/docs/proxy/users“"team_alias": "my-new-team_4", ... "rpm_limit": 99”

Schedule recurring jobs or workflowsweight 2

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Version, review, and roll back my automationsweight 1

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh