LiteLLM

Plug MCP servers into this product so it can use their tools G

Agent access

Connect an agent via an official MCP server G

Agent access

Drive the product through a documented public API G

Agent access

Delegate tasks to a built-in AI assistant inside the product G

Agentic features

Build against official SDKs G

Agent access

Issue scoped/least-privilege API credentials for an agent G

Agent access

Point an agent at llms.txt or agent-oriented docs G

Agent access

Run the product headlessly / in CI for automation G

Agent access

Use an official CLI G

Agent access

Download a machine-readable API spec (OpenAPI or equivalent) G

Api quality

Explore an interactive API reference with runnable examples G

Api quality

Rely on versioned APIs with a documented deprecation policy G

Api quality

Get AI-generated insights and suggestions from my data inside the product G

Agentic features

Operate the product with natural-language commands G

Agentic features

Set up automations that run autonomously in the background G

Agentic features

Subscribe to events via webhooks G

Agent access

Test against a sandbox environment without touching production data G

Api quality

Configure automatic fallback to another model or provider when one fails C

Fallbacks

Self-host the core product G

Set hard budgets and spend limits per key, team, or user C

Budgets

Track spend per model, key, team, or user across all providers in one place C

Spend tracking

Call many model providers through one consistent API C

One endpoint

Make tool and function calls across different providers with a consistent schema C

Tool calling

Point existing OpenAI-compatible code at the gateway by changing only the base URL and key C

Compatibility

Mint gateway-managed keys for teams and apps without exposing raw provider keys C

Virtual keys

My agent can switch models mid-task by policy — cost, capability, or availability — through gateway routing rules C

Policy routing

Stream token-by-token responses through the gateway from any provider C

Streaming

Inspect logged requests and responses with latency, token counts, and cost attached C

Logs

Define rules that trigger actions automatically on events G

Provision gateways, keys, and budgets programmatically through an admin API C

Programmatic admin

Export all of my data in open formats and leave G

Prevent my data from being used to train AI models G

Cache responses at the gateway to cut cost and latency on repeated requests C

Caching

Give an autonomous agent its own key with budget and rate guardrails so it cannot run away on spend C

Agent guardrails

Set automatic retry policies for transient provider errors C

Retries

Smooth provider rate limits by spreading traffic across keys and queuing or throttling requests C

Rate limits

Bring my own provider API keys and have the gateway use them for my traffic G

Byok

Do everything through the API that I can do in the UI G

Control data retention and deletion G

Load-balance traffic across providers, deployments, or keys by weight, latency, or cost C

Load balancing

Read the product's source under an open license G

Run traffic through gateway infrastructure that adds minimal latency overhead to provider calls C

Latency

Opt out of telemetry and usage tracking G

Browse or query a catalog of available models with pricing and context-window metadata G

Catalog

Choose where my data is stored (region/residency) G

Perform bulk operations across many items at once G

Schedule recurring jobs or workflows G

Export gateway logs and traces to my own observability stack G

Integrations

Request structured JSON-schema outputs across providers C

Tool calling

Version, review, and roll back my automations G