Skip to content
ProductArena

How Vapi’s scores are calculated

The full audit trail, recomputed from the verdict data at build time through the same code that produced the leaderboard: verdict × quality × story weight per cell, cells sum to dimension scores, dimensions blend into the PA Score. Every number on the product page is reproducible from this page alone; for why the formula looks like this, see the methodology.

verdict factors: full ×1.0 · partial ×0.6 · disputed ×0.3 · none ×0.0 · n/a excluded from both sides · cell points = weight × quality × factor · cell max = weight × 10

PA Score28/100

Agent-ready 61.3 × 0.30 = 18.39

API quality 10.0 × 0.20 = 2.00

Openness 16.0 × 0.20 = 3.20

Built-in AI 20.9 × 0.15 = 3.13

Automation 11.3 × 0.15 = 1.70

(18.39 + 2.00 + 3.20 + 3.13 + 1.70) ÷ (0.30 + 0.20 + 0.20 + 0.15 + 0.15) = 28.42 ÷ 1.00 = 28.4

Scores are stored to 1 decimal; the product page’s pills round to whole numbers for display. Each dimension below shows the stories, verdicts, and cited evidence behind its number.

Agent-ready61.3/100×0.30 of the PA blend

Outside-in: can YOUR agent reach and drive this product — API, MCP, CLI, headless runs, agent docs.

Point an agent at llms.txt or agent-oriented docsweight 2

2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max

  • [probe] https://docs.vapi.ai/llms.txt“PROBE llms.txt: HTTP 200 at https://docs.vapi.ai/llms.txt # Vapi ## Instructions for AI Agents - For clean Markdown of any page, append `.md` to the page URL - For section-spec”

Run the product headlessly / in CI for automationweight 2

2 (weight) × 6 (quality) × 0.6 (partial) = 7.2 of 20 max

  • [claimed-docs] https://docs.vapi.ai/cli.md“Manage assistants, phone numbers, and calls from your terminal”
  • [claimed-docs] https://docs.vapi.ai/cli.md“Build, test, and deploy voice AI applications without leaving your development environment.”
  • [claimed-docs] https://docs.vapi.ai/cli.md“The CLI auto-detects your tech stack and sets up everything you need.”
  • [probe] https://docs.vapi.ai/cli“PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bash`) and `vapi --version` printed `vapi version 0.2.1` keylessly, with man pages for assistant and call management.”
  • [claimed-docs] https://docs.vapi.ai/observability/evals-quickstart.md“Evals is Vapi's AI agent testing framework that enables you to systematically test assistants and squads using mock conversations with automated validation.”
  • [claimed-docs] https://docs.vapi.ai/observability/evals-quickstart.md“you'll create mock conversations, define expected behaviors, and validate your agents work correctly before production”
  • [probe] https://docs.vapi.ai/sdk/mcp-server“PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — the endpoint is live and bearer-key-gated exactly as documented (API key as Authorization: Bearer).”

Plug MCP servers into this product so it can use their toolsweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

  • [claimed-docs] https://docs.vapi.ai/sdk/mcp-server.md“so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
  • [claimed-docs] https://docs.vapi.ai/sdk/mcp-server.md“The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant”
  • [claimed-docs] https://docs.vapi.ai/tools.md“Create your own webhook-based tools to extend assistant capabilities”
  • [claimed-docs] https://docs.vapi.ai/tools.md“Custom tools that you create... interact with your systems via webhooks”
  • [probe] https://docs.vapi.ai/sdk/mcp-server“official MCP server documented at https://docs.vapi.ai/sdk/mcp-server”

Connect an agent via an official MCP serverweight 3

3 (weight) × 9 (quality) × 1.0 (full) = 27.0 of 30 max

  • [claimed-docs] https://docs.vapi.ai/sdk/mcp-server.md“so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
  • [claimed-docs] https://docs.vapi.ai/sdk/mcp-server.md“The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant”
  • [probe] https://docs.vapi.ai/sdk/mcp-server“official MCP server documented at https://docs.vapi.ai/sdk/mcp-server”
  • [probe] https://docs.vapi.ai/sdk/mcp-server“PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — the endpoint is live and bearer-key-gated exactly as documented (API key as Authorization: Bearer).”

Use an official CLIweight 2

2 (weight) × 9 (quality) × 1.0 (full) = 18.0 of 20 max

  • [claimed-docs] https://docs.vapi.ai/cli.md“Manage assistants, phone numbers, and calls from your terminal”
  • [claimed-docs] https://docs.vapi.ai/cli.md“vapi listen --forward-to localhost:3000/webhook”
  • [claimed-docs] https://docs.vapi.ai/cli.md“Switch between organizations and environments seamlessly”
  • [claimed-docs] https://docs.vapi.ai/cli.md“Forward webhooks to your local server for debugging”
  • [claimed-docs] https://docs.vapi.ai/cli.md“Build, test, and deploy voice AI applications without leaving your development environment.”
  • [claimed-docs] https://docs.vapi.ai/cli.md“The CLI auto-detects your tech stack and sets up everything you need.”
  • [probe] https://docs.vapi.ai/cli“official CLI documented at https://docs.vapi.ai/cli”
  • [probe] https://docs.vapi.ai/cli“PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bash`) and `vapi --version` printed `vapi version 0.2.1` keylessly, with man pages for assistant and call management.”

Drive the product through a documented public APIweight 3

3 (weight) × 9 (quality) × 1.0 (full) = 27.0 of 30 max

  • [claimed-docs] https://docs.vapi.ai/quickstart/phone.md“const assistant = await vapi.assistant”
  • [claimed-docs] https://docs.vapi.ai/cli.md“Manage assistants, phone numbers, and calls from your terminal”
  • [claimed-docs] https://docs.vapi.ai/sdk/mcp-server.md“so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
  • [claimed-docs] https://docs.vapi.ai/sdk/mcp-server.md“The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant”
  • [probe] https://docs.vapi.ai/llms.txt“PROBE llms.txt: HTTP 200 at https://docs.vapi.ai/llms.txt # Vapi ## Instructions for AI Agents - For clean Markdown of any page, append `.md` to the page URL - For section-spec”
  • [probe] https://docs.vapi.ai/sdk/mcp-server“official MCP server documented at https://docs.vapi.ai/sdk/mcp-server”
  • [probe] https://docs.vapi.ai/cli“official CLI documented at https://docs.vapi.ai/cli”
  • [probe] https://docs.vapi.ai/cli“PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bash`) and `vapi --version` printed `vapi version 0.2.1` keylessly, with man pages for assistant and call management.”
  • [probe] https://docs.vapi.ai/sdk/mcp-server“PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — the endpoint is live and bearer-key-gated exactly as documented (API key as Authorization: Bearer).”

Issue scoped/least-privilege API credentials for an agentweight 2

2 (weight) × 3 (quality) × 0.6 (partial) = 3.6 of 20 max

  • [claimed-docs] https://docs.vapi.ai/enterprise.md“Single Sign On (SSO) supported for Okta, Azure AD, SAML, and OIDC * Role Based Access Control (RBAC)”
  • [claimed-docs] https://docs.vapi.ai/enterprise.md“Single Sign On (SSO) supported for Okta, Azure AD, SAML, and OIDC”
  • [probe] https://docs.vapi.ai/sdk/mcp-server“PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — the endpoint is live and bearer-key-gated exactly as documented (API key as Authorization: Bearer).”

Build against official SDKsweight 2

2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max

  • [claimed-docs] https://docs.vapi.ai/quickstart/phone.md“const assistant = await vapi.assistant”
  • [claimed-docs] https://docs.vapi.ai/cli.md“Manage assistants, phone numbers, and calls from your terminal”
  • [claimed-docs] https://docs.vapi.ai/sdk/mcp-server.md“so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
  • [claimed-docs] https://docs.vapi.ai/sdk/mcp-server.md“The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant”
  • [claimed-docs] https://docs.vapi.ai/cli.md“Your IDE's AI assistant (Cursor, Windsurf, VSCode) gains complete, accurate knowledge of Vapi's APIs and best practices. No more hallucinated code or outdated examples.”
  • [probe] https://docs.vapi.ai/sdk/mcp-server“official MCP server documented at https://docs.vapi.ai/sdk/mcp-server”
  • [probe] https://docs.vapi.ai/cli“official CLI documented at https://docs.vapi.ai/cli”
  • [probe] https://docs.vapi.ai/cli“PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bash`) and `vapi --version` printed `vapi version 0.2.1` keylessly, with man pages for assistant and call management.”
  • [probe] https://docs.vapi.ai/sdk/mcp-server“PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — the endpoint is live and bearer-key-gated exactly as documented (API key as Authorization: Bearer).”

Subscribe to events via webhooksweight 2

2 (weight) × 7 (quality) × 1.0 (full) = 14.0 of 20 max

  • [claimed-docs] https://docs.vapi.ai/tools.md“Create your own webhook-based tools to extend assistant capabilities”
  • [claimed-docs] https://docs.vapi.ai/server-url/events.md“When tools are triggered, your Server URL receives a `tool-calls` message”
  • [claimed-docs] https://docs.vapi.ai/server-url/events.md“Vapi supports OpenAI-style tool/function calling. Assistants can ping your server to perform actions.”
  • [claimed-docs] https://docs.vapi.ai/cli.md“Forward webhooks to your local server for debugging”
  • [claimed-docs] https://docs.vapi.ai/tools.md“Custom tools that you create... interact with your systems via webhooks”

Agent-ready = 128.8 ÷ 210 × 100 = 61.3

API quality10.0/100×0.20 of the PA blend

The programmable surface once an agent is there — machine-readable spec, interactive docs, sandbox, versioning discipline.

Explore an interactive API reference with runnable examplesweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Download a machine-readable API spec (OpenAPI or equivalent)weight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Test against a sandbox environment without touching production dataweight 1

1 (weight) × 7 (quality) × 1.0 (full) = 7.0 of 10 max

  • [claimed-docs] https://docs.vapi.ai/test/voice-testing.md“Voice Test Suites enable you to test your AI voice agents through simulated phone conversations.”
  • [claimed-docs] https://docs.vapi.ai/observability/evals-quickstart.md“Evals is Vapi's AI agent testing framework that enables you to systematically test assistants and squads using mock conversations with automated validation.”
  • [claimed-docs] https://docs.vapi.ai/test/voice-testing.md“Our AI tester calls your voice agent and follows a script that simulates real customer behavior.”
  • [claimed-docs] https://docs.vapi.ai/observability/evals-quickstart.md“you'll create mock conversations, define expected behaviors, and validate your agents work correctly before production”
  • [claimed-docs] https://docs.vapi.ai/cli.md“Forward webhooks to your local server for debugging”
  • [claimed-docs] https://docs.vapi.ai/cli.md“Build, test, and deploy voice AI applications without leaving your development environment.”

Rely on versioned APIs with a documented deprecation policyweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

API quality = 7.0 ÷ 70 × 100 = 10.0

Openness16.0/100×0.20 of the PA blend

Can you leave, inspect, or self-host — data export, open source, portability.

Do everything through the API that I can do in the UIweight 2

2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max

  • [claimed-docs] https://docs.vapi.ai/cli.md“Manage assistants, phone numbers, and calls from your terminal”
  • [claimed-docs] https://docs.vapi.ai/cli.md“vapi listen --forward-to localhost:3000/webhook”
  • [claimed-docs] https://docs.vapi.ai/sdk/mcp-server.md“so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
  • [claimed-docs] https://docs.vapi.ai/sdk/mcp-server.md“The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant”
  • [claimed-docs] https://docs.vapi.ai/cli.md“Build, test, and deploy voice AI applications without leaving your development environment.”
  • [claimed-docs] https://docs.vapi.ai/cli.md“The CLI auto-detects your tech stack and sets up everything you need.”
  • [probe] https://docs.vapi.ai/cli“PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bash`) and `vapi --version` printed `vapi version 0.2.1` keylessly, with man pages for assistant and call management.”
  • [probe] https://docs.vapi.ai/sdk/mcp-server“PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — the endpoint is live and bearer-key-gated exactly as documented (API key as Authorization: Bearer).”
  • [claimed-docs] https://docs.vapi.ai/customization/custom-voices/custom-voice.md“You can use your own custom voice with any supported provider by setting the `voice` property in your assistant configuration”
  • [claimed-docs] https://docs.vapi.ai/providers/voice/overview.md“Browse and preview voices there, then copy a voice's ID to use on an assistant.”

Export all of my data in open formats and leaveweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Read the product's source under an open licenseweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [community] https://hn.algolia.com/api/v1/items/45884165“Vapi is also built on media framework by daily.co. They have an open source version of voice ai called pipecat.”

Self-host the core productweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

  • [community] https://hn.algolia.com/api/v1/items/45884165“Vapi is also built on media framework by daily.co. They have an open source version of voice ai called pipecat.”
  • [probe] https://docs.vapi.ai/sdk/mcp-server“PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — the endpoint is live and bearer-key-gated exactly as documented (API key as Authorization: Bearer).”
  • [claimed-docs] https://docs.vapi.ai/sdk/mcp-server.md“The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant”

Openness = 16.0 ÷ 100 × 100 = 16.0

Built-in AI20.9/100×0.15 of the PA blend

Inside-out: how agentic the product itself is for its users — built-in assistants, autonomous features.

Get AI-generated insights and suggestions from my data inside the productweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Set up automations that run autonomously in the backgroundweight 2

2 (weight) × 4 (quality) × 0.6 (partial) = 4.8 of 20 max

  • [claimed-docs] https://docs.vapi.ai/tools.md“Create your own webhook-based tools to extend assistant capabilities”
  • [claimed-docs] https://docs.vapi.ai/server-url/events.md“When tools are triggered, your Server URL receives a `tool-calls` message”
  • [claimed-docs] https://docs.vapi.ai/server-url/events.md“Vapi supports OpenAI-style tool/function calling. Assistants can ping your server to perform actions.”
  • [claimed-docs] https://docs.vapi.ai/tools.md“Custom tools that you create... interact with your systems via webhooks”
  • [claimed-docs] https://docs.vapi.ai/tools/custom-tools.md“This guide shows you how to create custom tools, including Function Tools, for your Vapi assistants.”

Delegate tasks to a built-in AI assistant inside the productweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

  • [claimed-docs] https://docs.vapi.ai/cli.md“Your IDE's AI assistant (Cursor, Windsurf, VSCode) gains complete, accurate knowledge of Vapi's APIs and best practices. No more hallucinated code or outdated examples.”
  • [claimed-docs] https://docs.vapi.ai/cli.md“Manage assistants, phone numbers, and calls from your terminal”
  • [claimed-docs] https://docs.vapi.ai/sdk/mcp-server.md“so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”

Operate the product with natural-language commandsweight 2

2 (weight) × 7 (quality) × 1.0 (full) = 14.0 of 20 max

  • [claimed-docs] https://docs.vapi.ai/sdk/mcp-server.md“so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
  • [claimed-docs] https://docs.vapi.ai/sdk/mcp-server.md“The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant”
  • [claimed-docs] https://docs.vapi.ai/cli.md“Your IDE's AI assistant (Cursor, Windsurf, VSCode) gains complete, accurate knowledge of Vapi's APIs and best practices. No more hallucinated code or outdated examples.”
  • [probe] https://docs.vapi.ai/sdk/mcp-server“official MCP server documented at https://docs.vapi.ai/sdk/mcp-server”
  • [probe] https://docs.vapi.ai/sdk/mcp-server“PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — the endpoint is live and bearer-key-gated exactly as documented (API key as Authorization: Bearer).”
  • [probe] https://docs.vapi.ai/cli“PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bash`) and `vapi --version` printed `vapi version 0.2.1` keylessly, with man pages for assistant and call management.”
  • [probe] https://docs.vapi.ai/llms.txt“PROBE llms.txt: HTTP 200 at https://docs.vapi.ai/llms.txt # Vapi ## Instructions for AI Agents - For clean Markdown of any page, append `.md` to the page URL - For section-spec”

Built-in AI = 18.8 ÷ 90 × 100 = 20.9

Automation11.3/100×0.15 of the PA blend

Depth of automation primitives — rules, scheduling, bulk operations, webhooks.

Perform bulk operations across many items at onceweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Define rules that trigger actions automatically on eventsweight 3

3 (weight) × 5 (quality) × 0.6 (partial) = 9.0 of 30 max

  • [claimed-docs] https://docs.vapi.ai/server-url/events.md“When tools are triggered, your Server URL receives a `tool-calls` message”
  • [claimed-docs] https://docs.vapi.ai/server-url/events.md“Vapi supports OpenAI-style tool/function calling. Assistants can ping your server to perform actions.”
  • [claimed-docs] https://docs.vapi.ai/tools.md“Custom tools that you create... interact with your systems via webhooks”
  • [claimed-docs] https://docs.vapi.ai/tools.md“Create your own webhook-based tools to extend assistant capabilities”

Schedule recurring jobs or workflowsweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Version, review, and roll back my automationsweight 1

1 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 10 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Automation = 9.0 ÷ 80 × 100 = 11.3