Vapi vs ElevenLabs Agents

Agent ops

1. ai-native userMy coding agent can provision a complete voice agent end to end — create the agent, attach a number, and place a call — through the API, CLI, or MCP without touching the dashboard

   weight 3 · round to Vapi
   Vapi✓fullprobed8/10

   Vapi documents API/SDK-based assistant creation, phone number attachment, and call placement (vapi-docs-1, vapi-docs-18, vapi-docs-19), a CLI for managing assistants, phone numbers, and calls entirely from the terminal (vapi-docs-2, vapi-docs-31, vapi-probe-rt-1 confirming keyless install/version), and an official MCP server exposing these same operations to any MCP-compatible agent, verified live and auth-gated in a runtime probe (vapi-docs-33, vapi-probe-rt-2). Together these three surfaces (API, CLI, MCP) cover the full agent-provision-number-call workflow without dashboard use. missing for 10: no single end-to-end hands-on trace showing one agent chaining create→attach→call purely via CLI/MCP/API in one session, and no independent (non-vendor) confirmation of the full workflow succeeding.

   • [claimed-docs] “const assistant = await vapi.assistant”
   • [claimed-docs] “Manage assistants, phone numbers, and calls from your terminal”
   • [claimed-docs] “Create a voice assistant, connect it to a phone number, and make your first calls.”
   • [claimed-docs] “In under 5 minutes, you'll create a voice assistant and start talking to it over the phone.”
   • [claimed-docs] “Build, test, and deploy voice AI applications without leaving your development environment.”
   • [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
   • [probe] “official MCP server documented at https://docs.vapi.ai/sdk/mcp-server”
   • [probe] “official CLI documented at https://docs.vapi.ai/cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bas…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”

   proof ↗⚑ flag

   ElevenLabs Agents~partialprobed6/10

   Docs and runtime probes confirm agents can be created and managed via API, a real CLI (@elevenlabs/cli, verified live on npm), and a hosted MCP server (verified live with OAuth), all without the dashboard (docs-1,3,4,27,28,38; probe-rt-1,2). Phone-number attachment is also supported via SIP trunking to route calls to agents (docs-12,36), but there is no explicit documentation or evidence of actually placing/initiating an outbound call end-to-end through the API/CLI/MCP — only routing inbound infrastructure integration is described. Missing for 10: explicit API/CLI/MCP call to place an outbound call, and a concrete end-to-end walkthrough combining create+attach+call in one non-dashboard flow.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “Use the ElevenLabs agents skill to build and manage voice agents from your AI coding assistant: npx skills add elevenlabs/skills --skill age…”
   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
   • [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”
   • [claimed-docs] “Route calls to AI agents without changing your existing phone infrastructure”
   • [claimed-docs] “connect the hosted MCP server to Claude or any other MCP client to create and manage agents through natural language”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”

   proof ↗⚑ flag

2. ai-native userThe platform's own AI helps me author agents — generating or improving prompts, flows, and test cases from a description

   weight 1 · round to ElevenLabs Agents
   Vapi—none0/10

   Evidence shows Vapi provides CLI, MCP server, tool/webhook infrastructure, and testing frameworks (Evals, Voice Test Suites), plus a Customer Support template, but nothing indicates the platform itself uses AI to generate or improve prompts, conversation flows, or test cases from a natural-language description — testing tools require manually defined mock conversations/scripts rather than AI-authored ones.

   • [claimed-docs] “Voice Test Suites enable you to test your AI voice agents through simulated phone conversations.”
   • [claimed-docs] “Evals is Vapi's AI agent testing framework that enables you to systematically test assistants and squads using mock conversations with autom…”
   • [claimed-docs] “Our AI tester calls your voice agent and follows a script that simulates real customer behavior.”
   • [claimed-docs] “you'll create mock conversations, define expected behaviors, and validate your agents work correctly before production”
   • [claimed-docs] “Select the down arrow next to Create Assistant, then choose Customer Support.”

   proof ↗⚑ flag

   ElevenLabs Agents~partialprobed5/10

   ElevenLabs documents that an external AI assistant (e.g., Claude) can create, configure, and manage agents through natural language via the hosted MCP server, and agent-testing docs describe turning real conversations into test cases for regression testing — both are AI-assisted authoring flows. However, this relies on a third-party AI client (Claude) rather than a built-in, platform-native 'describe your agent and we generate it' copilot for prompts/flows within the dashboard itself. Missing for 10: evidence of a first-party generative wizard that authors prompts/flows purely from a natural-language description inside the ElevenLabs UI, and evidence of AI-generated (not just conversation-derived) test-case authoring.

   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
   • [claimed-docs] “connect the hosted MCP server to Claude or any other MCP client to create and manage agents through natural language”
   • [claimed-docs] “Transform real conversations into test cases when you find an interaction where the agent underperformed.”
   • [claimed-docs] “Agent testing lets you verify conversational responses, tool usage, and full multi-turn outcomes before you deploy.”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”

   proof ↗⚑ flag

Build

1. developerBuild a working phone voice agent — prompt, voice, and phone number — and take my first live call within an hour

   weight 3 · round to Vapi
   Vapi✓fullcommunity9/10

   Docs explicitly walk through creating a voice assistant, connecting a phone number, and making a first call in under 5 minutes (vapi-docs-18, vapi-docs-19), with voice selection via the Voice Library (vapi-docs-8, vapi-docs-24) and dashboard-driven assistant creation with templates (vapi-docs-16); community evidence corroborates a working live-call demo (vapi-comm-1). Missing for 10: independent third-party benchmarking of the full 'within an hour' timing claim beyond vendor docs and a single anecdotal community comment.

   • [claimed-docs] “Create a voice assistant, connect it to a phone number, and make your first calls.”
   • [claimed-docs] “In under 5 minutes, you'll create a voice assistant and start talking to it over the phone.”
   • [claimed-docs] “Select the down arrow next to Create Assistant, then choose Customer Support.”
   • [claimed-docs] “The Voice Library in the Vapi Dashboard lists every voice available to your organization. Browse and preview voices there”
   • [claimed-docs] “Browse and preview voices there, then copy a voice's ID to use on an assistant.”
   • [community] “Called the demo number, sounds smooth! Good luck.”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed7/10

   Docs show a clear quickstart path — create an agent via dashboard/API, pick from 5k+ voices, and connect phone numbers via SIP trunking to route calls to the agent without changing existing phone infrastructure — which together cover the prompt+voice+phone number pieces of the story. However, there's no explicit end-to-end 'first call in under an hour' walkthrough or independent timing evidence, and phone-number provisioning specifics (vs. just SIP integration) are thin. Missing for 10: an explicit timed/complete quickstart-to-live-call walkthrough, first-party or independent evidence of the actual time-to-first-call, and clearer native phone-number purchase/setup docs beyond SIP trunking.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “Select from 5k+ voices across 31 languages with customization options”
   • [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”
   • [claimed-docs] “Route calls to AI agents without changing your existing phone infrastructure”
   • [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”
   • [claimed-docs] “This assistant can be embedded into your website or app to provide real-time support to your customers”

   proof ↗⚑ flag

2. developerRun conversations in multiple languages, including detecting and switching language mid-call

   weight 2 · round to ElevenLabs Agents
   Vapi—none0/10

   The evidence pack covers CLI, MCP server, tools, voices, compliance, and testing, but contains no mention of multi-language support, language detection, or mid-call language switching capabilities. Absence of evidence for this applicable axis (multilingual voice agents are a fair capability to expect from a voice AI platform) means it must be judged 'none'. missing for 10: any mention of multi-language conversation support, language auto-detection, or mid-call language switching features.

   ⚑ flag

   ElevenLabs Agents~partialclaimed4/10

   Docs confirm broad multi-language support (70+ languages, language-specific voice settings) and voice-switching for multi-character use, but there is no explicit documentation of automatic language detection or switching languages mid-call. Missing for 10: explicit language-detection/mid-call switching feature docs, and any hands-on demonstration of a live language switch during a single conversation.

   • [claimed-docs] “Configure, deploy and monitor natural, human-sounding agents in 70+ languages with leading accuracy and ultra-low latency across voice or ch…”
   • [claimed-docs] “You can customize various aspects of your AI agent's voice to create a more natural and engaging conversation experience. This includes cont…”
   • [claimed-docs] “Enable your agent to switch between different voices for multi-character conversations, storytelling, and language tutoring.”
   • [claimed-docs] “Select from 5k+ voices across 31 languages with customization options”

   proof ↗⚑ flag

3. founderDesign multi-step conversation flows in a visual builder with branching, states, and handoffs without writing code

   weight 2 · round to ElevenLabs Agents
   Vapi—none0/10

   Evidence covers Vapi's assistant configuration, tools, CLI, MCP server, voices, and testing, but there is no mention of a visual flow builder with branching, states, or handoffs — Vapi appears configuration/code-driven (assistant config, function tools, webhooks) rather than a no-code visual conversation designer.

   ⚑ flag

   ElevenLabs Agents~partialclaimed6/10

   Docs explicitly claim a 'visual workflow builder' for multi-step workflows, plus dashboard-based agent creation, conversation-flow settings (turn-taking, interruptions, timeouts), and tools/handoff-like actions (webhooks, client tools) — supporting the no-code, branching, multi-step story. However, evidence never details branching logic, explicit 'states' or agent-to-agent 'handoffs' within the builder, nor independent hands-on confirmation of the visual builder's depth. Missing for 10: concrete detail/screenshots of branching and state logic in the builder, explicit handoff-between-agents feature description, and independent/third-party corroboration of the builder's no-code sufficiency.

   • [claimed-docs] “Build multi-step workflows with visual workflow builder”
   • [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”
   • [claimed-docs] “Configure turn-taking, interruptions, and timeout settings”
   • [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”
   • [claimed-docs] “Enable agents to call clients & APIs to perform actions”

   proof ↗⚑ flag

Personalization

1. developerInject dynamic variables and per-caller context at call time so each conversation is personalized

   weight 2 · round to ElevenLabs Agents
   Vapi—none0/10

   The evidence pack covers assistant creation, tools/webhooks, CLI, MCP server, voices, and testing, but contains no documentation of variable injection, assistantOverrides, or per-caller context personalization at call time. Missing for 10: docs on dynamic variable substitution (e.g., {{variableName}} templating), call-time overrides/metadata injection, and any example showing per-caller personalization.

   ⚑ flag

   ElevenLabs Agents✓fullclaimed8/10

   Docs explicitly state 'Use dynamic variables and overrides for per-conversation customization' as a core build feature, directly matching the story of injecting per-caller context at call time. Missing for 10: no independent/hands-on example showing a real dynamic-variable injection call in code, and no detail on variable scoping/limits beyond the single doc mention.

   • [claimed-docs] “Use dynamic variables and overrides for per-conversation customization”

   proof ↗⚑ flag

2. developerGround the agent on my documents with a built-in knowledge base or RAG so it answers from my content

   weight 2 · round to ElevenLabs Agents
   Vapi—none0/10

   Vapi is a voice-agent platform focused on assistants, phone numbers, custom webhook tools, and telephony/testing; the evidence pack shows no built-in knowledge base or RAG feature for grounding assistants on uploaded documents. Custom tools/webhooks could be used to build a workaround, but no document ingestion or retrieval capability is documented.

   • [claimed-docs] “Create your own webhook-based tools to extend assistant capabilities”
   • [claimed-docs] “This guide shows you how to create custom tools, including Function Tools, for your Vapi assistants.”
   • [claimed-docs] “Custom tools that you create... interact with your systems via webhooks”

   proof ↗⚑ flag

   ElevenLabs Agents✓fullclaimed8/10

   Docs explicitly state agents support uploading documents and enabling RAG for grounded responses, directly matching the story's built-in knowledge base/RAG capability. Missing for 10: independent/hands-on corroboration of RAG quality and details on knowledge base management (chunking, size limits, sources).

   • [claimed-docs] “Upload documents and enable RAG for grounded responses”

   proof ↗⚑ flag

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to ElevenLabs Agents
   Vapi✓fullprobed8/10

   Vapi confirms an llms.txt file exists and is served at docs.vapi.ai/llms.txt (HTTP 200), explicitly instructing agents on how to fetch clean Markdown per page, which directly satisfies the story. missing for 10: no independent/community corroboration of agents actually consuming llms.txt in practice, only first-party probe evidence.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.vapi.ai/llms.txt # Vapi ## Instructions for AI Agents - For clean Markdown of any page, append `.…”

   proof ↗⚑ flag

   ElevenLabs Agents✓fullprobed9/10

   ElevenLabs publishes a working llms.txt at the root (HTTP 200) and structures its docs as machine-readable .md pages with an explicit pointer to a full docs/llms.txt index, exactly matching the ai-native 'point an agent at llms.txt' pattern; this was independently verified via live probes rather than just claimed docs. Missing for 10: no evidence of a hands-on agent actually consuming the file end-to-end to complete a task (only that the endpoint/format is live and documented).

   • [probe] “PROBE llms.txt: HTTP 200 at https://elevenlabs.io/llms.txt # ElevenLabs ElevenLabs is an AI research and product company transforming how w…”
   • [probe] “PROBE docs-md: HTTP 200 at https://elevenlabs.io/docs/eleven-agents/overview.md > This is a page from the ElevenLabs documentation. For a co…”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round drawn
   Vapi~partialprobed6/10

   Vapi ships an official CLI (vapi-docs-2/31/32, confirmed working keylessly in vapi-probe-rt-1) that can manage assistants, calls, and forward webhooks, which supports scripted/CI-style usage, and SDKs/REST APIs imply headless programmatic calls. However there is no explicit CI/automation documentation, no examples of running in a pipeline, and the MCP server requires bearer-key auth (vapi-probe-rt-2) which is unaddressed for CI contexts. missing for 10: explicit CI/automation examples or docs, non-interactive auth/service-account flow for CI, evidence of headless voice-testing/evals running in a pipeline.

   • [claimed-docs] “Manage assistants, phone numbers, and calls from your terminal”
   • [claimed-docs] “Build, test, and deploy voice AI applications without leaving your development environment.”
   • [claimed-docs] “The CLI auto-detects your tech stack and sets up everything you need.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bas…”
   • [claimed-docs] “Evals is Vapi's AI agent testing framework that enables you to systematically test assistants and squads using mock conversations with autom…”
   • [claimed-docs] “you'll create mock conversations, define expected behaviors, and validate your agents work correctly before production”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”

   proof ↗⚑ flag

   ElevenLabs Agents~partialprobed6/10

   ElevenLabs Agents can be managed via a full REST API and an official CLI (verified runtime install/run), enabling scripted/headless creation and configuration of agents outside the dashboard, which supports CI-style automation. However, evidence is docs/probe-based with no CI pipeline examples, no first-class 'run in CI' guidance, and Agent testing is described as pre-deploy validation without confirmation it can be triggered non-interactively/headlessly. Missing for 10: explicit CI/CD integration examples, headless testing invocation docs, and independent hands-on confirmation of automated pipelines.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”
   • [claimed-docs] “Agent testing lets you verify conversational responses, tool usage, and full multi-turn outcomes before you deploy.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round to ElevenLabs Agents
   Vapi—none0/10

   The evidence shows Vapi exposes its own APIs as an MCP server (so external MCP clients like Claude Desktop can control Vapi), but this is the opposite direction from the story — there's no evidence that Vapi assistants can consume/plug-in external MCP servers to use their tools within a conversation. Vapi's tool docs describe webhook-based custom tools and OpenAI-style function calling only, with no mention of MCP-server ingestion.

   • [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
   • [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
   • [claimed-docs] “Create your own webhook-based tools to extend assistant capabilities”
   • [claimed-docs] “Custom tools that you create... interact with your systems via webhooks”
   • [probe] “official MCP server documented at https://docs.vapi.ai/sdk/mcp-server”

   proof ↗⚑ flag

   ElevenLabs Agents✓fullclaimed8/10

   Docs explicitly describe connecting conversational agents to external MCP servers so agents can access data sources and tools via MCP, directly matching the story, and this is corroborated by dedicated documentation pages (mcp.md) rather than just passing mention. Missing for 10: independent/hands-on verification of an actual third-party MCP server being plugged in and tool-calls succeeding in production (only docs and a runtime probe on the separate hosted-server feature are present).

   • [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers. This enables your agents to: Access and process informa…”
   • [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers.”
   • [claimed-docs] “Access and process information from various data sources via the MCP server”

   proof ↗⚑ flag

4. ai-native userConnect an agent via an official MCP server

   weight 3 · round drawn
   Vapi✓fullprobed9/10

   Vapi ships an official hosted MCP server (mcp.vapi.ai) exposing its assistant/phone/call APIs as MCP tools for any MCP-compatible agent (Claude Desktop, agent frameworks), documented in first-party docs and confirmed live via a runtime probe (401 bearer-key gate exactly as documented). Missing for 10: independent/community hands-on testimonials specifically about using the MCP server (community evidence only covers general demo/founder trivia, not MCP usage).

   • [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
   • [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
   • [probe] “official MCP server documented at https://docs.vapi.ai/sdk/mcp-server”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”

   proof ↗⚑ flag

   ElevenLabs Agents✓fullprobed9/10

   ElevenLabs explicitly documents and operates a hosted MCP server (api.elevenlabs.io/v1/mcp) that lets external MCP clients like Claude connect to create, configure, and manage agents through natural language, with a runtime probe confirming the live endpoint correctly responds with the MCP OAuth challenge. This is a first-party 'official MCP server' offering, not just client-side MCP consumption, matching the story precisely. Missing for 10: independent/community hands-on confirmation beyond ElevenLabs' own docs and the single runtime probe.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
   • [claimed-docs] “connect the hosted MCP server to Claude or any other MCP client to create and manage agents through natural language”
   • [claimed-docs] “create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally”
   • [probe] “official MCP server documented at https://elevenlabs.io/docs/eleven-agents/operate/hosted-mcp”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”

   proof ↗⚑ flag

5. ai-native userUse an official CLI

   weight 2 · round drawn
   Vapi✓fullprobed9/10

   Vapi ships a documented official CLI for managing assistants, phone numbers, and calls from the terminal, with webhook forwarding, org/environment switching, and tech-stack auto-detection, and this was independently verified in a runtime probe (successful install and `--version` output). missing for 10: no independent third-party review or community discussion of the CLI's day-to-day usage beyond the vendor docs and single install probe.

   • [claimed-docs] “Manage assistants, phone numbers, and calls from your terminal”
   • [claimed-docs] “vapi listen --forward-to localhost:3000/webhook”
   • [claimed-docs] “Switch between organizations and environments seamlessly”
   • [claimed-docs] “Forward webhooks to your local server for debugging”
   • [claimed-docs] “Build, test, and deploy voice AI applications without leaving your development environment.”
   • [claimed-docs] “The CLI auto-detects your tech stack and sets up everything you need.”
   • [probe] “official CLI documented at https://docs.vapi.ai/cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bas…”

   proof ↗⚑ flag

   ElevenLabs Agents✓fullprobed9/10

   ElevenLabs documents an official Agents CLI for managing agents, and runtime probing confirms `npx @elevenlabs/cli` installs and runs keylessly from npm, corroborating the docs claim with hands-on evidence. Missing for 10: deeper documentation of full CLI command surface/subcommands beyond basic install confirmation.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “Use the ElevenLabs agents skill to build and manage voice agents from your AI coding assistant: npx skills add elevenlabs/skills --skill age…”
   • [claimed-docs] “Use the ElevenLabs agents skill to build and manage voice agents from your AI coding assistant”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”

   proof ↗⚑ flag

6. ai-native userDrive the product through a documented public API

   weight 3 · round drawn
   Vapi✓fullprobed9/10

   Vapi ships a documented public API/SDK (assistant management, calls, tools), a CLI for terminal-driven workflows, and a hosted MCP server exposing the API as tools, all confirmed by runtime probes (CLI installs and runs, MCP endpoint live and auth-gated as documented). This directly satisfies programmatic/AI-native control via a documented public interface. Missing for 10: no independent third-party developer report deeply exercising the API beyond docs/probes.

   • [claimed-docs] “const assistant = await vapi.assistant”
   • [claimed-docs] “Manage assistants, phone numbers, and calls from your terminal”
   • [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
   • [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.vapi.ai/llms.txt # Vapi ## Instructions for AI Agents - For clean Markdown of any page, append `.…”
   • [probe] “official MCP server documented at https://docs.vapi.ai/sdk/mcp-server”
   • [probe] “official CLI documented at https://docs.vapi.ai/cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bas…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”

   proof ↗⚑ flag

   ElevenLabs Agents✓fullprobed9/10

   ElevenLabs Agents ships a documented public API alongside a dashboard, CLI, and hosted MCP server for programmatic agent creation/management, with quickstart docs explicitly noting 'creating an agent via the API' and full customization (tools, voice, analysis, testing) accessible programmatically. Runtime probes confirm the CLI installs keylessly and the hosted MCP endpoint is live and functional. Missing for 10: no independent third-party API reference/SDK usage examples or community corroboration beyond first-party docs.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”
   • [claimed-docs] “Enable agents to call clients & APIs to perform actions”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”

   proof ↗⚑ flag

7. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round to Vapi
   Vapi~partialprobed3/10

   Vapi's enterprise docs mention Role Based Access Control (RBAC) and SSO, implying some access-scoping capability at the org level, and its MCP server/API require bearer API keys — but there is no documented mechanism for issuing per-agent, least-privilege scoped API keys or tokens tailored to a specific agent's permissions. missing for 10: explicit scoped API key creation/management UI or API, documentation of key-level permission granularity, and independent confirmation that RBAC restricts agent credentials rather than just human dashboard users.

   • [claimed-docs] “Single Sign On (SSO) supported for Okta, Azure AD, SAML, and OIDC * Role Based Access Control (RBAC)”
   • [claimed-docs] “Single Sign On (SSO) supported for Okta, Azure AD, SAML, and OIDC”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”

   proof ↗⚑ flag

   ElevenLabs Agents—none0/10

   Evidence covers agent-side authentication (protecting inbound access to a deployed agent) and OAuth for the hosted MCP server, but there is no documentation of issuing scoped or least-privilege API keys/tokens that limit what an agent itself can do via the ElevenLabs API. Missing for 10: any docs on API key permission scopes, workspace role-based tokens, or restricted-credential issuance for agents.

   • [claimed-docs] “Secure agent access | Authentication | Implement custom authentication for protected agent access”
   • [claimed-docs] “Implement custom authentication for protected agent access”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”

   proof ↗⚑ flag

8. ai-native userBuild against official SDKs

   weight 2 · round to Vapi
   Vapi✓fullprobed8/10

   Vapi documents an official JS/TS SDK usage pattern, official CLI (probe-confirmed installed and runnable), and official MCP server (probe-confirmed live and auth-gated), all clearly aimed at AI-native/agentic developer workflows including IDE assistant integration (Cursor/Windsurf/VSCode). missing for 10: independent third-party corroboration of SDK code quality and coverage across multiple languages beyond the docs snippets.

   • [claimed-docs] “const assistant = await vapi.assistant”
   • [claimed-docs] “Manage assistants, phone numbers, and calls from your terminal”
   • [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
   • [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
   • [claimed-docs] “Your IDE's AI assistant (Cursor, Windsurf, VSCode) gains complete, accurate knowledge of Vapi's APIs and best practices. No more hallucinate…”
   • [probe] “official MCP server documented at https://docs.vapi.ai/sdk/mcp-server”
   • [probe] “official CLI documented at https://docs.vapi.ai/cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bas…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”

   proof ↗⚑ flag

   ElevenLabs Agents~partialprobed4/10

   Docs confirm programmatic access via the ElevenLabs API and an official CLI (`@elevenlabs/cli`, verified runtime) for building agents, but no evidence pack item explicitly documents a first-party language SDK (Python/Node/etc.) for Agents beyond the generic API reference. Missing for 10: explicit official SDK docs/repos, SDK code examples, and independent confirmation of SDK feature parity with the API.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”

   proof ↗⚑ flag

9. ai-native userSubscribe to events via webhooks

   weight 2 · round to Vapi
   Vapi✓fullclaimed7/10

   Vapi supports webhook-based events via Server URL (tool-calls messages), custom webhook tools, and CLI webhook forwarding for local debugging, giving AI-native users a documented event subscription mechanism. missing for 10: independent/hands-on confirmation of webhook delivery reliability, a full event-type catalog/schema, and signature/verification documentation.

   • [claimed-docs] “Create your own webhook-based tools to extend assistant capabilities”
   • [claimed-docs] “When tools are triggered, your Server URL receives a `tool-calls` message”
   • [claimed-docs] “Vapi supports OpenAI-style tool/function calling. Assistants can ping your server to perform actions.”
   • [claimed-docs] “Forward webhooks to your local server for debugging”
   • [claimed-docs] “Custom tools that you create... interact with your systems via webhooks”

   proof ↗⚑ flag

   ElevenLabs Agents—none0/10

   The evidence describes 'webhook tools' that let an agent make outbound calls to external endpoints during a conversation (docs-14, docs-29, docs-31), which is the opposite of subscribing to platform-emitted events via webhooks. There is no documented mechanism for a developer to register a webhook URL to receive event notifications (e.g., conversation-ended, call-completed) from ElevenLabs Agents.

   • [claimed-docs] “Tools give your assistant the ability to make external function calls to third party apps so you can get real-time information.”
   • [claimed-docs] “Tools allow ElevenLabs agents to perform actions beyond generating text responses.”
   • [claimed-docs] “allow an assistant to trigger authenticated actions based on the conversation, like scheduling meetings or initiating order returns”

   proof ↗⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round to ElevenLabs Agents
   Vapi—none0/10

   Evidence covers building/testing assistants (Evals, Voice Test Suites), CLI/MCP management tools, and tool-calling, but nothing describes the product itself surfacing AI-generated insights or suggestions derived from a user's own call/usage data (e.g., analytics dashboards with AI-generated recommendations).

   ⚑ flag

   ElevenLabs Agents~partialclaimed5/10

   Agent-analysis features (sentiment analysis, success evaluation, structured data extraction, semantic conversation search) generate AI-derived insights from the user's own conversation data, which partially satisfies the story. However, these are evaluation/reporting metrics rather than proactive AI-generated suggestions or recommendations for next actions, and there is no independent/hands-on corroboration of this analytics experience. Missing for 10: evidence of proactive suggestion generation (e.g., recommended actions/optimizations), a dashboard summarization view, and independent verification beyond first-party docs.

   • [claimed-docs] “Success evaluation: Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Data collection: Extract specific data points from conversations such as contact information, issue details, or any structured information”
   • [claimed-docs] “Sentiment analysis: Understand user sentiment across completed conversations”
   • [claimed-docs] “Searching conversations: Find messages by keyword or meaning across your conversation history.”
   • [claimed-docs] “Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Extract structured information from conversations such as contact details and business data”

   proof ↗⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round drawn
   Vapi~partialclaimed4/10

   Vapi assistants operate autonomously once a call starts—handling conversation flow, invoking custom tools via webhooks, and triggering server-side automations without further human input (vapi-docs-6, vapi-docs-17, vapi-docs-23, vapi-docs-34). However, this autonomy is scoped to an active call session; there is no evidence of scheduled/cron-style background jobs or agent loops that run independently of a triggered call or user interaction. Missing for 10: evidence of scheduled/background triggers outside live calls, persistent autonomous task queues, or proactive (non-call-triggered) automation runs.

   • [claimed-docs] “Create your own webhook-based tools to extend assistant capabilities”
   • [claimed-docs] “When tools are triggered, your Server URL receives a `tool-calls` message”
   • [claimed-docs] “Vapi supports OpenAI-style tool/function calling. Assistants can ping your server to perform actions.”
   • [claimed-docs] “Custom tools that you create... interact with your systems via webhooks”
   • [claimed-docs] “This guide shows you how to create custom tools, including Function Tools, for your Vapi assistants.”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed4/10

   ElevenLabs Agents supports tool-calling, webhook actions, and multi-step workflows that let a voice agent perform actions (scheduling, API calls, MCP access) during a conversation, and agents can be created/managed via API for programmatic triggering, which give some automation flavor. But all documented action-taking is tied to a live conversational turn (voice/chat) rather than an autonomous background process that runs without a triggering conversation or human interaction, so the 'runs autonomously in the background' framing is only weakly supported. Missing for 10: evidence of scheduled/cron-triggered autonomous runs, evidence of multi-step tasks executing without a live conversation session, and independent confirmation of background automation outside the demoed conversational/tool-call flow.

   • [claimed-docs] “Build multi-step workflows with visual workflow builder”
   • [claimed-docs] “Enable agents to call clients & APIs to perform actions”
   • [claimed-docs] “Tools give your assistant the ability to make external function calls to third party apps so you can get real-time information.”
   • [claimed-docs] “allow an assistant to trigger authenticated actions based on the conversation, like scheduling meetings or initiating order returns”
   • [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers. This enables your agents to: Access and process informa…”
   • [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”

   proof ↗⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round to ElevenLabs Agents
   Vapi—none0/10

   Vapi's docs describe building and configuring voice AI assistants for end-users (phone/web calls, tools, testing) and even an MCP server/CLI for managing those assistants, but there is no evidence of a built-in AI assistant/copilot inside the Vapi product itself that the AI-native user can delegate platform tasks to (e.g., an in-dashboard copilot that configures assistants or writes tools for you). The CLI mentions external IDE assistants (Cursor/Windsurf) gaining API knowledge, but that's a third-party tool, not a built-in in-product assistant.

   • [claimed-docs] “Your IDE's AI assistant (Cursor, Windsurf, VSCode) gains complete, accurate knowledge of Vapi's APIs and best practices. No more hallucinate…”
   • [claimed-docs] “Manage assistants, phone numbers, and calls from your terminal”
   • [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”

   proof ↗⚑ flag

   ElevenLabs Agents~partialprobed5/10

   ElevenLabs documents a first-party hosted MCP server letting an assistant like Claude create, configure, and manage agents in the workspace via natural language, with nothing to install locally — a form of task delegation to an AI assistant for managing the product. However this is not a native in-product chat/assistant UI; it requires an external MCP client (e.g., Claude) to act as the assistant, and delegation is scoped to agent-management tasks rather than general product tasks. Missing for 10: a built-in assistant surface inside the ElevenLabs dashboard itself, evidence of broader task delegation beyond agent CRUD, and independent hands-on confirmation of the natural-language workflow.

   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language”
   • [claimed-docs] “connect the hosted MCP server to Claude or any other MCP client to create and manage agents through natural language”
   • [claimed-docs] “create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”

   proof ↗⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round to ElevenLabs Agents
   Vapi✓fullprobed7/10

   Vapi exposes an official MCP server that lets any MCP-compatible AI assistant (e.g. Claude Desktop) manage assistants, phone numbers, and calls via natural-language tool calls, and this endpoint is confirmed live and auth-gated in a runtime probe. It also ships a CLI and llms.txt docs optimized for AI agents to operate it. Missing for 10: no first-party evidence of a built-in chat/NL command console inside the Vapi product itself (beyond MCP/CLI proxies), and no independent hands-on report of an agent successfully performing multi-step tasks via MCP.

   • [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
   • [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
   • [claimed-docs] “Your IDE's AI assistant (Cursor, Windsurf, VSCode) gains complete, accurate knowledge of Vapi's APIs and best practices. No more hallucinate…”
   • [probe] “official MCP server documented at https://docs.vapi.ai/sdk/mcp-server”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bas…”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.vapi.ai/llms.txt # Vapi ## Instructions for AI Agents - For clean Markdown of any page, append `.…”

   proof ↗⚑ flag

   ElevenLabs Agents✓fullprobed8/10

   ElevenLabs Agents ships a hosted MCP server explicitly designed so assistants like Claude can 'create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally,' and a runtime probe confirms the endpoint is live and speaks the MCP OAuth flow. This is a direct, first-party natural-language control surface, not just a client-side integration. Missing for 10: independent/community hands-on confirmation of a full natural-language agent-management session (only endpoint liveness was probed, not an actual successful NL task).

   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language”
   • [claimed-docs] “connect the hosted MCP server to Claude or any other MCP client to create and manage agents through natural language”
   • [claimed-docs] “create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”
   • [probe] “official MCP server documented at https://elevenlabs.io/docs/eleven-agents/operate/hosted-mcp”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   Vapi—none0/10

   The evidence pack shows static markdown documentation, code snippets, a CLI, and an MCP server, but no interactive API reference or runnable/try-it-out examples are described anywhere in the docs pages cited.

   ⚑ flag

   ElevenLabs Agents—none0/10

   The evidence pack contains extensive markdown documentation for ElevenLabs Agents (quickstart, customization, tools, etc.) but nothing describes an interactive API reference page with runnable/'try it' examples — no mention of a Swagger/OpenAPI explorer, live code sandbox, or embedded runnable snippets. Since ElevenLabs Agents exposes an API, this axis is applicable, but no evidence supports the specific capability.

   ⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   Vapi—none0/10

   The evidence pack covers Vapi's CLI, MCP server, docs, and SDKs, but no citation mentions an OpenAPI spec, API reference schema, or any machine-readable spec file available for download. This is a fair axis for an API-first product, but no supporting evidence exists in the pack.

   ⚑ flag

   ElevenLabs Agents—none0/10

   The evidence pack documents the API, CLI, dashboard, and hosted MCP server for ElevenLabs Agents, but nowhere mentions a downloadable OpenAPI/Swagger spec or machine-readable schema for the API. This axis clearly applies to an API-driven product, but no evidence shows it being delivered.

   ⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round to Vapi
   Vapi✓fullclaimed7/10

   Vapi offers dedicated Voice Test Suites (AI tester simulating conversations) and an Evals framework explicitly for creating mock conversations to 'validate your agents work correctly before production,' plus CLI webhook forwarding to localhost for local development/debugging — all separate from live production calls. Missing for 10: explicit vendor use of the term 'sandbox environment' and independent/hands-on verification that test runs are fully isolated from production data.

   • [claimed-docs] “Voice Test Suites enable you to test your AI voice agents through simulated phone conversations.”
   • [claimed-docs] “Evals is Vapi's AI agent testing framework that enables you to systematically test assistants and squads using mock conversations with autom…”
   • [claimed-docs] “Our AI tester calls your voice agent and follows a script that simulates real customer behavior.”
   • [claimed-docs] “you'll create mock conversations, define expected behaviors, and validate your agents work correctly before production”
   • [claimed-docs] “Forward webhooks to your local server for debugging”
   • [claimed-docs] “Build, test, and deploy voice AI applications without leaving your development environment.”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed4/10

   Docs describe an 'agent testing' feature to verify conversational responses, tool usage, and multi-turn outcomes 'before you deploy', implying a pre-production test workflow, and tool execution can run in a 'sandboxed environment' on ElevenLabs' infrastructure. However there is no explicit documentation of a separate sandbox/staging environment or test data isolation from production conversation/analytics data. Missing for 10: explicit sandbox/staging environment separate from production data, documentation of test-vs-prod data isolation, independent confirmation that test runs don't affect production analytics or logs.

   • [claimed-docs] “Agent testing lets you verify conversational responses, tool usage, and full multi-turn outcomes before you deploy.”
   • [claimed-docs] “Agent testing lets you verify conversational responses, tool usage, and full multi-turn outcomes before you deploy”
   • [claimed-docs] “Custom JavaScript executed in a sandboxed environment on ElevenLabs' infrastructure.”
   • [claimed-docs] “Custom JavaScript executed in a sandboxed environment on ElevenLabs' infrastructure”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Vapi—none0/10

   No evidence pack item mentions API versioning scheme, version numbers in endpoints, or any documented deprecation/sunset policy for Vapi's APIs; the docs cover features (CLI, MCP, tools, voices) but not API lifecycle governance.

   ⚑ flag

   ElevenLabs Agents—none0/10

   No evidence in the pack mentions API versioning scheme, version numbers, or any documented deprecation policy for ElevenLabs Agents' API/CLI/MCP interfaces. missing for 10: versioned API endpoints or version headers, changelog/release notes, explicit deprecation policy or sunset timelines, migration guides for breaking changes.

   ⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round drawn
   Vapi—none0/10

   The evidence pack documents CLI/SDK/MCP management of individual assistants, phone numbers, and calls, but nowhere shows bulk/batch endpoints or commands (e.g., batch-create, bulk-update, mass-delete) for operating across many items at once.

   ⚑ flag

   ElevenLabs Agents—none0/10

   The evidence documents API, CLI, and MCP-based management of agents, but nothing describes bulk/batch operations (e.g., batch-create, batch-update, or batch-delete across many agents or conversations at once). This axis is fair to ask given the product exposes an API and CLI, but no evidence confirms bulk-operation support.

   ⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to ElevenLabs Agents
   Vapi~partialclaimed5/10

   Vapi supports event-driven server-side tools/webhooks that fire on call events (tool-calls messages, server-url events) which act as a rule-trigger mechanism, but this is scoped to voice-call events only rather than a general-purpose automation/rules engine for arbitrary triggers. missing for 10: a documented general condition/trigger-action rules engine spanning non-call events, cross-system automation, or independent hands-on evidence of complex conditional automation chains.

   • [claimed-docs] “When tools are triggered, your Server URL receives a `tool-calls` message”
   • [claimed-docs] “Vapi supports OpenAI-style tool/function calling. Assistants can ping your server to perform actions.”
   • [claimed-docs] “Custom tools that you create... interact with your systems via webhooks”
   • [claimed-docs] “Create your own webhook-based tools to extend assistant capabilities”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed6/10

   ElevenLabs Agents supports tool-calling and webhook actions that are triggered automatically based on conversation events (e.g., 'trigger authenticated actions based on the conversation, like scheduling meetings or initiating order returns'), plus multi-step visual workflows and MCP-based external actions. This covers event-triggered automation within a conversation but the evidence doesn't show general rule-based triggers on arbitrary system events (e.g., scheduled/external triggers outside a live conversation) — missing for 10: documentation of standalone event/rule engine independent of a live conversation, examples of conditional branching logic beyond tool calls, and independent verification of automation reliability.

   • [claimed-docs] “Enable agents to call clients & APIs to perform actions”
   • [claimed-docs] “Tools give your assistant the ability to make external function calls to third party apps so you can get real-time information.”
   • [claimed-docs] “allow an assistant to trigger authenticated actions based on the conversation, like scheduling meetings or initiating order returns”
   • [claimed-docs] “Build multi-step workflows with visual workflow builder”
   • [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers. This enables your agents to: Access and process informa…”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round drawn
   Vapi—none0/10

   No evidence in the pack shows Vapi supports scheduling recurring jobs, workflows, or automated recurring calls/tasks — the docs cover assistants, tools, CLI, MCP server, testing, and voice customization but nothing about cron-like scheduling or recurring automation triggers.

   ⚑ flag

   ElevenLabs Agents—none0/10

   The evidence pack covers multi-step workflow building, tool calling, MCP integration, and analysis, but no documentation mentions scheduling recurring jobs, cron-like triggers, or automated recurring workflow execution. This is a fair capability to expect from an automation-focused agent platform, but no evidence supports it.

   ⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round drawn
   Vapi—none0/10

   Vapi's docs cover assistant creation, tools, CLI, MCP server, and testing/evals, but there is no mention of versioning assistant configurations, review/approval workflows, or rollback to prior versions of automations. No evidence pack item addresses version history or rollback capability.

   ⚑ flag

   ElevenLabs Agents—none0/10

   The docs describe building, testing, and analyzing agents (agent-testing, agent-analysis) but there is no mention of version history, change review workflows, or rollback/revert capability for agent configurations. Missing for 10: version history UI, rollback/restore of prior agent configs, change-review/approval workflow, audit trail of edits.

   ⚑ flag

Compliance

1. founderMeet call-recording consent and disclosure obligations with per-call recording controls and configurable data retention

   weight 2 · round to ElevenLabs Agents
   Vapi—none0/10

   The evidence pack covers HIPAA toggle, SSO/RBAC, testing/evals, and CLI/MCP tooling, but contains no mention of call-recording enable/disable controls, consent disclosure features, or configurable data retention policies for call recordings — a fair and applicable axis for a voice-calling platform.

   ⚑ flag

   ElevenLabs Agents~partialclaimed4/10

   Docs confirm configurable data-retention settings for conversation transcripts and audio recordings, which addresses the retention half of the story, plus HIPAA/BAA support suggesting compliance posture. However, there is no evidence of per-call recording enable/disable controls or built-in consent/disclosure messaging mechanisms for calls. Missing for 10: per-call recording toggle, consent/disclosure prompt configuration, and any independent verification of retention enforcement.

   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings.”
   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings”
   • [claimed-docs] “ElevenLabs allows such customers to develop AI-powered voice agents for the handling Protected Health Information (PHI)”
   • [claimed-docs] “ElevenLabs Agents is one of ElevenLabs' HIPAA-eligible services, and we offer Business Associate Agreements (BAAs) to eligible customers”

   proof ↗⚑ flag

2. platform-engineerRun regulated workloads with HIPAA/BAA support, SOC 2, and data-residency options

   weight 2 · round drawn
   Vapi~partialclaimed5/10

   Vapi documents HIPAA compliance toggle in the dashboard and enterprise SSO/RBAC, indicating some regulated-workload support, but evidence lacks explicit SOC 2 certification details, BAA execution process, or data-residency region options. missing for 10: SOC 2 report/certification evidence, explicit BAA process details, data-residency/region selection options, independent audit confirmation.

   • [claimed-docs] “HIPAA compliance can be ensured by turning on HIPAA compliance in the Dashboard.”
   • [claimed-docs] “Single Sign On (SSO) supported for Okta, Azure AD, SAML, and OIDC * Role Based Access Control (RBAC)”
   • [claimed-docs] “Single Sign On (SSO) supported for Okta, Azure AD, SAML, and OIDC”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed5/10

   ElevenLabs Agents documents HIPAA eligibility and BAA availability for eligible customers, directly supporting regulated PHI workloads. However, there is no evidence in the pack of SOC 2 certification or data-residency options, both explicitly named in the story. Missing for 10: SOC 2 attestation/report evidence, data-residency region selection or documentation, independent audit corroboration of HIPAA/BAA claims.

   • [claimed-docs] “ElevenLabs allows such customers to develop AI-powered voice agents for the handling Protected Health Information (PHI)”
   • [claimed-docs] “ElevenLabs allows such customers to develop AI-powered voice agents for the handling Protected Health Information (PHI).”
   • [claimed-docs] “ElevenLabs Agents is one of ElevenLabs' HIPAA-eligible services, and we offer Business Associate Agreements (BAAs) to eligible customers”

   proof ↗⚑ flag

Scale

1. platform-engineerSee documented concurrency limits and scale to many simultaneous calls without manual capacity begging

   weight 2 · round drawn
   Vapi—none0/10

   No evidence pack items document concurrency limits, per-org call caps, autoscaling behavior, or capacity-request processes; the docs cover CLI, MCP server, tools, voices, and enterprise SSO but nothing about simultaneous-call scaling limits or how to raise them.

   ⚑ flag

   ElevenLabs Agents—none0/10

   No evidence pack item documents concurrency limits, per-account call caps, autoscaling behavior, or a process for scaling to many simultaneous calls without contacting sales/support; docs cover features like tools, voice customization, SIP trunking, HIPAA, and MCP but nothing on capacity/scale numbers or self-service scaling guarantees.

   ⚑ flag

Self host

1. platform-engineerSelf-host the voice agent runtime from open-source code on my own infrastructure

   weight 3 · round drawn
   Vapi—none0/10

   Vapi is presented entirely as a hosted SaaS platform with a CLI, MCP server, and dashboard for managing cloud-hosted assistants; there is no evidence of open-source runtime code or any self-hosting option. One community note even references that a different product (pipecat by daily.co) is the open-source alternative, implicitly contrasting with Vapi's closed hosted model.

   • [community] “Vapi is also built on media framework by daily.co. They have an open source version of voice ai called pipecat.”
   • [claimed-docs] “Build, test, and deploy voice AI applications without leaving your development environment.”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”

   proof ↗⚑ flag

   ElevenLabs Agents—none0/10

   ElevenLabs Agents is entirely a managed/hosted service — the CLI and MCP server are clients/interfaces to ElevenLabs' cloud infrastructure, not open-source runtime code that can be deployed on a platform-engineer's own servers. Nothing in the evidence pack mentions open-sourcing the runtime, self-hosting instructions, or on-prem deployment options; all references (dashboard, API, CLI, hosted MCP, SIP trunking) point to ElevenLabs-operated infrastructure.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
   • [claimed-docs] “create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”

   proof ↗⚑ flag

Latency

1. platform-engineerSee documented end-to-end voice latency numbers or tuning guidance backing the platform's speed claims

   weight 3 · round to ElevenLabs Agents
   Vapi~partialclaimed3/10

   Vapi's docs mention that 'Ultra Fast' vs 'High Intelligence' model settings trade off latency, giving conceptual tuning guidance, but there are no documented end-to-end latency numbers (e.g., ms benchmarks) or detailed tuning playbooks in the evidence pack. Missing for 10: concrete latency benchmark figures, component-level latency breakdowns (STT/LLM/TTS), and independent verification of speed claims.

   • [claimed-docs] “Ultra Fast uses speed-optimized models for lower latency. High Intelligence accepts higher latency for greater capability.”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed4/10

   ElevenLabs markets 'ultra-low latency' (docs-37) and does document conversation-flow tuning knobs like turn-taking, interruption handling and timeouts (docs-33, docs-42, docs-19/49/54), which counts as tuning guidance for latency/turn-taking behavior. However there are no documented end-to-end latency benchmark numbers (e.g., ms figures, percentile breakdowns) anywhere in the evidence pack. Missing for 10: concrete latency benchmark numbers/metrics, methodology for measuring end-to-end latency, and any independent corroboration of the 'ultra-low latency' marketing claim.

   • [claimed-docs] “Configure, deploy and monitor natural, human-sounding agents in 70+ languages with leading accuracy and ultra-low latency across voice or ch…”
   • [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”
   • [claimed-docs] “Configure turn-taking, interruptions, and timeout settings”
   • [claimed-docs] “The Max conversation duration setting limits the total time a conversation can remain active. ... The default is 600 seconds (10 minutes).”

   proof ↗⚑ flag

Turn taking

1. developerRely on the agent to handle interruptions (barge-in) gracefully — stopping speech, updating context, and recovering the turn

   weight 3 · round to ElevenLabs Agents
   Vapi—none0/10

   The evidence pack covers CLI, MCP server, tools, voices, compliance, and testing, but contains no documentation or evidence about interruption handling, barge-in behavior, context updates during interruptions, or turn recovery mechanics. This is a core latency/turn-taking capability for voice agents, but nothing in the pack addresses it directly.

   ⚑ flag

   ElevenLabs Agents~partialclaimed6/10

   Docs explicitly document configuring turn-taking, interruptions, and timeout settings, and conversation-flow settings that 'determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavior,' confirming barge-in handling is a first-class configurable feature. However, evidence lacks detail on how context is updated after an interruption or how turn recovery specifically behaves, and there's no hands-on/independent validation of graceful recovery in practice. Missing for 10: technical detail on context truncation/update after interruption, independent or hands-on testing confirming graceful recovery, and latency benchmarks for barge-in specifically.

   • [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”
   • [claimed-docs] “Configure turn-taking, interruptions, and timeout settings”
   • [claimed-docs] “The Max conversation duration setting limits the total time a conversation can remain active. ... The default is 600 seconds (10 minutes).”

   proof ↗⚑ flag

2. developerEnable noise suppression or audio filtering so the agent stays coherent on noisy real-world calls

   weight 1 · round drawn
   Vapi—none0/10

   No evidence pack items mention noise suppression, background noise filtering, or audio denoising features for handling noisy real-world calls; the pack covers voices, tools, CLI, MCP, latency modes, and testing but nothing about audio filtering.

   ⚑ flag

   ElevenLabs Agents—none0/10

   The evidence pack covers voice customization, turn-taking/interruption settings, tools, testing, and telephony (SIP) integration, but nowhere mentions noise suppression, background noise handling, or audio filtering for noisy real-world calls. This is a fair capability to expect for a voice-agent platform handling phone calls, but no documentation or feature reference confirms it exists.

   • [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”
   • [claimed-docs] “Configure turn-taking, interruptions, and timeout settings”
   • [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”

   proof ↗⚑ flag

3. developerUse model-based end-of-turn detection beyond simple VAD silence timeouts so the agent doesn't talk over slow speakers

   weight 2 · round drawn
   Vapi—none0/10

   The evidence pack lacks any mention of model-based end-of-turn detection, turn-taking models, or semantic VAD alternatives; only latency tuning between 'Ultra Fast' and 'High Intelligence' model modes is documented, which does not address end-of-turn detection beyond silence-based VAD. Missing for 10: any documentation of a model-based or semantic end-of-turn detector, comparison to VAD-only timeout behavior, or configuration options for turn-taking sensitivity.

   • [claimed-docs] “Ultra Fast uses speed-optimized models for lower latency. High Intelligence accepts higher latency for greater capability.”

   proof ↗⚑ flag

   ElevenLabs Agents—none0/10

   Docs mention configurable turn-taking, interruption handling, and timeout settings (docs-33, docs-42), but nothing in the evidence describes a model-based end-of-turn detector that goes beyond a VAD/silence-timeout mechanism — no mention of semantic or ML-based turn-completion detection to avoid interrupting slow speakers. Missing for 10: any documentation of a dedicated end-of-turn prediction model, benchmarks or comparisons versus simple VAD timeouts, or hands-on evidence that the agent waits appropriately for slow speakers.

   • [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”
   • [claimed-docs] “Configure turn-taking, interruptions, and timeout settings”
   • [claimed-docs] “The Max conversation duration setting limits the total time a conversation can remain active. ... The default is 600 seconds (10 minutes).”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Vapi
   Vapi✓fullprobed8/10

   Vapi's API/SDK covers assistant, phone number, and call management, backed by a full CLI (vapi-docs-2 through 4, 27, 31-32, verified live in vapi-probe-rt-1) and an official MCP server exposing the same operations for external agents (vapi-docs-5,33, confirmed live and auth-gated in vapi-probe-rt-2). Dashboard-only conveniences like Voice Library browsing/preview remain UI-centric, though voice IDs and custom voices can be set via API (vapi-docs-9,24,26,35). Missing for 10: explicit confirmation that every dashboard-only feature (e.g., visual Voice Library browsing, HIPAA toggle) has a full API equivalent, and independent (non-vendor) verification of full UI/API parity.

   • [claimed-docs] “Manage assistants, phone numbers, and calls from your terminal”
   • [claimed-docs] “vapi listen --forward-to localhost:3000/webhook”
   • [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
   • [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
   • [claimed-docs] “Build, test, and deploy voice AI applications without leaving your development environment.”
   • [claimed-docs] “The CLI auto-detects your tech stack and sets up everything you need.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bas…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”
   • [claimed-docs] “You can use your own custom voice with any supported provider by setting the `voice` property in your assistant configuration”
   • [claimed-docs] “Browse and preview voices there, then copy a voice's ID to use on an assistant.”

   proof ↗⚑ flag

   ElevenLabs Agents~partialprobed7/10

   Docs show agents can be created, configured, and managed via the API, CLI, dashboard, or hosted MCP server, and quickstart explicitly says agents can be created 'via the API or the web dashboard,' implying broad parity across surfaces (elevenlabs-agents-docs-1, docs-39, docs-44). However, there's no explicit feature-by-feature confirmation that every dashboard capability (e.g., visual workflow builder, analytics dashboards, testing UI) has a full API equivalent, nor independent verification of parity. Missing for 10: an explicit parity statement or changelog confirming all dashboard features (workflow builder, analytics, testing suite) are fully scriptable via API, and independent/hands-on confirmation of that parity.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”
   • [claimed-docs] “create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally”
   • [claimed-docs] “Build multi-step workflows with visual workflow builder”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round drawn
   Vapi—none0/10

   No evidence of a data export feature, open-format data dumps, or account portability/deletion tooling; the docs cover CLI, MCP server, tools, and voice features but nothing about exporting assistants/call data/transcripts in open formats for user exit. Missing for 10: documented export functionality, open data formats supported, and any account/data portability guarantees.

   ⚑ flag

   ElevenLabs Agents—none0/10

   No evidence describes a data export feature, open-format data dumps, or account deletion/portability tooling; only conversation retention settings are documented, not export mechanisms. Missing for 10: any documented export API/CLI command, supported open export formats (e.g., JSON/CSV), and confirmation of full data portability/account closure process.

   ⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round drawn
   Vapi—none0/10

   Vapi is a closed, commercial SaaS platform; nothing in the evidence indicates its core source (server, dashboard, or model runtime) is published under an open license—only SDKs/CLI tooling and docs are mentioned, and the one open-source reference (pipecat) is a third-party project, not Vapi itself.

   • [community] “Vapi is also built on media framework by daily.co. They have an open source version of voice ai called pipecat.”

   proof ↗⚑ flag

   ElevenLabs Agents—none0/10

   ElevenLabs Agents is a closed, proprietary SaaS product; the CLI is open on npm but no evidence indicates the core Agents platform/backend source is available under an open license. No license file, GitHub repo, or open-source statement is present in the evidence pack.

   ⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round drawn
   Vapi—none0/10

   Vapi is presented exclusively as a hosted cloud platform (dashboard, hosted MCP server, hosted API/CLI against cloud endpoints); nothing in the evidence pack mentions a self-hostable core engine, open-source repo for the core product, or on-prem deployment option (the mentioned pipecat is a different open-source project, not Vapi itself). This is a fair axis for a voice-AI platform, but no evidence supports self-hosting.

   • [community] “Vapi is also built on media framework by daily.co. They have an open source version of voice ai called pipecat.”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”
   • [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”

   proof ↗⚑ flag

   ElevenLabs Agents—none0/10

   ElevenLabs Agents is entirely a hosted SaaS product—managed via dashboard, API, CLI, and hosted MCP server, all pointing to ElevenLabs' own infrastructure; there is no evidence of an open-source release, downloadable server binary, or self-hosting option for the core voice-agent runtime. missing for 10: any open-source repo or self-hosted deployment package, docs describing running the core service on one's own infrastructure, independent confirmation of self-hosting.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
   • [claimed-docs] “create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally”

   proof ↗⚑ flag

Pricing

1. founderSee published per-minute or usage pricing and estimate cost per call before committing

   weight 2 · round drawn
   Vapi—none0/10

   The evidence pack contains no mention of pricing pages, per-minute rates, usage-based cost breakdowns, or any pricing calculator/estimator; all evidence is about docs, CLI, MCP server, tools, and testing features. Absence of evidence for this applicable axis yields none.

   ⚑ flag

   ElevenLabs Agents—none0/10

   No evidence item in the pack references pricing, per-minute rates, usage-based billing, or cost calculators for ElevenLabs Agents; all citations are product/docs feature descriptions. Since founders evaluating a voice-agent SaaS could reasonably expect published pricing, the axis applies but is entirely unaddressed in this evidence pack.

   ⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round drawn
   Vapi—none0/10

   The evidence pack contains no mention of data residency, regional storage options, or geographic data controls anywhere in Vapi's docs (only HIPAA toggle and SSO/RBAC are mentioned as compliance features). Missing for 10: any documentation of region selection, data residency guarantees, or storage location controls.

   ⚑ flag

   ElevenLabs Agents—none0/10

   Evidence covers data retention duration settings, HIPAA/BAA eligibility, and infrastructure details, but nothing addresses selectable data region/residency options for storage location. missing for 10: any mention of region selection, data residency controls, or geographic storage options.

   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings.”
   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings”
   • [claimed-docs] “ElevenLabs Agents is one of ElevenLabs' HIPAA-eligible services, and we offer Business Associate Agreements (BAAs) to eligible customers”

   proof ↗⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   Vapi—none0/10

   No evidence in the pack addresses data-training opt-out, model-training data policies, or privacy controls preventing AI training use; only HIPAA and SSO/RBAC enterprise features are mentioned, which are unrelated to this specific claim. missing for 10: explicit data-training opt-out policy, DPA/ToS language on training use, dashboard/API setting to exclude data from model training.

   ⚑ flag

   ElevenLabs Agents—none0/10

   The evidence pack covers data retention settings (how long transcripts/audio are stored) and HIPAA/BAA compliance, but nowhere mentions an explicit opt-out or policy regarding use of customer data/conversations for AI model training. missing for 10: explicit training-data opt-out policy or setting, terms-of-service language on model training use, any statement distinguishing enterprise vs free-tier data usage for training.

   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings.”
   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings”
   • [claimed-docs] “ElevenLabs Agents is one of ElevenLabs' HIPAA-eligible services, and we offer Business Associate Agreements (BAAs) to eligible customers”

   proof ↗⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round to ElevenLabs Agents
   Vapi—none0/10

   The evidence pack covers HIPAA, SSO/RBAC, tooling, CLI, and MCP server features but contains no documentation about data retention policies, call/recording deletion controls, or user-initiated data export/erasure mechanisms. Missing for 10: retention period settings, deletion/erasure APIs or dashboard controls, data export tools, and any policy documentation on how long call data/transcripts are stored.

   ⚑ flag

   ElevenLabs Agents~partialclaimed6/10

   Docs explicitly document configurable retention settings for conversation transcripts and audio recordings, giving users control over how long data is stored. However, evidence does not detail a deletion mechanism (e.g., data subject deletion requests, API/CLI commands to purge stored data) beyond retention duration settings. Missing for 10: explicit deletion/export controls, granular per-conversation deletion API, and independent verification of retention enforcement.

   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings.”
   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings”

   proof ↗⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Vapi—none0/10

   No evidence pack item addresses telemetry opt-out or usage tracking controls; Vapi's docs cover HIPAA/SSO/RBAC compliance but not a telemetry toggle.

   ⚑ flag

   ElevenLabs Agents—none0/10

   The evidence pack covers conversation transcript/audio retention settings (docs-26, docs-52) but nothing about opting out of product telemetry or usage tracking (e.g., CLI analytics, dashboard usage metrics) for the AI-native user. Retention configuration is about customer conversation data, not the vendor's own telemetry collection, so this axis has no supporting evidence.

   ⚑ flag

Call control

1. developerEscalate a live call to a human with warm or blind transfer, passing context along

   weight 2 · round drawn
   Vapi—none0/10

   No evidence in the pack mentions call transfer, warm/blind transfer, or passing context to a human agent; the evidence covers assistants, CLI, MCP server, tools/webhooks, voices, and compliance, but nothing about live-call escalation/transfer capabilities.

   ⚑ flag

   ElevenLabs Agents—none0/10

   The evidence pack covers telephony features like SIP trunking, tools, webhooks, and MCP integrations, but nowhere documents call transfer (warm or blind) to a human agent or passing conversation context during handoff. Missing for 10: any mention of transfer_to_number/transfer_to_agent tool types, warm/blind transfer configuration, or context-passing during escalation.

   ⚑ flag

2. developerMy agent can send DTMF keypresses, navigate IVR menus, and detect or leave voicemail

   weight 1 · round drawn
   Vapi—none0/10

   The evidence pack contains no mention of DTMF keypress sending, IVR menu navigation, or voicemail detection/leaving functionality anywhere in the docs or community items — these telephony-specific capabilities are entirely unevidenced despite being a plausible axis for a voice AI telephony product.

   ⚑ flag

   ElevenLabs Agents—none0/10

   The evidence pack confirms ElevenLabs Agents supports telephony integration via SIP trunking and phone numbers, so the telephony axis clearly applies, but nothing in the docs mentions DTMF tone sending, IVR menu navigation, or voicemail detection/leaving capabilities.

   • [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”
   • [claimed-docs] “Route calls to AI agents without changing your existing phone infrastructure”

   proof ↗⚑ flag

Campaigns

1. founderRun batch outbound call campaigns with scheduling and throughput controls

   weight 2 · round drawn
   Vapi—none0/10

   Evidence covers assistant creation, tools, CLI, MCP server, testing, and voice customization, but nothing addresses batch/outbound campaign management, call scheduling, or throughput/concurrency controls for bulk dialing. No mention of a campaigns API, CSV/list upload, dialer pacing, or rate-limiting controls for outbound calling at scale.

   ⚑ flag

   ElevenLabs Agents—none0/10

   The evidence pack documents phone-number/SIP trunking integration for routing calls to agents, but contains no mention of batch outbound calling, campaign scheduling, or throughput/rate controls for outbound call volume. Since telephony campaign management is a fair capability to expect from a voice-agent telephony product, absence of evidence here is 'none' rather than 'na'.

   • [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”
   • [claimed-docs] “Route calls to AI agents without changing your existing phone infrastructure”

   proof ↗⚑ flag

Numbers

1. developerProvision phone numbers and run both inbound and outbound calls through the platform's API

   weight 3 · round to Vapi
   Vapi✓fullprobed8/10

   Docs explicitly cover creating an assistant, connecting it to a phone number, and making inbound/outbound calls via the quickstart, plus CLI/MCP support for managing phone numbers and calls, and SIP for advanced telephony integration, with runtime probes confirming the CLI and MCP endpoints work as documented. Missing for 10: independent hands-on confirmation of actual outbound call placement via raw API (only demo/inbound anecdote in community evidence) and explicit multi-number provisioning workflow details.

   • [claimed-docs] “const assistant = await vapi.assistant”
   • [claimed-docs] “Create a voice assistant, connect it to a phone number, and make your first calls.”
   • [claimed-docs] “In under 5 minutes, you'll create a voice assistant and start talking to it over the phone.”
   • [claimed-docs] “Manage assistants, phone numbers, and calls from your terminal”
   • [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
   • [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
   • [claimed-docs] “Use any SIP softphone (e.g., Zoiper, Linphone) to dial your SIP URI”
   • [claimed-docs] “This guide shows you how to set up and test SIP calls to your Vapi assistant using any SIP client or softphone.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bas…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”
   • [community] “Called the demo number, sounds smooth! Good luck.”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed6/10

   Docs confirm phone/telephony integration via SIP trunking, letting agents route inbound/outbound calls over existing phone infrastructure, and agents can be created/configured via the API. However, evidence does not explicitly show a documented API endpoint for provisioning new phone numbers or explicitly triggering outbound calls programmatically vs. via SIP trunk setup. Missing for 10: explicit phone-number provisioning API endpoint docs, explicit outbound-call-initiation API example, and independent/hands-on confirmation of inbound+outbound call flows.

   • [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”
   • [claimed-docs] “Route calls to AI agents without changing your existing phone infrastructure”
   • [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”

   proof ↗⚑ flag

Sip

1. platform-engineerConnect my own carrier or PBX via SIP trunking (or import Twilio/Telnyx numbers) instead of being locked to bundled telephony

   weight 2 · round to ElevenLabs Agents
   Vapi~partialclaimed5/10

   Docs confirm SIP connectivity: platform engineers can point any SIP client/softphone at a Vapi SIP URI to make/receive calls (vapi-docs-14, vapi-docs-22), which supports self-hosted/carrier SIP trunking use cases. However, the evidence never explicitly documents PBX/carrier trunk configuration options or a number-import workflow for Twilio/Telnyx accounts. missing for 10: explicit documentation of BYO-carrier/PBX trunk setup steps, and evidence of importing existing Twilio/Telnyx phone numbers into Vapi.

   • [claimed-docs] “Use any SIP softphone (e.g., Zoiper, Linphone) to dial your SIP URI”
   • [claimed-docs] “This guide shows you how to set up and test SIP calls to your Vapi assistant using any SIP client or softphone.”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed7/10

   Docs explicitly describe a SIP trunking integration that lets customers connect their existing phone systems/carrier infrastructure to ElevenLabs agents without changing carriers, directly addressing the core ask of bringing your own PBX/carrier. However, the pack has no explicit mention of importing existing Twilio/Telnyx numbers or detailed carrier onboarding steps, and no independent/hands-on confirmation of a live SIP trunk setup working end-to-end. Missing for 10: explicit Twilio/Telnyx number import instructions, and third-party/hands-on validation of the SIP trunking flow.

   • [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”
   • [claimed-docs] “Route calls to AI agents without changing your existing phone infrastructure”

   proof ↗⚑ flag

Analytics

1. ai-native userThe platform's AI reviews my calls for me — scoring quality, flagging failures, and analyzing resolution automatically

   weight 2 · round to ElevenLabs Agents
   Vapi~partialclaimed4/10

   Vapi's Evals framework provides automated validation of assistant behavior via mock conversations and its Voice Test Suites use an AI tester to simulate calls and check outcomes, which is adjacent to automated call review, but neither is documented as automatically scoring, flagging failures in, or analyzing resolution of real production calls after the fact. Missing for 10: evidence of automatic post-call quality scoring on live/production transcripts, explicit failure-flagging analytics, and resolution-outcome analysis dashboards distinct from pre-production test suites.

   • [claimed-docs] “Evals is Vapi's AI agent testing framework that enables you to systematically test assistants and squads using mock conversations with autom…”
   • [claimed-docs] “you'll create mock conversations, define expected behaviors, and validate your agents work correctly before production”
   • [claimed-docs] “Voice Test Suites enable you to test your AI voice agents through simulated phone conversations.”
   • [claimed-docs] “Our AI tester calls your voice agent and follows a script that simulates real customer behavior.”

   proof ↗⚑ flag

   ElevenLabs Agents✓fullclaimed7/10

   Docs explicitly describe automated post-call analysis: 'Success evaluation' scores conversation quality/goal achievement, 'Sentiment analysis' flags user sentiment, and 'Data collection' extracts structured resolution details — directly matching the story's scoring/flagging/resolution-analysis ask. Missing for 10: no independent/hands-on validation of analysis accuracy, and no explicit 'failure flagging' terminology beyond success-criteria evaluation.

   • [claimed-docs] “Success evaluation: Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Data collection: Extract specific data points from conversations such as contact information, issue details, or any structured information”
   • [claimed-docs] “Sentiment analysis: Understand user sentiment across completed conversations”
   • [claimed-docs] “Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Extract structured information from conversations such as contact details and business data.”

   proof ↗⚑ flag

2. founderSee call analytics — success rates, durations, outcomes, sentiment — in dashboards without building my own

   weight 2 · round to ElevenLabs Agents
   Vapi—none0/10

   The evidence pack documents assistant creation, tools, testing (Evals, Voice Test Suites), CLI/MCP management, and voice library features, but contains no mention of a dashboard showing call success rates, durations, outcomes, or sentiment analytics — the core of this story.

   ⚑ flag

   ElevenLabs Agents~partialclaimed6/10

   Docs confirm built-in agent-analysis features—success evaluation, sentiment analysis, data extraction, and conversation search—managed via the ElevenAgents dashboard, which covers most of the story's asked-for metrics without custom-building analytics. However, there's no explicit evidence of a dashboard showing aggregated success-rate percentages or call-duration analytics/rollups across calls, only per-conversation criteria and settings. Missing for 10: dashboard screenshots/evidence of aggregate success-rate metrics, call-duration reporting/rollups, and independent corroboration that these appear in a unified analytics view.

   • [claimed-docs] “Success evaluation: Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Data collection: Extract specific data points from conversations such as contact information, issue details, or any structured information”
   • [claimed-docs] “Sentiment analysis: Understand user sentiment across completed conversations”
   • [claimed-docs] “Searching conversations: Find messages by keyword or meaning across your conversation history.”
   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “Extract structured information from conversations such as contact details and business data.”

   proof ↗⚑ flag

Monitoring

1. platform-engineerMonitor live calls in production and get alerts when agents misbehave or error rates spike

   weight 1 · round to ElevenLabs Agents
   Vapi—none0/10

   The evidence pack covers pre-production testing (Voice Test Suites, Evals) and webhook debugging via the CLI, but contains no mention of live call monitoring dashboards, real-time alerting, or error-rate-spike detection for production traffic. This is a fair axis for a voice AI platform, but nothing in the pack demonstrates it.

   ⚑ flag

   ElevenLabs Agents~partialclaimed4/10

   Docs show post-call analytics (sentiment analysis, success evaluation, data extraction, conversation search) and a marketing claim to 'monitor' agents, but there is no evidence of live/real-time call monitoring dashboards, error-rate metrics, or alerting/notification mechanisms for misbehaving agents. Missing for 10: real-time monitoring dashboard, error-rate/latency metrics, alerting/notification system, on-call integration (e.g., PagerDuty/Slack alerts).

   • [claimed-docs] “Success evaluation: Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Sentiment analysis: Understand user sentiment across completed conversations”
   • [claimed-docs] “Searching conversations: Find messages by keyword or meaning across your conversation history.”
   • [claimed-docs] “Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Configure, deploy and monitor natural, human-sounding agents in 70+ languages with leading accuracy and ultra-low latency across voice or ch…”

   proof ↗⚑ flag

Testing

1. developerTest agents with simulated conversations or evals before putting them on real phone calls

   weight 2 · round to Vapi
   Vapi✓fullclaimed9/10

   Vapi documents two dedicated testing features directly matching the story: Voice Test Suites for simulated phone conversations via an AI tester following scripted customer behavior, and Evals, a testing framework for mock conversations with automated validation before production. Missing for 10: independent/hands-on developer corroboration of these specific testing features beyond vendor docs.

   • [claimed-docs] “Voice Test Suites enable you to test your AI voice agents through simulated phone conversations.”
   • [claimed-docs] “Our AI tester calls your voice agent and follows a script that simulates real customer behavior.”
   • [claimed-docs] “Evals is Vapi's AI agent testing framework that enables you to systematically test assistants and squads using mock conversations with autom…”
   • [claimed-docs] “you'll create mock conversations, define expected behaviors, and validate your agents work correctly before production”

   proof ↗⚑ flag

   ElevenLabs Agents✓fullclaimed8/10

   ElevenLabs documents an explicit Agent Testing feature that verifies conversational responses, tool usage, and full multi-turn outcomes before deployment, including converting real conversations into simulated test cases, plus success-evaluation criteria for assessing conversation quality/goal achievement — directly matching the pre-deployment simulated testing story. Missing for 10: no independent/hands-on report validating test accuracy or CI-integration details, and no explicit mention of automated eval scoring pipelines beyond docs description.

   • [claimed-docs] “Agent testing lets you verify conversational responses, tool usage, and full multi-turn outcomes before you deploy.”
   • [claimed-docs] “Transform real conversations into test cases when you find an interaction where the agent underperformed.”
   • [claimed-docs] “Agent testing lets you verify conversational responses, tool usage, and full multi-turn outcomes before you deploy”
   • [claimed-docs] “Success evaluation: Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”

   proof ↗⚑ flag

Post call

1. developerExtract structured data from every call — outcomes, entities, dispositions — delivered via API or webhook after the call

   weight 2 · round to ElevenLabs Agents
   Vapi—none0/10

   The evidence pack shows Vapi's webhook/tool-calling system for live in-call actions (tool-calls messages, custom webhook tools, function calling) but contains no mention of a post-call structured-data/analysis feature (outcomes, entities, dispositions) delivered via API or webhook after the call ends. Missing for 10: any docs on end-of-call reports, structured data extraction schemas, call analysis/summary webhooks, or an API endpoint returning call outcome/entity data.

   • [claimed-docs] “When tools are triggered, your Server URL receives a `tool-calls` message”
   • [claimed-docs] “Vapi supports OpenAI-style tool/function calling. Assistants can ping your server to perform actions.”
   • [claimed-docs] “Custom tools that you create... interact with your systems via webhooks”
   • [claimed-docs] “Create your own webhook-based tools to extend assistant capabilities”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed6/10

   Docs show structured post-call outputs (data collection with contact/business data extraction, success-evaluation criteria, sentiment analysis) available via 'agent-analysis' features, and the platform is manageable via API. However, the pack never explicitly documents a webhook payload/callback delivering this structured analysis after a call, only webhook *tools* for in-call actions — missing for 10: explicit post-call webhook delivery mechanism/schema for outcomes and dispositions, and confirmation these analysis fields are retrievable via a documented API endpoint.

   • [claimed-docs] “Success evaluation: Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Data collection: Extract specific data points from conversations such as contact information, issue details, or any structured information”
   • [claimed-docs] “Sentiment analysis: Understand user sentiment across completed conversations”
   • [claimed-docs] “Extract structured information from conversations such as contact details and business data”
   • [claimed-docs] “Extract structured information from conversations such as contact details and business data.”
   • [claimed-docs] “Tools give your assistant the ability to make external function calls to third party apps so you can get real-time information.”
   • [claimed-docs] “allow an assistant to trigger authenticated actions based on the conversation, like scheduling meetings or initiating order returns”

   proof ↗⚑ flag

Tools

1. ai-native userMy voice agent can plug in MCP servers as tool sources so one integration grants it whole toolsets mid-call

   weight 2 · round to ElevenLabs Agents
   Vapi—none0/10

   Evidence only shows Vapi exposing its own APIs as an MCP *server* for external AI assistants (e.g., Claude Desktop) to manage calls/assistants — the reverse direction of the story. There is no documentation or probe showing a Vapi voice assistant can itself act as an MCP *client*, plugging in external MCP servers as tool sources mid-call; tool integration is instead described only via webhook-based custom tools and OpenAI-style function calling.

   • [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
   • [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
   • [claimed-docs] “Create your own webhook-based tools to extend assistant capabilities”
   • [claimed-docs] “Vapi supports OpenAI-style tool/function calling. Assistants can ping your server to perform actions.”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”

   proof ↗⚑ flag

   ElevenLabs Agents✓fullclaimed8/10

   Docs explicitly describe connecting conversational agents to external MCP servers as a tool source, enabling access to data/processing capabilities via that server mid-call, directly matching the story. Missing for 10: independent hands-on confirmation of mid-call multi-tool retrieval from a connected MCP server (only vendor docs, no third-party validation) and detail on how many tools/toolsets a single MCP connection exposes.

   • [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers. This enables your agents to: Access and process informa…”
   • [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers.”
   • [claimed-docs] “Access and process information from various data sources via the MCP server”

   proof ↗⚑ flag

2. developerMy agent can call external APIs and custom functions mid-conversation and speak the result without awkward dead air

   weight 3 · round to ElevenLabs Agents
   Vapi~partialclaimed6/10

   Vapi's docs clearly show mid-call function/tool calling via webhook Server URLs and OpenAI-style tool-calls messages, letting the assistant fetch external API results and use them in conversation (vapi-docs-6, vapi-docs-7, vapi-docs-17, vapi-docs-20, vapi-docs-23, vapi-docs-34). However, the pack lacks explicit evidence about mechanisms for avoiding 'dead air' during the API call latency (e.g., async tool config, filler phrases, or interim speech) — only general latency-tuning docs for model selection are present (vapi-docs-15). Missing for 10: explicit documentation/demo of filler/interim speech or async tool handling during function execution, and independent hands-on confirmation that the conversation flow feels seamless during a live tool call.

   • [claimed-docs] “Create your own webhook-based tools to extend assistant capabilities”
   • [claimed-docs] “Server URL: The endpoint where your function is hosted”
   • [claimed-docs] “When tools are triggered, your Server URL receives a `tool-calls` message”
   • [claimed-docs] “This guide shows you how to create custom tools, including Function Tools, for your Vapi assistants.”
   • [claimed-docs] “Vapi supports OpenAI-style tool/function calling. Assistants can ping your server to perform actions.”
   • [claimed-docs] “Custom tools that you create... interact with your systems via webhooks”
   • [claimed-docs] “Ultra Fast uses speed-optimized models for lower latency. High Intelligence accepts higher latency for greater capability.”

   proof ↗⚑ flag

   ElevenLabs Agents✓fullclaimed7/10

   Docs clearly describe multiple tool-calling mechanisms — webhook tools for external API calls, client-side tools, sandboxed custom JS, and MCP server integration — explicitly for making external function calls mid-conversation to fetch real-time info and trigger authenticated actions. Conversation-flow settings (turn-taking, interruption handling, ultra-low latency) address smooth conversational pacing, though not explicitly framed as avoiding dead air during tool execution. Missing for 10: explicit documentation of filler/hold-speech behavior while awaiting API results, and independent/hands-on verification that tool calls don't introduce audible pauses.

   • [claimed-docs] “Enable agents to call clients & APIs to perform actions”
   • [claimed-docs] “Tools executed directly on the client-side application (e.g., web browser, mobile app).”
   • [claimed-docs] “Tools give your assistant the ability to make external function calls to third party apps so you can get real-time information.”
   • [claimed-docs] “Custom JavaScript executed in a sandboxed environment on ElevenLabs' infrastructure.”
   • [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers. This enables your agents to: Access and process informa…”
   • [claimed-docs] “Tools allow ElevenLabs agents to perform actions beyond generating text responses.”
   • [claimed-docs] “allow an assistant to trigger authenticated actions based on the conversation, like scheduling meetings or initiating order returns”
   • [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”
   • [claimed-docs] “Configure, deploy and monitor natural, human-sounding agents in 70+ languages with leading accuracy and ultra-low latency across voice or ch…”

   proof ↗⚑ flag

Recording

1. platform-engineerRetrieve full call recordings and transcripts programmatically for every call

   weight 2 · round to ElevenLabs Agents
   Vapi—none0/10

   The evidence pack contains no documentation, API reference, or example demonstrating retrieval of call recordings or transcripts, despite this being a natural, expected capability for a voice AI platform; only tangential tooling (CLI, MCP server, custom tools, testing/evals) is covered.

   ⚑ flag

   ElevenLabs Agents~partialclaimed5/10

   Docs confirm that conversation transcripts and audio recordings are stored with configurable retention, and that conversations can be searched, analyzed, and have structured data extracted (docs-26/52, docs-21/23/56), implying underlying storage of call recordings/transcripts. However, there is no explicit documentation of a specific API endpoint or method for programmatically retrieving full call recordings and transcripts on a per-call basis. Missing for 10: an explicit 'get conversation' or 'get recording/transcript' API reference, example request/response showing programmatic export of audio + transcript per call.

   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings.”
   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings”
   • [claimed-docs] “Data collection: Extract specific data points from conversations such as contact information, issue details, or any structured information”
   • [claimed-docs] “Searching conversations: Find messages by keyword or meaning across your conversation history.”
   • [claimed-docs] “Extract structured information from conversations such as contact details and business data.”

   proof ↗⚑ flag

Transcription

1. developerGet accurate real-time transcription with control over the STT provider, language models, or key terms

   weight 2 · round to ElevenLabs Agents
   Vapi—none0/10

   The evidence pack contains no documentation about selecting/configuring an STT provider, choosing language models for transcription, or defining custom key terms/vocabulary for real-time transcription — topics like 'transcriber', 'Deepgram', or keyword boosting are absent. While Vapi is clearly a voice AI platform where such controls are a fair axis, none of the provided docs, community items, or probes address it.

   ⚑ flag

   ElevenLabs Agents~partialclaimed4/10

   Docs confirm agents let developers choose or bring custom LLMs (docs-41) and mention real-time voice interaction (docs-37), but there is no evidence of selectable STT providers, transcription accuracy tuning, or 'key terms'/vocabulary boosting controls anywhere in the pack. Missing for 10: STT provider selection, transcription-specific accuracy settings, key-term/glossary support, and any independent verification of transcription accuracy claims.

   • [claimed-docs] “Choose from supported LLMs or bring your own custom model”
   • [claimed-docs] “Configure, deploy and monitor natural, human-sounding agents in 70+ languages with leading accuracy and ultra-low latency across voice or ch…”
   • [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”

   proof ↗⚑ flag

Voices

1. founderClone a custom brand voice and use it for my agents, with a documented consent process

   weight 2 · round drawn
   Vapi—none0/10

   Vapi docs show you can plug in a 'custom voice' by setting the voice property with a provider ID (vapi-docs-9, vapi-docs-26) and browse a Voice Library (vapi-docs-8, vapi-docs-35), but nothing describes an actual voice-cloning workflow or any documented consent/verification process required before cloning a brand voice. Missing for 10: a described voice-cloning feature/flow, a documented consent or identity-verification process, and any policy language governing voice cloning.

   • [claimed-docs] “You can use your own custom voice with any supported provider by setting the `voice` property in your assistant configuration”
   • [claimed-docs] “You can use your own custom voice with any supported provider by setting the voice property in your assistant configuration”
   • [claimed-docs] “The Voice Library in the Vapi Dashboard lists every voice available to your organization. Browse and preview voices there”
   • [claimed-docs] “The **Voice Library** in the [Vapi Dashboard] lists every voice available to your organization. Browse and preview voices there, then copy a…”

   proof ↗⚑ flag

   ElevenLabs Agents—none0/10

   The evidence pack describes selecting from 5k+ pre-built voices and voice customization (speed, multi-voice switching) but contains no mention of a voice-cloning feature for creating a custom brand voice, nor any documented consent process for cloning (e.g., voice verification, spoken consent statement) required by ElevenLabs' broader platform. Missing for 10: any docs on voice cloning/Instant or Professional Voice Cloning for Agents, and any consent-verification workflow.

   • [claimed-docs] “Select from 5k+ voices across 31 languages with customization options”
   • [claimed-docs] “Enable your agent to switch between different voices for multi-character conversations, storytelling, and language tutoring.”
   • [claimed-docs] “You can customize various aspects of your AI agent's voice to create a more natural and engaging conversation experience. This includes cont…”

   proof ↗⚑ flag

2. developerChoose from a broad voice library or plug in multiple TTS providers to get the voice I want

   weight 2 · round to Vapi
   Vapi~partialclaimed6/10

   Docs confirm a Voice Library with many previewable voices and support for custom voices with any supported provider, indicating multi-provider TTS flexibility, but the evidence never names or lists specific TTS providers (e.g., ElevenLabs, PlayHT, Azure) or details plugging in third-party/custom TTS engines beyond voice ID selection. missing for 10: explicit list of supported TTS providers, documentation of custom/BYO TTS provider integration mechanics, independent hands-on confirmation of voice quality/variety.

   • [claimed-docs] “The Voice Library in the Vapi Dashboard lists every voice available to your organization. Browse and preview voices there”
   • [claimed-docs] “You can use your own custom voice with any supported provider by setting the `voice` property in your assistant configuration”
   • [claimed-docs] “Browse and preview voices there, then copy a voice's ID to use on an assistant.”
   • [claimed-docs] “You can use your own custom voice with any supported provider by setting the voice property in your assistant configuration”
   • [claimed-docs] “The **Voice Library** in the [Vapi Dashboard] lists every voice available to your organization. Browse and preview voices there, then copy a…”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed5/10

   Docs confirm a broad in-house voice library (5k+ voices across 31 languages, docs-6) with customization like speed, pronunciation, and multi-voice switching (docs-17/32/48), satisfying the first half of the story. However, there is no evidence of plugging in third-party/alternative TTS providers — the docs only mention bring-your-own LLM (docs-41), not bring-your-own TTS engine. Missing for 10: any documented support for external/third-party TTS provider integration, evidence of provider-agnostic voice pipeline, independent confirmation of using non-ElevenLabs voices.

   • [claimed-docs] “Select from 5k+ voices across 31 languages with customization options”
   • [claimed-docs] “Enable your agent to switch between different voices for multi-character conversations, storytelling, and language tutoring.”
   • [claimed-docs] “You can customize various aspects of your AI agent's voice to create a more natural and engaging conversation experience. This includes cont…”
   • [claimed-docs] “Enable your agent to switch between different voices for multi-character conversations, storytelling, and language tutoring”
   • [claimed-docs] “Choose from supported LLMs or bring your own custom model”

   proof ↗⚑ flag