Vapi vs Retell AI

Agent ops

1. ai-native userMy coding agent can provision a complete voice agent end to end — create the agent, attach a number, and place a call — through the API, CLI, or MCP without touching the dashboard

   weight 3 · round to Vapi
   Vapi✓fullprobed8/10

   Vapi documents API/SDK-based assistant creation, phone number attachment, and call placement (vapi-docs-1, vapi-docs-18, vapi-docs-19), a CLI for managing assistants, phone numbers, and calls entirely from the terminal (vapi-docs-2, vapi-docs-31, vapi-probe-rt-1 confirming keyless install/version), and an official MCP server exposing these same operations to any MCP-compatible agent, verified live and auth-gated in a runtime probe (vapi-docs-33, vapi-probe-rt-2). Together these three surfaces (API, CLI, MCP) cover the full agent-provision-number-call workflow without dashboard use. missing for 10: no single end-to-end hands-on trace showing one agent chaining create→attach→call purely via CLI/MCP/API in one session, and no independent (non-vendor) confirmation of the full workflow succeeding.

   • [claimed-docs] “const assistant = await vapi.assistant”
   • [claimed-docs] “Manage assistants, phone numbers, and calls from your terminal”
   • [claimed-docs] “Create a voice assistant, connect it to a phone number, and make your first calls.”
   • [claimed-docs] “In under 5 minutes, you'll create a voice assistant and start talking to it over the phone.”
   • [claimed-docs] “Build, test, and deploy voice AI applications without leaving your development environment.”
   • [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
   • [probe] “official MCP server documented at https://docs.vapi.ai/sdk/mcp-server”
   • [probe] “official CLI documented at https://docs.vapi.ai/cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bas…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”

   proof ↗⚑ flag

   Retell AI~partialprobed6/10

   Retell ships an official Node/Python SDK with full voice endpoint coverage, a CLI for managing agents and phone numbers, and an MCP server whose meta-tools (list/get/invoke_api_endpoint) expose the entire API — including agent create/update/publish — to MCP clients like Cursor or Claude Code, and a keyless runtime probe confirms the MCP handshake works end-to-end. However, the flagship quick-start walkthrough is dashboard-centric (create in UI, assign number in 'configuration settings', test/call via dashboard button), and no evidence explicitly shows a CLI/API/MCP call sequence that attaches a number and places a live call without touching the UI. Missing for 10: an explicit end-to-end CLI/MCP example showing number-attach and call-placement commands, and independent confirmation that non-dashboard number provisioning/outbound calling works in practice.

   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”
   • [claimed-docs] “Use Retell's MCP server to build and manage voice agents from MCP-capable clients like Cursor, Claude Desktop, and Claude Code via Retell AP…”
   • [claimed-docs] “Retell supports the Model Context Protocol (MCP) so you can build Retell AI voice agents directly from MCP-capable clients (Cursor, Claude D…”
   • [claimed-docs] “Agents: create, update, publish, list, and fetch agent versions.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”
   • [claimed-docs] “Assign your agent to the number in the configuration settings”

   proof ↗⚑ flag

2. ai-native userThe platform's own AI helps me author agents — generating or improving prompts, flows, and test cases from a description

   weight 1 · round to Retell AI
   Vapi—none0/10

   Evidence shows Vapi provides CLI, MCP server, tool/webhook infrastructure, and testing frameworks (Evals, Voice Test Suites), plus a Customer Support template, but nothing indicates the platform itself uses AI to generate or improve prompts, conversation flows, or test cases from a natural-language description — testing tools require manually defined mock conversations/scripts rather than AI-authored ones.

   • [claimed-docs] “Voice Test Suites enable you to test your AI voice agents through simulated phone conversations.”
   • [claimed-docs] “Evals is Vapi's AI agent testing framework that enables you to systematically test assistants and squads using mock conversations with autom…”
   • [claimed-docs] “Our AI tester calls your voice agent and follows a script that simulates real customer behavior.”
   • [claimed-docs] “you'll create mock conversations, define expected behaviors, and validate your agents work correctly before production”
   • [claimed-docs] “Select the down arrow next to Create Assistant, then choose Customer Support.”

   proof ↗⚑ flag

   Retell AI~partialclaimed5/10

   Retell docs mention a 'Generate from prompt' feature where Conductor drafts an agent from a plain-English description (retell-docs-30), directly supporting AI-assisted agent authoring. However, there's no evidence of AI-driven improvement of existing prompts/flows or automatic test-case generation, and no independent/hands-on confirmation of Conductor's output quality. Missing for 10: documentation on iterative prompt/flow refinement by the platform's AI, automated test-case generation, and community or hands-on verification of Conductor's generated agents.

   • [claimed-docs] “click **Generate from prompt** (marked Suggested) to let [Conductor](/conductor/create-agent) draft an agent from a plain-English descriptio…”
   • [claimed-docs] “Retell agents are node-based flows or single prompts, with call handling, a knowledge base, and integrations built in.”
   • [claimed-docs] “Drag-and-drop, node-based flows for structured, high-stakes calls.”
   • [claimed-docs] “Single prompt agent... Define your whole agent with one prompt.”

   proof ↗⚑ flag

Build

1. developerBuild a working phone voice agent — prompt, voice, and phone number — and take my first live call within an hour

   weight 3 · round to Vapi
   Vapi✓fullcommunity9/10

   Docs explicitly walk through creating a voice assistant, connecting a phone number, and making a first call in under 5 minutes (vapi-docs-18, vapi-docs-19), with voice selection via the Voice Library (vapi-docs-8, vapi-docs-24) and dashboard-driven assistant creation with templates (vapi-docs-16); community evidence corroborates a working live-call demo (vapi-comm-1). Missing for 10: independent third-party benchmarking of the full 'within an hour' timing claim beyond vendor docs and a single anecdotal community comment.

   • [claimed-docs] “Create a voice assistant, connect it to a phone number, and make your first calls.”
   • [claimed-docs] “In under 5 minutes, you'll create a voice assistant and start talking to it over the phone.”
   • [claimed-docs] “Select the down arrow next to Create Assistant, then choose Customer Support.”
   • [claimed-docs] “The Voice Library in the Vapi Dashboard lists every voice available to your organization. Browse and preview voices there”
   • [claimed-docs] “Browse and preview voices there, then copy a voice's ID to use on an assistant.”
   • [community] “Called the demo number, sounds smooth! Good luck.”

   proof ↗⚑ flag

   Retell AI✓fullcommunity8/10

   Docs walk through the exact flow described: create account, pick/generate a template, set prompt and voice, test in dashboard, assign a phone number, and make a live call, with a 15-minute quickstart and free trial credits removing payment friction. Community feedback confirms the demo/agent works end-to-end for real calls, though it also shows occasional conversational glitches (contradictions, confusion) that are quality issues rather than build-flow blockers. missing for 10: independent third-party timing confirmation that a full build-to-first-call truly completes within an hour, and evidence of phone-number provisioning speed in practice.

   • [claimed-docs] “Build your first Retell AI phone agent in 15 minutes: create an account, pick a template, test in the dashboard, deploy to a phone number, a…”
   • [claimed-docs] “Click the "Test" button to start a web call with your agent... This step is free and doesn't need a phone number or payment method.”
   • [claimed-docs] “click **Generate from prompt** (marked Suggested) to let [Conductor](/conductor/create-agent) draft an agent from a plain-English descriptio…”
   • [claimed-docs] “Click the "Test" button to start a web call with your agent”
   • [claimed-docs] “Assign your agent to the number in the configuration settings”
   • [claimed-docs] “New accounts start with \$10 in free trial credits, no payment method required.”
   • [claimed-docs] “Add custom voices to your Retell agent — search ElevenLabs community voices, import a voice clone, or train a clone for a unique brand-speci…”
   • [community] “Curious what model the dentist bot is running on? Tried it out, was surprisingly good, though eventually it contradicted itself (booked a sl…”
   • [community] “I tried to fool the Dental Office demo trying to book on Sunday or outside of the slots it had indicated, and it did a better job than many …”

   proof ↗⚑ flag

2. developerRun conversations in multiple languages, including detecting and switching language mid-call

   weight 2 · round drawn
   Vapi—none0/10

   The evidence pack covers CLI, MCP server, tools, voices, compliance, and testing, but contains no mention of multi-language support, language detection, or mid-call language switching capabilities. Absence of evidence for this applicable axis (multilingual voice agents are a fair capability to expect from a voice AI platform) means it must be judged 'none'. missing for 10: any mention of multi-language conversation support, language auto-detection, or mid-call language switching features.

   ⚑ flag

   Retell AI—none0/10

   No documentation in the evidence pack describes multi-language conversation support, language detection, or mid-call language switching; the only related evidence is a founder comment stating multilingual support is merely 'on our roadmap' rather than shipped, indicating the capability does not currently exist.

   • [community] “Founder said multilingual support is on roadmap: 'It's definitely in our roadmap. After the core product—the voice AI part—becomes humanlike…”

   proof ↗⚑ flag

3. founderDesign multi-step conversation flows in a visual builder with branching, states, and handoffs without writing code

   weight 2 · round to Retell AI
   Vapi—none0/10

   Evidence covers Vapi's assistant configuration, tools, CLI, MCP server, voices, and testing, but there is no mention of a visual flow builder with branching, states, or handoffs — Vapi appears configuration/code-driven (assistant config, function tools, webhooks) rather than a no-code visual conversation designer.

   ⚑ flag

   Retell AI✓fullclaimed7/10

   Retell's docs describe node-based, drag-and-drop conversation flows for 'structured, high-stakes calls' as an alternative to single-prompt agents, with function calling enabling transfers, call-ending, and API calls that serve as handoff points — directly matching the branching/states/handoff story without requiring code. The quick-start flow (pick template, test in dashboard, deploy) reinforces a no-code workflow. Missing for 10: independent/hands-on evidence of the visual builder's branching UI itself (community evidence only covers conversational behavior, not the builder), and no detail on how 'states' are represented/connected beyond the general node-based description.

   • [claimed-docs] “Retell agents are node-based flows or single prompts, with call handling, a knowledge base, and integrations built in.”
   • [claimed-docs] “Drag-and-drop, node-based flows for structured, high-stakes calls.”
   • [claimed-docs] “Single prompt agent... Define your whole agent with one prompt.”
   • [claimed-docs] “Function calling lets Retell single or multi-prompt agents take real actions — transfer calls, end calls, book appointments, send SMS, and c…”
   • [claimed-docs] “Function calling transforms your AI agent from a conversational interface into an action-oriented assistant.”
   • [claimed-docs] “Build your first Retell AI phone agent in 15 minutes: create an account, pick a template, test in the dashboard, deploy to a phone number, a…”
   • [claimed-docs] “click **Generate from prompt** (marked Suggested) to let [Conductor](/conductor/create-agent) draft an agent from a plain-English descriptio…”

   proof ↗⚑ flag

Personalization

1. developerInject dynamic variables and per-caller context at call time so each conversation is personalized

   weight 2 · round drawn
   Vapi—none0/10

   The evidence pack covers assistant creation, tools/webhooks, CLI, MCP server, voices, and testing, but contains no documentation of variable injection, assistantOverrides, or per-caller context personalization at call time. Missing for 10: docs on dynamic variable substitution (e.g., {{variableName}} templating), call-time overrides/metadata injection, and any example showing per-caller personalization.

   ⚑ flag

   Retell AI—none0/10

   The evidence pack covers function calling, webhooks, custom telephony, knowledge base retrieval, CRM field sync, and SDK/CLI/MCP tooling, but nowhere describes injecting dynamic variables or per-caller context (e.g., a startCall/create-call parameter for passing caller-specific data into the prompt/LLM at runtime). This is a fair, plausible capability for a voice-agent platform, so absence of evidence yields 'none' rather than 'na'.

   ⚑ flag

2. developerGround the agent on my documents with a built-in knowledge base or RAG so it answers from my content

   weight 2 · round to Retell AI
   Vapi—none0/10

   Vapi is a voice-agent platform focused on assistants, phone numbers, custom webhook tools, and telephony/testing; the evidence pack shows no built-in knowledge base or RAG feature for grounding assistants on uploaded documents. Custom tools/webhooks could be used to build a workaround, but no document ingestion or retrieval capability is documented.

   • [claimed-docs] “Create your own webhook-based tools to extend assistant capabilities”
   • [claimed-docs] “This guide shows you how to create custom tools, including Function Tools, for your Vapi assistants.”
   • [claimed-docs] “Custom tools that you create... interact with your systems via webhooks”

   proof ↗⚑ flag

   Retell AI~partialclaimed6/10

   Retell explicitly ships a knowledge base feature where you can crawl websites or upload documents for the agent to retrieve from, and AI QA scores 'knowledge base accuracy' as a call metric, confirming grounding is a first-class capability. However, evidence is thin — only brief homepage-level mentions rather than a dedicated deep-dive doc on KB architecture/RAG internals, and there's no independent/hands-on confirmation of retrieval quality. Missing for 10: dedicated knowledge-base documentation page detailing chunking/retrieval mechanics, and independent verification that answers are accurately grounded in uploaded content.

   • [claimed-docs] “Retell agents are node-based flows or single prompts, with call handling, a knowledge base, and integrations built in.”
   • [claimed-docs] “Crawl a website or upload documents your agent retrieves from.”
   • [claimed-docs] “AI QA scores Retell calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage to surface quality trends and issues…”

   proof ↗⚑ flag

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to Retell AI
   Vapi✓fullprobed8/10

   Vapi confirms an llms.txt file exists and is served at docs.vapi.ai/llms.txt (HTTP 200), explicitly instructing agents on how to fetch clean Markdown per page, which directly satisfies the story. missing for 10: no independent/community corroboration of agents actually consuming llms.txt in practice, only first-party probe evidence.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.vapi.ai/llms.txt # Vapi ## Instructions for AI Agents - For clean Markdown of any page, append `.…”

   proof ↗⚑ flag

   Retell AI✓fullprobed9/10

   A direct probe confirms an llms.txt file exists at docs.retellai.com/llms.txt returning HTTP 200 with a structured index of the docs, and Retell's docs are also agent-oriented enough to support MCP-based discovery/management of resources. This directly satisfies the story of pointing an agent at llms.txt or agent-oriented docs. Missing for 10: no independent third-party corroboration of llms.txt usage in the wild.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.retellai.com/llms.txt # Retell AI > Explore Retell AI docs to learn how to build, test, deploy, an…”
   • [claimed-docs] “Use Retell's MCP server to build and manage voice agents from MCP-capable clients like Cursor, Claude Desktop, and Claude Code via Retell AP…”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to Retell AI
   Vapi~partialprobed6/10

   Vapi ships an official CLI (vapi-docs-2/31/32, confirmed working keylessly in vapi-probe-rt-1) that can manage assistants, calls, and forward webhooks, which supports scripted/CI-style usage, and SDKs/REST APIs imply headless programmatic calls. However there is no explicit CI/automation documentation, no examples of running in a pipeline, and the MCP server requires bearer-key auth (vapi-probe-rt-2) which is unaddressed for CI contexts. missing for 10: explicit CI/automation examples or docs, non-interactive auth/service-account flow for CI, evidence of headless voice-testing/evals running in a pipeline.

   • [claimed-docs] “Manage assistants, phone numbers, and calls from your terminal”
   • [claimed-docs] “Build, test, and deploy voice AI applications without leaving your development environment.”
   • [claimed-docs] “The CLI auto-detects your tech stack and sets up everything you need.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bas…”
   • [claimed-docs] “Evals is Vapi's AI agent testing framework that enables you to systematically test assistants and squads using mock conversations with autom…”
   • [claimed-docs] “you'll create mock conversations, define expected behaviors, and validate your agents work correctly before production”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”

   proof ↗⚑ flag

   Retell AI~partialprobed7/10

   Retell ships official Node.js/Python SDKs with API-key auth and a terminal CLI for managing agents, phone numbers, and resources, both of which are scriptable outside the dashboard, and a runtime probe confirms the CLI installs and runs keylessly via npm — all consistent with headless/CI use. However, there is no explicit CI/automation documentation (e.g., GitHub Actions example, testing-in-pipeline guide) confirming an officially supported headless workflow. Missing for 10: explicit CI/pipeline documentation or example, confirmation of non-interactive auth flow for CI secrets, and independent evidence of real-world CI usage.

   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”
   • [claimed-docs] “The Retell CLI lets you manage Retell resources from your terminal.”
   • [claimed-docs] “Retell provides official SDKs for Node.js and Python to simplify integration with our platform.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”
   • [probe] “official CLI documented at https://docs.retellai.com/get-started/cli”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round to Retell AI
   Vapi—none0/10

   The evidence shows Vapi exposes its own APIs as an MCP server (so external MCP clients like Claude Desktop can control Vapi), but this is the opposite direction from the story — there's no evidence that Vapi assistants can consume/plug-in external MCP servers to use their tools within a conversation. Vapi's tool docs describe webhook-based custom tools and OpenAI-style function calling only, with no mention of MCP-server ingestion.

   • [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
   • [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
   • [claimed-docs] “Create your own webhook-based tools to extend assistant capabilities”
   • [claimed-docs] “Custom tools that you create... interact with your systems via webhooks”
   • [probe] “official MCP server documented at https://docs.vapi.ai/sdk/mcp-server”

   proof ↗⚑ flag

   Retell AI✓fullclaimed8/10

   Retell explicitly documents connecting a live voice/chat agent to a remote MCP server so it can invoke that server's tools during calls, directly matching the story of plugging in external MCP servers for tool use. This is a first-party, well-specified capability (not just Retell exposing its own API as MCP) covering both single- and multi-prompt agents. Missing for 10: independent/hands-on verification that a third-party MCP server's tools work reliably mid-call in production.

   • [claimed-docs] “Connect a Retell single- or multi-prompt agent to a remote MCP server so it can call the server's tools during a live voice or chat conversa…”
   • [claimed-docs] “Connect your single- or multi-prompt agent to a remote Model Context Protocol (MCP) server, and the agent can call that server's tools durin…”
   • [claimed-docs] “Connect your single- or multi-prompt agent to a remote [Model Context Protocol (MCP)] server, and the agent can call that server's tools dur…”
   • [claimed-docs] “Function calling transforms your AI agent from a conversational interface into an action-oriented assistant.”

   proof ↗⚑ flag

4. ai-native userConnect an agent via an official MCP server

   weight 3 · round drawn
   Vapi✓fullprobed9/10

   Vapi ships an official hosted MCP server (mcp.vapi.ai) exposing its assistant/phone/call APIs as MCP tools for any MCP-compatible agent (Claude Desktop, agent frameworks), documented in first-party docs and confirmed live via a runtime probe (401 bearer-key gate exactly as documented). Missing for 10: independent/community hands-on testimonials specifically about using the MCP server (community evidence only covers general demo/founder trivia, not MCP usage).

   • [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
   • [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
   • [probe] “official MCP server documented at https://docs.vapi.ai/sdk/mcp-server”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”

   proof ↗⚑ flag

   Retell AI✓fullprobed9/10

   Retell documents and hosts an official MCP server (mcp.retellai.com) that lets MCP-capable clients like Cursor, Claude Desktop, and Claude Code build and manage Retell voice agents via the Retell API, exposing meta-tools for listing/invoking endpoints. A runtime probe confirms the server completes a full keyless JSON-RPC initialize handshake and exposes its tool set, corroborating the vendor docs. Missing for 10: independent/community usage reports of the MCP server in practice beyond vendor docs and the probe.

   • [claimed-docs] “Use Retell's MCP server to build and manage voice agents from MCP-capable clients like Cursor, Claude Desktop, and Claude Code via Retell AP…”
   • [claimed-docs] “Retell supports the Model Context Protocol (MCP) so you can build Retell AI voice agents directly from MCP-capable clients (Cursor, Claude D…”
   • [claimed-docs] “Retell supports the [Model Context Protocol (MCP)] so you can build Retell AI voice agents directly from MCP-capable clients (Cursor, Claude…”
   • [claimed-docs] “Agents: create, update, publish, list, and fetch agent versions.”
   • [probe] “official MCP server documented at https://docs.retellai.com/get-started/mcp-server”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”

   proof ↗⚑ flag

5. ai-native userUse an official CLI

   weight 2 · round drawn
   Vapi✓fullprobed9/10

   Vapi ships a documented official CLI for managing assistants, phone numbers, and calls from the terminal, with webhook forwarding, org/environment switching, and tech-stack auto-detection, and this was independently verified in a runtime probe (successful install and `--version` output). missing for 10: no independent third-party review or community discussion of the CLI's day-to-day usage beyond the vendor docs and single install probe.

   • [claimed-docs] “Manage assistants, phone numbers, and calls from your terminal”
   • [claimed-docs] “vapi listen --forward-to localhost:3000/webhook”
   • [claimed-docs] “Switch between organizations and environments seamlessly”
   • [claimed-docs] “Forward webhooks to your local server for debugging”
   • [claimed-docs] “Build, test, and deploy voice AI applications without leaving your development environment.”
   • [claimed-docs] “The CLI auto-detects your tech stack and sets up everything you need.”
   • [probe] “official CLI documented at https://docs.vapi.ai/cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bas…”

   proof ↗⚑ flag

   Retell AI✓fullprobed9/10

   Retell ships an official CLI (`@retell-ai/retell-cli`) documented for managing agents, phone numbers, and knowledge bases from the terminal, and a runtime probe confirms it installs and runs keylessly via npx. Missing for 10: independent/community usage reports of the CLI itself (only docs and one probe run corroborate it).

   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”
   • [claimed-docs] “The Retell CLI lets you manage Retell resources from your terminal.”
   • [probe] “official CLI documented at https://docs.retellai.com/get-started/cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”

   proof ↗⚑ flag

6. ai-native userDrive the product through a documented public API

   weight 3 · round drawn
   Vapi✓fullprobed9/10

   Vapi ships a documented public API/SDK (assistant management, calls, tools), a CLI for terminal-driven workflows, and a hosted MCP server exposing the API as tools, all confirmed by runtime probes (CLI installs and runs, MCP endpoint live and auth-gated as documented). This directly satisfies programmatic/AI-native control via a documented public interface. Missing for 10: no independent third-party developer report deeply exercising the API beyond docs/probes.

   • [claimed-docs] “const assistant = await vapi.assistant”
   • [claimed-docs] “Manage assistants, phone numbers, and calls from your terminal”
   • [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
   • [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.vapi.ai/llms.txt # Vapi ## Instructions for AI Agents - For clean Markdown of any page, append `.…”
   • [probe] “official MCP server documented at https://docs.vapi.ai/sdk/mcp-server”
   • [probe] “official CLI documented at https://docs.vapi.ai/cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bas…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”

   proof ↗⚑ flag

   Retell AI✓fullprobed9/10

   Retell documents official Node.js/Python SDKs with typed clients and full endpoint coverage, a CLI for managing resources from the terminal, and API references (e.g., clone-voice endpoint) — and a runtime probe confirms the CLI installs/runs keylessly and the hosted MCP server exposes the entire API via meta-tools, proving the API is genuinely agent-drivable. Missing for 10: a publicly discoverable OpenAPI/swagger spec (all standard paths returned 404 in the probe).

   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”
   • [claimed-docs] “Retell provides official SDKs for Node.js and Python to simplify integration with our platform.”
   • [claimed-docs] “Type safety: Full TypeScript support with autocomplete ... Error handling: Structured error responses with detailed messages”
   • [probe] “official MCP server documented at https://docs.retellai.com/get-started/mcp-server”
   • [probe] “official CLI documented at https://docs.retellai.com/get-started/cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.retellai.com/openapi.json, https://docs.retellai.com/swagger.json, https://docs.retella…”

   proof ↗⚑ flag

7. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round to Vapi
   Vapi~partialprobed3/10

   Vapi's enterprise docs mention Role Based Access Control (RBAC) and SSO, implying some access-scoping capability at the org level, and its MCP server/API require bearer API keys — but there is no documented mechanism for issuing per-agent, least-privilege scoped API keys or tokens tailored to a specific agent's permissions. missing for 10: explicit scoped API key creation/management UI or API, documentation of key-level permission granularity, and independent confirmation that RBAC restricts agent credentials rather than just human dashboard users.

   • [claimed-docs] “Single Sign On (SSO) supported for Okta, Azure AD, SAML, and OIDC * Role Based Access Control (RBAC)”
   • [claimed-docs] “Single Sign On (SSO) supported for Okta, Azure AD, SAML, and OIDC”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”

   proof ↗⚑ flag

   Retell AI—none0/10

   The evidence pack documents API key auth, SDKs, CLI, and an MCP server, but nowhere describes scoped, role-based, or least-privilege API key/credential issuance (e.g., per-agent or per-permission keys) for Retell. This is a fair axis for an API/voice-agent platform, but no docs or probes show scoped credential support, so it is unproven.

   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Retell provides official SDKs for Node.js and Python to simplify integration with our platform.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”
   • [claimed-docs] “Use Retell's MCP server to build and manage voice agents from MCP-capable clients like Cursor, Claude Desktop, and Claude Code via Retell AP…”

   proof ↗⚑ flag

8. ai-native userBuild against official SDKs

   weight 2 · round to Retell AI
   Vapi✓fullprobed8/10

   Vapi documents an official JS/TS SDK usage pattern, official CLI (probe-confirmed installed and runnable), and official MCP server (probe-confirmed live and auth-gated), all clearly aimed at AI-native/agentic developer workflows including IDE assistant integration (Cursor/Windsurf/VSCode). missing for 10: independent third-party corroboration of SDK code quality and coverage across multiple languages beyond the docs snippets.

   • [claimed-docs] “const assistant = await vapi.assistant”
   • [claimed-docs] “Manage assistants, phone numbers, and calls from your terminal”
   • [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
   • [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
   • [claimed-docs] “Your IDE's AI assistant (Cursor, Windsurf, VSCode) gains complete, accurate knowledge of Vapi's APIs and best practices. No more hallucinate…”
   • [probe] “official MCP server documented at https://docs.vapi.ai/sdk/mcp-server”
   • [probe] “official CLI documented at https://docs.vapi.ai/cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bas…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”

   proof ↗⚑ flag

   Retell AI✓fullprobed9/10

   Retell documents and ships official typed SDKs for Node.js and Python with API key auth, full endpoint coverage, structured errors, and TypeScript autocomplete, plus an official CLI and MCP server for programmatic/agentic access — all independently confirmed by runtime probes (CLI installs and runs, MCP server completes handshake). Missing for 10: no independent third-party review of SDK quality/DX beyond vendor docs and no public OpenAPI spec discovered.

   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Retell provides official SDKs for Node.js and Python to simplify integration with our platform.”
   • [claimed-docs] “Type safety: Full TypeScript support with autocomplete ... Error handling: Structured error responses with detailed messages”
   • [probe] “official CLI documented at https://docs.retellai.com/get-started/cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”
   • [probe] “official MCP server documented at https://docs.retellai.com/get-started/mcp-server”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”

   proof ↗⚑ flag

9. ai-native userSubscribe to events via webhooks

   weight 2 · round to Retell AI
   Vapi✓fullclaimed7/10

   Vapi supports webhook-based events via Server URL (tool-calls messages), custom webhook tools, and CLI webhook forwarding for local debugging, giving AI-native users a documented event subscription mechanism. missing for 10: independent/hands-on confirmation of webhook delivery reliability, a full event-type catalog/schema, and signature/verification documentation.

   • [claimed-docs] “Create your own webhook-based tools to extend assistant capabilities”
   • [claimed-docs] “When tools are triggered, your Server URL receives a `tool-calls` message”
   • [claimed-docs] “Vapi supports OpenAI-style tool/function calling. Assistants can ping your server to perform actions.”
   • [claimed-docs] “Forward webhooks to your local server for debugging”
   • [claimed-docs] “Custom tools that you create... interact with your systems via webhooks”

   proof ↗⚑ flag

   Retell AI✓fullclaimed8/10

   Retell has dedicated first-party documentation for webhooks describing real-time event notifications pushed to your application as events occur, enabling event-driven integrations rather than polling — directly matching the story of subscribing to events via webhooks. Missing for 10: no independent/hands-on corroboration of webhook reliability or event catalog completeness beyond the docs themselves.

   • [claimed-docs] “Webhooks allow your application to receive real-time notifications about events that occur in your Retell AI account.”
   • [claimed-docs] “webhooks push data to your application as events happen, making your integrations more efficient and responsive.”

   proof ↗⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round to Retell AI
   Vapi—none0/10

   Evidence covers building/testing assistants (Evals, Voice Test Suites), CLI/MCP management tools, and tool-calling, but nothing describes the product itself surfacing AI-generated insights or suggestions derived from a user's own call/usage data (e.g., analytics dashboards with AI-generated recommendations).

   ⚑ flag

   Retell AI~partialclaimed6/10

   Retell's AI QA feature automatically scores calls on hallucination, knowledge-base accuracy, latency, sentiment, and tool usage to surface quality trends and issues, and custom analytics dashboards let users chart/filter call and chat metrics — both are AI-derived insights from the product's own call data. However, this is scoring/quality analytics rather than proactive generative 'suggestions' (e.g., recommended actions, next-best-response, or coaching tips), and there's no independent/hands-on evidence validating the accuracy or usefulness of these AI-generated insights. Missing for 10: evidence of proactive suggestion/recommendation generation beyond scoring, and third-party corroboration of AI QA insight quality.

   • [claimed-docs] “AI QA scores Retell calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage to surface quality trends and issues…”
   • [claimed-docs] “AI QA automatically evaluates a sampled set of your calls against rules and metrics you configure.”
   • [claimed-docs] “Build custom Retell analytics dashboards to track call and chat metrics like success rate, latency, cost, and concurrency, with charts, filt…”
   • [claimed-docs] “The Analytics dashboard charts your call and chat data so you can see how your agents are performing over time.”
   • [claimed-docs] “Built-in call success and sentiment scoring, plus custom fields synced to your CRM.”

   proof ↗⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Retell AI
   Vapi~partialclaimed4/10

   Vapi assistants operate autonomously once a call starts—handling conversation flow, invoking custom tools via webhooks, and triggering server-side automations without further human input (vapi-docs-6, vapi-docs-17, vapi-docs-23, vapi-docs-34). However, this autonomy is scoped to an active call session; there is no evidence of scheduled/cron-style background jobs or agent loops that run independently of a triggered call or user interaction. Missing for 10: evidence of scheduled/background triggers outside live calls, persistent autonomous task queues, or proactive (non-call-triggered) automation runs.

   • [claimed-docs] “Create your own webhook-based tools to extend assistant capabilities”
   • [claimed-docs] “When tools are triggered, your Server URL receives a `tool-calls` message”
   • [claimed-docs] “Vapi supports OpenAI-style tool/function calling. Assistants can ping your server to perform actions.”
   • [claimed-docs] “Custom tools that you create... interact with your systems via webhooks”
   • [claimed-docs] “This guide shows you how to create custom tools, including Function Tools, for your Vapi assistants.”

   proof ↗⚑ flag

   Retell AI~partialcommunity6/10

   Retell agents act autonomously during live calls — function calling lets them independently book appointments, send SMS, transfer calls, and call external APIs without human intervention (retell-docs-5, retell-docs-35), and webhooks push real-time events to downstream systems (retell-docs-12, retell-docs-24). This shows in-call autonomous action-taking, which is a form of background automation once a call is triggered. However, there's no evidence of standalone scheduled/triggered automations running independent of a live voice/chat session (e.g., cron-like outbound campaigns or autonomous multi-step workflows outside conversation context). Missing for 10: evidence of scheduled or event-triggered background automations outside live calls, and independent/hands-on confirmation that autonomous function-calling reliably completes tasks unattended (community reports show some confusion/looping during autonomous task execution, retell-comm-1, retell-comm-3, retell-comm-5).

   • [claimed-docs] “Function calling lets Retell single or multi-prompt agents take real actions — transfer calls, end calls, book appointments, send SMS, and c…”
   • [claimed-docs] “Function calling transforms your AI agent from a conversational interface into an action-oriented assistant.”
   • [claimed-docs] “Webhooks allow your application to receive real-time notifications about events that occur in your Retell AI account.”
   • [claimed-docs] “webhooks push data to your application as events happen, making your integrations more efficient and responsive.”
   • [community] “The AI contradicted itself when layering conditionals - it got confused about morning vs afternoon time, kept asking to repeat despite corre…”
   • [community] “Voice sounds great, but: told it unavailable until next year, it confirmed Feb 4th 'next year' but thought current year was 2022. Also got s…”

   proof ↗⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round to Retell AI
   Vapi—none0/10

   Vapi's docs describe building and configuring voice AI assistants for end-users (phone/web calls, tools, testing) and even an MCP server/CLI for managing those assistants, but there is no evidence of a built-in AI assistant/copilot inside the Vapi product itself that the AI-native user can delegate platform tasks to (e.g., an in-dashboard copilot that configures assistants or writes tools for you). The CLI mentions external IDE assistants (Cursor/Windsurf) gaining API knowledge, but that's a third-party tool, not a built-in in-product assistant.

   • [claimed-docs] “Your IDE's AI assistant (Cursor, Windsurf, VSCode) gains complete, accurate knowledge of Vapi's APIs and best practices. No more hallucinate…”
   • [claimed-docs] “Manage assistants, phone numbers, and calls from your terminal”
   • [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”

   proof ↗⚑ flag

   Retell AI~partialclaimed3/10

   Retell's dashboard includes 'Conductor', a built-in AI assistant that can draft an entire agent from a plain-English prompt (docs-30), which is a genuine instance of delegating a task to an in-product AI assistant. However this is a single thin mention with no further detail on scope, limits, or other delegable tasks beyond initial agent creation. Missing for 10: documentation of Conductor's full capabilities/limits, evidence of delegating other tasks (not just agent drafting) to a built-in assistant, and independent/hands-on corroboration of Conductor actually working.

   • [claimed-docs] “click **Generate from prompt** (marked Suggested) to let [Conductor](/conductor/create-agent) draft an agent from a plain-English descriptio…”

   proof ↗⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round drawn
   Vapi✓fullprobed7/10

   Vapi exposes an official MCP server that lets any MCP-compatible AI assistant (e.g. Claude Desktop) manage assistants, phone numbers, and calls via natural-language tool calls, and this endpoint is confirmed live and auth-gated in a runtime probe. It also ships a CLI and llms.txt docs optimized for AI agents to operate it. Missing for 10: no first-party evidence of a built-in chat/NL command console inside the Vapi product itself (beyond MCP/CLI proxies), and no independent hands-on report of an agent successfully performing multi-step tasks via MCP.

   • [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
   • [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
   • [claimed-docs] “Your IDE's AI assistant (Cursor, Windsurf, VSCode) gains complete, accurate knowledge of Vapi's APIs and best practices. No more hallucinate…”
   • [probe] “official MCP server documented at https://docs.vapi.ai/sdk/mcp-server”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bas…”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.vapi.ai/llms.txt # Vapi ## Instructions for AI Agents - For clean Markdown of any page, append `.…”

   proof ↗⚑ flag

   Retell AI✓fullprobed7/10

   Retell supports natural-language operation via Conductor's 'Generate from prompt' feature that drafts an agent from a plain-English description, and via its official MCP server which exposes the full API as agent-callable tools so AI-native clients (Cursor, Claude Desktop, Claude Code) can build/manage agents through natural-language MCP tool calls — confirmed live by a keyless JSON-RPC probe against the hosted MCP endpoint. missing for 10: independent/hands-on evaluation of Conductor's prompt-to-agent quality, and no evidence of NL commands for other everyday operations (e.g., dashboard chat-based control) beyond agent creation/management.

   • [claimed-docs] “click **Generate from prompt** (marked Suggested) to let [Conductor](/conductor/create-agent) draft an agent from a plain-English descriptio…”
   • [claimed-docs] “Use Retell's MCP server to build and manage voice agents from MCP-capable clients like Cursor, Claude Desktop, and Claude Code via Retell AP…”
   • [claimed-docs] “Retell supports the Model Context Protocol (MCP) so you can build Retell AI voice agents directly from MCP-capable clients (Cursor, Claude D…”
   • [claimed-docs] “Retell supports the [Model Context Protocol (MCP)] so you can build Retell AI voice agents directly from MCP-capable clients (Cursor, Claude…”
   • [claimed-docs] “Agents: create, update, publish, list, and fetch agent versions.”
   • [probe] “official MCP server documented at https://docs.retellai.com/get-started/mcp-server”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   Vapi—none0/10

   The evidence pack shows static markdown documentation, code snippets, a CLI, and an MCP server, but no interactive API reference or runnable/try-it-out examples are described anywhere in the docs pages cited.

   ⚑ flag

   Retell AI—none0/10

   Evidence shows Retell has API reference doc pages (e.g., api-references/clone-voice.md) and SDKs/CLI/MCP tooling, but no evidence of an interactive, runnable API reference (e.g., embedded 'try it' console or Swagger UI). The probe explicitly found no OpenAPI/Swagger spec exposed (all candidate paths 404), undermining any claim of an interactive reference.

   • [claimed-docs] “Clone a voice from audio files”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.retellai.com/openapi.json, https://docs.retellai.com/swagger.json, https://docs.retella…”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   Vapi—none0/10

   The evidence pack covers Vapi's CLI, MCP server, docs, and SDKs, but no citation mentions an OpenAPI spec, API reference schema, or any machine-readable spec file available for download. This is a fair axis for an API-first product, but no supporting evidence exists in the pack.

   ⚑ flag

   Retell AI—none0/10

   A direct probe for standard OpenAPI/Swagger spec locations (openapi.json, swagger.json, etc.) on Retell's docs domain returned 404 for all candidates, and no evidence pack item shows a downloadable, machine-readable API spec being published or linked from docs. The CLI's version string mentions an internal 'OpenAPI 3.0.0' schema, but this is not shown to be an artifact users can download or fetch programmatically.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.retellai.com/openapi.json, https://docs.retellai.com/swagger.json, https://docs.retella…”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round to Vapi
   Vapi✓fullclaimed7/10

   Vapi offers dedicated Voice Test Suites (AI tester simulating conversations) and an Evals framework explicitly for creating mock conversations to 'validate your agents work correctly before production,' plus CLI webhook forwarding to localhost for local development/debugging — all separate from live production calls. Missing for 10: explicit vendor use of the term 'sandbox environment' and independent/hands-on verification that test runs are fully isolated from production data.

   • [claimed-docs] “Voice Test Suites enable you to test your AI voice agents through simulated phone conversations.”
   • [claimed-docs] “Evals is Vapi's AI agent testing framework that enables you to systematically test assistants and squads using mock conversations with autom…”
   • [claimed-docs] “Our AI tester calls your voice agent and follows a script that simulates real customer behavior.”
   • [claimed-docs] “you'll create mock conversations, define expected behaviors, and validate your agents work correctly before production”
   • [claimed-docs] “Forward webhooks to your local server for debugging”
   • [claimed-docs] “Build, test, and deploy voice AI applications without leaving your development environment.”

   proof ↗⚑ flag

   Retell AI~partialclaimed4/10

   Retell's quick-start lets you click 'Test' to run a free web call with your agent before it ever touches a phone number or payment method, and new accounts get $10 trial credit to experiment risk-free — a lightweight way to try an agent without production telephony traffic. However, there's no documented dedicated sandbox environment, test-mode API flag, or explicit data isolation guarantee separating test calls from production records/analytics. Missing for 10: explicit sandbox/staging environment concept, test vs prod data isolation guarantees, and independent confirmation that test-call data doesn't mix with production analytics/QA.

   • [claimed-docs] “Click the "Test" button to start a web call with your agent... This step is free and doesn't need a phone number or payment method.”
   • [claimed-docs] “Click the "Test" button to start a web call with your agent”
   • [claimed-docs] “New accounts start with \$10 in free trial credits, no payment method required.”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Vapi—none0/10

   No evidence pack item mentions API versioning scheme, version numbers in endpoints, or any documented deprecation/sunset policy for Vapi's APIs; the docs cover features (CLI, MCP, tools, voices) but not API lifecycle governance.

   ⚑ flag

   Retell AI—none0/10

   The evidence pack covers SDKs, CLI, MCP server, and API endpoints extensively, but there is no documentation of API versioning scheme or a deprecation policy for breaking changes. The probe notes an OpenAPI version string (3.0.0, catalog v4) but this is not evidence of a documented deprecation/versioning policy for API consumers.

   ⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Retell AI
   Vapi—none0/10

   The evidence pack documents CLI/SDK/MCP management of individual assistants, phone numbers, and calls, but nowhere shows bulk/batch endpoints or commands (e.g., batch-create, bulk-update, mass-delete) for operating across many items at once.

   ⚑ flag

   Retell AI~partialprobed4/10

   Retell exposes a full-coverage SDK, CLI, and MCP server that let an AI agent script repeated single-item operations (create/update/list agents, phone numbers, knowledge bases) programmatically, enabling scripted bulk-like loops, but there is no documented native batch/bulk endpoint operating on many items in a single call. missing for 10: explicit batch/bulk API endpoints or CLI commands operating on multiple items in one call, evidence of rate-limit-safe bulk workflows, and hands-on confirmation of bulk usage at scale

   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”
   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Use Retell's MCP server to build and manage voice agents from MCP-capable clients like Cursor, Claude Desktop, and Claude Code via Retell AP…”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”
   • [claimed-docs] “Agents: create, update, publish, list, and fetch agent versions.”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to Retell AI
   Vapi~partialclaimed5/10

   Vapi supports event-driven server-side tools/webhooks that fire on call events (tool-calls messages, server-url events) which act as a rule-trigger mechanism, but this is scoped to voice-call events only rather than a general-purpose automation/rules engine for arbitrary triggers. missing for 10: a documented general condition/trigger-action rules engine spanning non-call events, cross-system automation, or independent hands-on evidence of complex conditional automation chains.

   • [claimed-docs] “When tools are triggered, your Server URL receives a `tool-calls` message”
   • [claimed-docs] “Vapi supports OpenAI-style tool/function calling. Assistants can ping your server to perform actions.”
   • [claimed-docs] “Custom tools that you create... interact with your systems via webhooks”
   • [claimed-docs] “Create your own webhook-based tools to extend assistant capabilities”

   proof ↗⚑ flag

   Retell AI~partialclaimed6/10

   Retell supports event-driven automation via webhooks that fire on account events (retell-docs-12, retell-docs-24), function calling that lets agents automatically transfer calls, end calls, book appointments, or call external APIs based on conversation logic (retell-docs-5, retell-docs-35), and node-based flows with conditional branching (retell-docs-25) plus AI QA rules evaluated against configured metrics (retell-docs-43). However, this is scattered across call-flow logic and webhook notifications rather than a unified 'if event then action' rule-definition interface. Missing for 10: a dedicated rules/trigger engine UI for arbitrary account-wide events, documentation of webhook-to-action automation chains, and independent verification that rule-based automation works reliably (community reports note conversational logic errors, e.g. retell-comm-1, retell-comm-3, retell-comm-7).

   • [claimed-docs] “Function calling lets Retell single or multi-prompt agents take real actions — transfer calls, end calls, book appointments, send SMS, and c…”
   • [claimed-docs] “Webhooks allow your application to receive real-time notifications about events that occur in your Retell AI account.”
   • [claimed-docs] “webhooks push data to your application as events happen, making your integrations more efficient and responsive.”
   • [claimed-docs] “Drag-and-drop, node-based flows for structured, high-stakes calls.”
   • [claimed-docs] “Function calling transforms your AI agent from a conversational interface into an action-oriented assistant.”
   • [claimed-docs] “AI QA automatically evaluates a sampled set of your calls against rules and metrics you configure.”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round drawn
   Vapi—none0/10

   No evidence in the pack shows Vapi supports scheduling recurring jobs, workflows, or automated recurring calls/tasks — the docs cover assistants, tools, CLI, MCP server, testing, and voice customization but nothing about cron-like scheduling or recurring automation triggers.

   ⚑ flag

   Retell AI—none0/10

   Retell's docs cover webhooks, function calling, MCP tool-calling, and telephony integration, but nothing describes scheduling recurring jobs, campaigns, or workflows (e.g., cron-like recurring outbound call batches or repeating automations). This is a fair question for a voice-agent platform (buyers often want scheduled/recurring outbound campaigns), so the axis applies, but no evidence in the pack shows this capability. missing for 10: any mention of a scheduler, recurring campaign/job feature, or cron-style automation trigger.

   ⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round to Retell AI
   Vapi—none0/10

   Vapi's docs cover assistant creation, tools, CLI, MCP server, and testing/evals, but there is no mention of versioning assistant configurations, review/approval workflows, or rollback to prior versions of automations. No evidence pack item addresses version history or rollback capability.

   ⚑ flag

   Retell AI~partialclaimed4/10

   Retell supports 'agent versions' with create/update/publish/list/fetch operations via CLI and MCP (retell-docs-47), implying some versioning and possibly rollback via republishing an older version, but there is no documented review workflow (diffs, approvals, changelogs) or explicit rollback command/UI. missing for 10: explicit rollback mechanism, version diff/review UI, changelog or audit trail, and any independent/hands-on confirmation that version history and rollback actually work as described.

   • [claimed-docs] “Agents: create, update, publish, list, and fetch agent versions.”
   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”
   • [claimed-docs] “Use Retell's MCP server to build and manage voice agents from MCP-capable clients like Cursor, Claude Desktop, and Claude Code via Retell AP…”

   proof ↗⚑ flag

Compliance

1. founderMeet call-recording consent and disclosure obligations with per-call recording controls and configurable data retention

   weight 2 · round to Retell AI
   Vapi—none0/10

   The evidence pack covers HIPAA toggle, SSO/RBAC, testing/evals, and CLI/MCP tooling, but contains no mention of call-recording enable/disable controls, consent disclosure features, or configurable data retention policies for call recordings — a fair and applicable axis for a voice-calling platform.

   ⚑ flag

   Retell AI~partialclaimed5/10

   Retell documents per-agent configurable data retention that auto-deletes transcripts, recordings, and logs after a set period, directly addressing retention obligations. However, there is no evidence of explicit per-call recording enable/disable controls or built-in consent-disclosure features (e.g., automated recording announcements) that founders would need for two-party consent compliance. Missing for 10: per-call recording on/off toggle, consent-disclosure/announcement mechanism, and any documentation tying these to legal consent requirements.

   • [claimed-docs] “Configure per-agent data retention to automatically delete call and chat data — transcripts, recordings, and logs — after a set period for c…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent. After the retention period expires, call and chat data associated with tha…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent.”

   proof ↗⚑ flag

2. platform-engineerRun regulated workloads with HIPAA/BAA support, SOC 2, and data-residency options

   weight 2 · round to Vapi
   Vapi~partialclaimed5/10

   Vapi documents HIPAA compliance toggle in the dashboard and enterprise SSO/RBAC, indicating some regulated-workload support, but evidence lacks explicit SOC 2 certification details, BAA execution process, or data-residency region options. missing for 10: SOC 2 report/certification evidence, explicit BAA process details, data-residency/region selection options, independent audit confirmation.

   • [claimed-docs] “HIPAA compliance can be ensured by turning on HIPAA compliance in the Dashboard.”
   • [claimed-docs] “Single Sign On (SSO) supported for Okta, Azure AD, SAML, and OIDC * Role Based Access Control (RBAC)”
   • [claimed-docs] “Single Sign On (SSO) supported for Okta, Azure AD, SAML, and OIDC”

   proof ↗⚑ flag

   Retell AI—none0/10

   Evidence shows only generic data-retention controls (docs-15/27/44) but no mention of HIPAA, BAA, SOC 2 certification, or data-residency options anywhere in the docs or community pack, despite this being a fair compliance axis for a voice-AI platform handling call recordings/transcripts.

   • [claimed-docs] “Configure per-agent data retention to automatically delete call and chat data — transcripts, recordings, and logs — after a set period for c…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent. After the retention period expires, call and chat data associated with tha…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent.”

   proof ↗⚑ flag

Scale

1. platform-engineerSee documented concurrency limits and scale to many simultaneous calls without manual capacity begging

   weight 2 · round to Retell AI
   Vapi—none0/10

   No evidence pack items document concurrency limits, per-org call caps, autoscaling behavior, or capacity-request processes; the docs cover CLI, MCP server, tools, voices, and enterprise SSO but nothing about simultaneous-call scaling limits or how to raise them.

   ⚑ flag

   Retell AI~partialclaimed5/10

   Retell documents a concurrency configuration feature (reserved inbound concurrency vs. outbound limit) showing platform-level concurrency controls exist, but there is no documented default/numeric concurrency limits per plan, no described self-service process to raise limits without contacting support, and no evidence (docs or community) confirming scaling to many simultaneous calls happens without manual requests. missing for 10: published default concurrency limits, self-service limit-increase mechanism, and independent confirmation of large-scale concurrent call handling without support intervention.

   • [claimed-docs] “Reserved inbound concurrency protects inbound calls from being crowded out by outbound traffic.”
   • [claimed-docs] “When `reserved_inbound_concurrency` is configured, outbound calls can use at most your concurrency limit minus the reserved amount.”

   proof ↗⚑ flag

Self host

1. platform-engineerSelf-host the voice agent runtime from open-source code on my own infrastructure

   weight 3 · round drawn
   Vapi—none0/10

   Vapi is presented entirely as a hosted SaaS platform with a CLI, MCP server, and dashboard for managing cloud-hosted assistants; there is no evidence of open-source runtime code or any self-hosting option. One community note even references that a different product (pipecat by daily.co) is the open-source alternative, implicitly contrasting with Vapi's closed hosted model.

   • [community] “Vapi is also built on media framework by daily.co. They have an open source version of voice ai called pipecat.”
   • [claimed-docs] “Build, test, and deploy voice AI applications without leaving your development environment.”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”

   proof ↗⚑ flag

   Retell AI—none0/10

   Retell AI is a hosted SaaS voice-agent platform; all evidence points to cloud-hosted APIs, SDKs, CLI, and MCP server that call Retell's own servers, with no mention of open-source runtime code or self-hosting instructions. There is no evidence a platform-engineer can download and run the voice agent runtime on their own infrastructure.

   • [claimed-docs] “Build your first Retell AI phone agent in 15 minutes: create an account, pick a template, test in the dashboard, deploy to a phone number, a…”
   • [claimed-docs] “Retell starts at $0, pay only for what you use.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”

   proof ↗⚑ flag

Latency

1. platform-engineerSee documented end-to-end voice latency numbers or tuning guidance backing the platform's speed claims

   weight 3 · round to Retell AI
   Vapi~partialclaimed3/10

   Vapi's docs mention that 'Ultra Fast' vs 'High Intelligence' model settings trade off latency, giving conceptual tuning guidance, but there are no documented end-to-end latency numbers (e.g., ms benchmarks) or detailed tuning playbooks in the evidence pack. Missing for 10: concrete latency benchmark figures, component-level latency breakdowns (STT/LLM/TTS), and independent verification of speed claims.

   • [claimed-docs] “Ultra Fast uses speed-optimized models for lower latency. High Intelligence accepts higher latency for greater capability.”

   proof ↗⚑ flag

   Retell AI~partialclaimed5/10

   Retell docs offer tuning guidance for latency (transcription mode trade-offs, endpointing settings) and let customers monitor latency via analytics dashboards and AI QA scoring, but there are no documented end-to-end latency benchmarks or numbers substantiating speed claims. missing for 10: published end-to-end latency figures/benchmarks, and independent verification of speed claims.

   • [claimed-docs] “Pick the right Retell transcription mode to balance accuracy against latency — interim results for speed or context-aware results for higher…”
   • [claimed-docs] “Real time transcription is often a trade off between latency and accuracy.”
   • [claimed-docs] “optimize for speed: uses the latest interim results with a low endpointing setting... optimize for accuracy: uses the results with a higher …”
   • [claimed-docs] “Build custom Retell analytics dashboards to track call and chat metrics like success rate, latency, cost, and concurrency, with charts, filt…”
   • [claimed-docs] “The Analytics dashboard charts your call and chat data so you can see how your agents are performing over time.”
   • [claimed-docs] “AI QA scores Retell calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage to surface quality trends and issues…”
   • [claimed-docs] “AI QA automatically evaluates a sampled set of your calls against rules and metrics you configure.”

   proof ↗⚑ flag

Turn taking

1. developerRely on the agent to handle interruptions (barge-in) gracefully — stopping speech, updating context, and recovering the turn

   weight 3 · round drawn
   Vapi—none0/10

   The evidence pack covers CLI, MCP server, tools, voices, compliance, and testing, but contains no documentation or evidence about interruption handling, barge-in behavior, context updates during interruptions, or turn recovery mechanics. This is a core latency/turn-taking capability for voice agents, but nothing in the pack addresses it directly.

   ⚑ flag

   Retell AI—none0/10

   The evidence pack covers transcription latency/accuracy trade-offs (retell-docs-9, retell-docs-39, retell-docs-48) but nowhere describes explicit barge-in/interruption handling — stopping agent speech, updating context, and recovering the turn when a caller interrupts. Community threads show turn-taking confusion (contradictions, loops, disconnects) but do not address interruption handling specifically. Missing for full/partial credit: any documentation or hands-on report confirming barge-in detection, speech-stop behavior, or turn recovery mechanics.

   ⚑ flag

2. developerEnable noise suppression or audio filtering so the agent stays coherent on noisy real-world calls

   weight 1 · round drawn
   Vapi—none0/10

   No evidence pack items mention noise suppression, background noise filtering, or audio denoising features for handling noisy real-world calls; the pack covers voices, tools, CLI, MCP, latency modes, and testing but nothing about audio filtering.

   ⚑ flag

   Retell AI—none0/10

   No evidence pack item mentions noise suppression, background-noise filtering, or audio-preprocessing features for handling noisy real-world call environments; the closest related feature (transcription-mode trade-off between latency and accuracy) addresses turn-taking speed, not noise robustness. This is a plausible axis for a voice-agent platform, so absence of evidence yields 'none' rather than 'na'.

   ⚑ flag

3. developerUse model-based end-of-turn detection beyond simple VAD silence timeouts so the agent doesn't talk over slow speakers

   weight 2 · round drawn
   Vapi—none0/10

   The evidence pack lacks any mention of model-based end-of-turn detection, turn-taking models, or semantic VAD alternatives; only latency tuning between 'Ultra Fast' and 'High Intelligence' model modes is documented, which does not address end-of-turn detection beyond silence-based VAD. Missing for 10: any documentation of a model-based or semantic end-of-turn detector, comparison to VAD-only timeout behavior, or configuration options for turn-taking sensitivity.

   • [claimed-docs] “Ultra Fast uses speed-optimized models for lower latency. High Intelligence accepts higher latency for greater capability.”

   proof ↗⚑ flag

   Retell AI—none0/10

   The transcription-mode docs only describe tuning an 'endpointing setting' (low vs high) as a speed/accuracy tradeoff for VAD-style silence detection, not a distinct model-based semantic end-of-turn detector that understands incomplete utterances from slow speakers. No evidence pack item describes a dedicated turn-detection model or algorithm beyond adjusting silence-timeout thresholds.

   • [claimed-docs] “Pick the right Retell transcription mode to balance accuracy against latency — interim results for speed or context-aware results for higher…”
   • [claimed-docs] “Real time transcription is often a trade off between latency and accuracy.”
   • [claimed-docs] “optimize for speed: uses the latest interim results with a low endpointing setting... optimize for accuracy: uses the results with a higher …”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round drawn
   Vapi✓fullprobed8/10

   Vapi's API/SDK covers assistant, phone number, and call management, backed by a full CLI (vapi-docs-2 through 4, 27, 31-32, verified live in vapi-probe-rt-1) and an official MCP server exposing the same operations for external agents (vapi-docs-5,33, confirmed live and auth-gated in vapi-probe-rt-2). Dashboard-only conveniences like Voice Library browsing/preview remain UI-centric, though voice IDs and custom voices can be set via API (vapi-docs-9,24,26,35). Missing for 10: explicit confirmation that every dashboard-only feature (e.g., visual Voice Library browsing, HIPAA toggle) has a full API equivalent, and independent (non-vendor) verification of full UI/API parity.

   • [claimed-docs] “Manage assistants, phone numbers, and calls from your terminal”
   • [claimed-docs] “vapi listen --forward-to localhost:3000/webhook”
   • [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
   • [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
   • [claimed-docs] “Build, test, and deploy voice AI applications without leaving your development environment.”
   • [claimed-docs] “The CLI auto-detects your tech stack and sets up everything you need.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bas…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”
   • [claimed-docs] “You can use your own custom voice with any supported provider by setting the `voice` property in your assistant configuration”
   • [claimed-docs] “Browse and preview voices there, then copy a voice's ID to use on an assistant.”

   proof ↗⚑ flag

   Retell AI✓fullprobed8/10

   Retell exposes full API coverage via official Node/Python SDKs, a CLI, and a hosted MCP server that dynamically exposes the entire API (list/get/invoke endpoint tools), confirmed by keyless runtime probes actually reaching the CLI and MCP handshake — meaning nearly anything doable in the dashboard (agents, phone numbers, knowledge bases, function calling, voice cloning, analytics) is API/CLI/MCP accessible. missing for 10: no public OpenAPI spec was found (404s across candidate paths), and no independent hands-on confirmation that every single UI feature (e.g., analytics dashboards, AI QA config) has a 1:1 API equivalent.

   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”
   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Use Retell's MCP server to build and manage voice agents from MCP-capable clients like Cursor, Claude Desktop, and Claude Code via Retell AP…”
   • [claimed-docs] “Retell supports the Model Context Protocol (MCP) so you can build Retell AI voice agents directly from MCP-capable clients (Cursor, Claude D…”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.retellai.com/openapi.json, https://docs.retellai.com/swagger.json, https://docs.retella…”
   • [claimed-docs] “Agents: create, update, publish, list, and fetch agent versions.”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round drawn
   Vapi—none0/10

   No evidence of a data export feature, open-format data dumps, or account portability/deletion tooling; the docs cover CLI, MCP server, tools, and voice features but nothing about exporting assistants/call data/transcripts in open formats for user exit. Missing for 10: documented export functionality, open data formats supported, and any account/data portability guarantees.

   ⚑ flag

   Retell AI—none0/10

   Docs describe API/SDK/CLI access, data retention (auto-deletion) and analytics dashboards, but nowhere document a data-export or portability feature that would let a user pull all call/chat data, transcripts, and recordings in an open format to leave the platform; retention docs only cover automatic deletion, not export.

   • [claimed-docs] “Configure per-agent data retention to automatically delete call and chat data — transcripts, recordings, and logs — after a set period for c…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent. After the retention period expires, call and chat data associated with tha…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent.”
   • [claimed-docs] “Retell provides official SDKs for Node.js and Python to simplify integration with our platform.”
   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”

   proof ↗⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round drawn
   Vapi—none0/10

   Vapi is a closed, commercial SaaS platform; nothing in the evidence indicates its core source (server, dashboard, or model runtime) is published under an open license—only SDKs/CLI tooling and docs are mentioned, and the one open-source reference (pipecat) is a third-party project, not Vapi itself.

   • [community] “Vapi is also built on media framework by daily.co. They have an open source version of voice ai called pipecat.”

   proof ↗⚑ flag

   Retell AI—none0/10

   Retell AI is a closed, proprietary SaaS platform; the evidence pack shows SDKs, CLI, MCP server and docs but no mention of source code being published under any open license, nor any GitHub repo for the core platform. Missing for 10: any open-source license grant, public source repository, or licensing terms for the core agent/voice engine.

   ⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round drawn
   Vapi—none0/10

   Vapi is presented exclusively as a hosted cloud platform (dashboard, hosted MCP server, hosted API/CLI against cloud endpoints); nothing in the evidence pack mentions a self-hostable core engine, open-source repo for the core product, or on-prem deployment option (the mentioned pipecat is a different open-source project, not Vapi itself). This is a fair axis for a voice-AI platform, but no evidence supports self-hosting.

   • [community] “Vapi is also built on media framework by daily.co. They have an open source version of voice ai called pipecat.”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”
   • [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”

   proof ↗⚑ flag

   Retell AI—none0/10

   Retell AI is presented entirely as a hosted SaaS (cloud dashboard, hosted APIs, hosted MCP server, usage-based pricing) with no mention anywhere in docs of a self-hosted or on-prem deployment option; all evidence points to a fully managed cloud product.

   • [claimed-docs] “Retell starts at $0, pay only for what you use.”
   • [claimed-docs] “Build your first Retell AI phone agent in 15 minutes: create an account, pick a template, test in the dashboard, deploy to a phone number, a…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”
   • [claimed-docs] “Integrate Retell voice agents with your own telephony provider using elastic SIP trunking or imported numbers from Twilio, Telnyx, and Vonag…”

   proof ↗⚑ flag

Pricing

1. founderSee published per-minute or usage pricing and estimate cost per call before committing

   weight 2 · round to Retell AI
   Vapi—none0/10

   The evidence pack contains no mention of pricing pages, per-minute rates, usage-based cost breakdowns, or any pricing calculator/estimator; all evidence is about docs, CLI, MCP server, tools, and testing features. Absence of evidence for this applicable axis yields none.

   ⚑ flag

   Retell AI✓fullcommunity8/10

   Retell's pricing page explicitly states pay-as-you-go pricing starting at $0 with no annual contracts, offers an interactive calculator to estimate approximate monthly cost by adjusting key parameters, and new accounts get $10 free trial credit to test before paying — directly enabling a founder to estimate call costs before committing. Community discussion also confirms a concrete per-minute rate (17¢/min) is being charged in practice, corroborating usage-based billing exists. missing for 10: the per-minute rate itself is not shown in first-party docs (only inferred from a community comment), and there's no independent verification of the calculator's accuracy.

   • [claimed-docs] “Retell starts at $0, pay only for what you use.”
   • [claimed-docs] “Adjust key parameters to see an approximate monthly cost for your Retell AI voice agents.”
   • [claimed-docs] “Most voice AI platforms require annual contracts before you write a single line of code. Retell starts at $0, pay only for what you use.”
   • [claimed-docs] “New accounts start with \$10 in free trial credits, no payment method required.”
   • [community] “Founder (yanyan_evie) noted voice provider costs differ: 'The different providers have different prices. openai tts & deepgram are cheaper, …”
   • [community] “A commenter noted per-minute pricing (17¢/min) could be exploited: a malicious caller could keep the bot on the line for hours across many p…”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round drawn
   Vapi—none0/10

   The evidence pack contains no mention of data residency, regional storage options, or geographic data controls anywhere in Vapi's docs (only HIPAA toggle and SSO/RBAC are mentioned as compliance features). Missing for 10: any documentation of region selection, data residency guarantees, or storage location controls.

   ⚑ flag

   Retell AI—none0/10

   No evidence in the pack mentions data residency, region selection, or geographic storage options; the closest related feature is per-agent data retention/deletion (retell-docs-15/27/44), which addresses retention duration, not location of storage. This axis is applicable to a cloud AI voice platform handling call/chat data, so absence of evidence yields 'none'. Missing for 10: any documentation of selectable regions, data residency guarantees, or geo-specific hosting.

   • [claimed-docs] “Configure per-agent data retention to automatically delete call and chat data — transcripts, recordings, and logs — after a set period for c…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent. After the retention period expires, call and chat data associated with tha…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent.”

   proof ↗⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   Vapi—none0/10

   No evidence in the pack addresses data-training opt-out, model-training data policies, or privacy controls preventing AI training use; only HIPAA and SSO/RBAC enterprise features are mentioned, which are unrelated to this specific claim. missing for 10: explicit data-training opt-out policy, DPA/ToS language on training use, dashboard/API setting to exclude data from model training.

   ⚑ flag

   Retell AI—none0/10

   The evidence only covers configurable data retention (auto-deleting transcripts/recordings/logs after a set period) for compliance, not an explicit opt-out or guarantee against using customer data for AI model training. No docs mention training-data usage policy, opt-out toggles, or contractual no-train clauses.

   • [claimed-docs] “Configure per-agent data retention to automatically delete call and chat data — transcripts, recordings, and logs — after a set period for c…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent. After the retention period expires, call and chat data associated with tha…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent.”

   proof ↗⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round to Retell AI
   Vapi—none0/10

   The evidence pack covers HIPAA, SSO/RBAC, tooling, CLI, and MCP server features but contains no documentation about data retention policies, call/recording deletion controls, or user-initiated data export/erasure mechanisms. Missing for 10: retention period settings, deletion/erasure APIs or dashboard controls, data export tools, and any policy documentation on how long call data/transcripts are stored.

   ⚑ flag

   Retell AI!disputedcontradicted5/10

   Retell's docs describe per-agent configurable data retention with automatic, permanent deletion of call/chat transcripts, recordings, and logs after a set period (retell-docs-15, retell-docs-27, retell-docs-44), which is solid first-party evidence for the retention-control axis. However, an independent community report describes the company refusing to delete a user's stored data (credit card) on request, forcing the user to escalate to bank disputes/consumer-affairs complaints — a concrete real-world case where deletion did not work as a customer expected (retell-comm-14). Missing for 10: independent verification that call/chat data deletion itself (not just billing data) works as documented, and no public response/resolution to the deletion complaint.

   • [claimed-docs] “Configure per-agent data retention to automatically delete call and chat data — transcripts, recordings, and logs — after a set period for c…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent. After the retention period expires, call and chat data associated with tha…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent.”
   • [community] “Thread titled 'Retellai won't delete my credit card' - a user complained the company would not delete their stored credit card; commenters s…”

   proof ↗⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Vapi—none0/10

   No evidence pack item addresses telemetry opt-out or usage tracking controls; Vapi's docs cover HIPAA/SSO/RBAC compliance but not a telemetry toggle.

   ⚑ flag

   Retell AI—none0/10

   The evidence pack covers data retention for call/chat data (transcripts, recordings, logs) but contains no mention of telemetry or usage-tracking opt-out controls for the product itself (e.g., CLI/SDK/dashboard analytics collection). Data retention (retell-docs-15/27/44) addresses deletion of customer call data, not opting out of Retell's own telemetry collection.

   ⚑ flag

Call control

1. developerEscalate a live call to a human with warm or blind transfer, passing context along

   weight 2 · round to Retell AI
   Vapi—none0/10

   No evidence in the pack mentions call transfer, warm/blind transfer, or passing context to a human agent; the evidence covers assistants, CLI, MCP server, tools/webhooks, voices, and compliance, but nothing about live-call escalation/transfer capabilities.

   ⚑ flag

   Retell AI~partialclaimed4/10

   Retell's function-calling docs list 'transfer calls' as a built-in agent action alongside ending calls, booking appointments, etc., which supports the general concept of escalating a call to a human. However, the evidence never distinguishes warm vs. blind transfer modes nor describes passing conversational context/metadata to the receiving human agent. missing for 10: explicit warm-transfer vs blind-transfer configuration, evidence of context/data hand-off to the human agent, and any hands-on confirmation of this feature working in practice.

   • [claimed-docs] “Function calling lets Retell single or multi-prompt agents take real actions — transfer calls, end calls, book appointments, send SMS, and c…”
   • [claimed-docs] “Function calling transforms your AI agent from a conversational interface into an action-oriented assistant.”

   proof ↗⚑ flag

2. developerMy agent can send DTMF keypresses, navigate IVR menus, and detect or leave voicemail

   weight 1 · round drawn
   Vapi—none0/10

   The evidence pack contains no mention of DTMF keypress sending, IVR menu navigation, or voicemail detection/leaving functionality anywhere in the docs or community items — these telephony-specific capabilities are entirely unevidenced despite being a plausible axis for a voice AI telephony product.

   ⚑ flag

   Retell AI—none0/10

   The evidence pack covers function calling, custom telephony/SIP trunking, transcription, and MCP tool integration, but nowhere mentions DTMF keypress sending, IVR menu navigation, or voicemail detection/leaving capabilities. This is a fair capability question for a telephony voice-agent platform, but no evidence confirms it is supported.

   ⚑ flag

Campaigns

1. founderRun batch outbound call campaigns with scheduling and throughput controls

   weight 2 · round drawn
   Vapi—none0/10

   Evidence covers assistant creation, tools, CLI, MCP server, testing, and voice customization, but nothing addresses batch/outbound campaign management, call scheduling, or throughput/concurrency controls for bulk dialing. No mention of a campaigns API, CSV/list upload, dialer pacing, or rate-limiting controls for outbound calling at scale.

   ⚑ flag

   Retell AI—none0/10

   Docs describe outbound vs inbound concurrency reservation (retell-docs-11, retell-docs-41) and custom telephony/SIP integration, but there is no evidence of a batch outbound campaign feature — no mention of scheduling calls, uploading contact lists, or campaign-level throughput controls beyond general concurrency limits. Axis is plausible for a voice-agent platform but unsupported by the evidence pack.

   • [claimed-docs] “Reserved inbound concurrency protects inbound calls from being crowded out by outbound traffic.”
   • [claimed-docs] “When `reserved_inbound_concurrency` is configured, outbound calls can use at most your concurrency limit minus the reserved amount.”
   • [claimed-docs] “Integrate Retell voice agents with your own telephony provider using elastic SIP trunking or imported numbers from Twilio, Telnyx, and Vonag…”

   proof ↗⚑ flag

Numbers

1. developerProvision phone numbers and run both inbound and outbound calls through the platform's API

   weight 3 · round drawn
   Vapi✓fullprobed8/10

   Docs explicitly cover creating an assistant, connecting it to a phone number, and making inbound/outbound calls via the quickstart, plus CLI/MCP support for managing phone numbers and calls, and SIP for advanced telephony integration, with runtime probes confirming the CLI and MCP endpoints work as documented. Missing for 10: independent hands-on confirmation of actual outbound call placement via raw API (only demo/inbound anecdote in community evidence) and explicit multi-number provisioning workflow details.

   • [claimed-docs] “const assistant = await vapi.assistant”
   • [claimed-docs] “Create a voice assistant, connect it to a phone number, and make your first calls.”
   • [claimed-docs] “In under 5 minutes, you'll create a voice assistant and start talking to it over the phone.”
   • [claimed-docs] “Manage assistants, phone numbers, and calls from your terminal”
   • [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
   • [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
   • [claimed-docs] “Use any SIP softphone (e.g., Zoiper, Linphone) to dial your SIP URI”
   • [claimed-docs] “This guide shows you how to set up and test SIP calls to your Vapi assistant using any SIP client or softphone.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bas…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”
   • [community] “Called the demo number, sounds smooth! Good luck.”

   proof ↗⚑ flag

   Retell AI✓fullprobed8/10

   Retell's docs show phone-number provisioning via native assignment (docs-32) or importing/SIP-trunking your own numbers from Twilio/Telnyx/Vonage (docs-10,40,49), and both inbound and outbound calling are explicitly supported and distinguished (reserved inbound concurrency vs. outbound traffic in docs-11/41). Full API/SDK coverage for voice endpoints (docs-2,33,46) and a CLI to manage phone numbers (docs-3,21) round out programmatic control, with a working keyless CLI/MCP probe corroborating API-level access (retell-probe-rt-1/2). Missing for 10: explicit documentation of a native 'buy a new number' provisioning endpoint (as opposed to importing existing numbers) and a concrete outbound-call code example.

   • [claimed-docs] “Build your first Retell AI phone agent in 15 minutes: create an account, pick a template, test in the dashboard, deploy to a phone number, a…”
   • [claimed-docs] “Integrate Retell voice agents with your own telephony provider using elastic SIP trunking or imported numbers from Twilio, Telnyx, and Vonag…”
   • [claimed-docs] “Reserved inbound concurrency protects inbound calls from being crowded out by outbound traffic.”
   • [claimed-docs] “This guide shows how to integrate Retell agents with your telephony provider and use your own numbers.”
   • [claimed-docs] “When `reserved_inbound_concurrency` is configured, outbound calls can use at most your concurrency limit minus the reserved amount.”
   • [claimed-docs] “You set up a SIP trunk, configure your number to point to it, and import that number to Retell.”
   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Retell provides official SDKs for Node.js and Python to simplify integration with our platform.”
   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”
   • [claimed-docs] “The Retell CLI lets you manage Retell resources from your terminal.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”

   proof ↗⚑ flag

Sip

1. platform-engineerConnect my own carrier or PBX via SIP trunking (or import Twilio/Telnyx numbers) instead of being locked to bundled telephony

   weight 2 · round to Retell AI
   Vapi~partialclaimed5/10

   Docs confirm SIP connectivity: platform engineers can point any SIP client/softphone at a Vapi SIP URI to make/receive calls (vapi-docs-14, vapi-docs-22), which supports self-hosted/carrier SIP trunking use cases. However, the evidence never explicitly documents PBX/carrier trunk configuration options or a number-import workflow for Twilio/Telnyx accounts. missing for 10: explicit documentation of BYO-carrier/PBX trunk setup steps, and evidence of importing existing Twilio/Telnyx phone numbers into Vapi.

   • [claimed-docs] “Use any SIP softphone (e.g., Zoiper, Linphone) to dial your SIP URI”
   • [claimed-docs] “This guide shows you how to set up and test SIP calls to your Vapi assistant using any SIP client or softphone.”

   proof ↗⚑ flag

   Retell AI✓fullclaimed8/10

   Official docs explicitly describe elastic SIP trunking and importing numbers from Twilio, Telnyx, and Vonage, walking through setting up a SIP trunk and pointing/importing numbers into Retell — directly matching the story. This is first-party documentation without independent hands-on corroboration of the SIP flow itself. Missing for 10: independent/community verification of a real SIP trunk setup working end-to-end, and detail on carrier-specific edge cases (codecs, failover, latency).

   • [claimed-docs] “Integrate Retell voice agents with your own telephony provider using elastic SIP trunking or imported numbers from Twilio, Telnyx, and Vonag…”
   • [claimed-docs] “This guide shows how to integrate Retell agents with your telephony provider and use your own numbers.”
   • [claimed-docs] “You set up a SIP trunk, configure your number to point to it, and import that number to Retell.”

   proof ↗⚑ flag

Analytics

1. ai-native userThe platform's AI reviews my calls for me — scoring quality, flagging failures, and analyzing resolution automatically

   weight 2 · round to Retell AI
   Vapi~partialclaimed4/10

   Vapi's Evals framework provides automated validation of assistant behavior via mock conversations and its Voice Test Suites use an AI tester to simulate calls and check outcomes, which is adjacent to automated call review, but neither is documented as automatically scoring, flagging failures in, or analyzing resolution of real production calls after the fact. Missing for 10: evidence of automatic post-call quality scoring on live/production transcripts, explicit failure-flagging analytics, and resolution-outcome analysis dashboards distinct from pre-production test suites.

   • [claimed-docs] “Evals is Vapi's AI agent testing framework that enables you to systematically test assistants and squads using mock conversations with autom…”
   • [claimed-docs] “you'll create mock conversations, define expected behaviors, and validate your agents work correctly before production”
   • [claimed-docs] “Voice Test Suites enable you to test your AI voice agents through simulated phone conversations.”
   • [claimed-docs] “Our AI tester calls your voice agent and follows a script that simulates real customer behavior.”

   proof ↗⚑ flag

   Retell AI~partialclaimed6/10

   Retell's AI QA feature automatically scores sampled calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage, and the Analytics dashboard tracks success rate and other metrics, directly matching the story's core ask of automated call scoring and quality flagging. However, evidence shows AI QA only evaluates a 'sampled set' of calls (not all calls) against user-configured rules, and there's no documented automatic 'resolution analysis' narrative beyond metrics/dashboards, nor independent hands-on validation of AI QA accuracy. missing for 10: evidence of full (non-sampled) coverage, independent/third-party validation of AI QA scoring accuracy, and detail on automated resolution/failure-cause analysis beyond configured metrics.

   • [claimed-docs] “AI QA scores Retell calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage to surface quality trends and issues…”
   • [claimed-docs] “AI QA automatically evaluates a sampled set of your calls against rules and metrics you configure.”
   • [claimed-docs] “Build custom Retell analytics dashboards to track call and chat metrics like success rate, latency, cost, and concurrency, with charts, filt…”
   • [claimed-docs] “The Analytics dashboard charts your call and chat data so you can see how your agents are performing over time.”
   • [claimed-docs] “Built-in call success and sentiment scoring, plus custom fields synced to your CRM.”

   proof ↗⚑ flag

2. founderSee call analytics — success rates, durations, outcomes, sentiment — in dashboards without building my own

   weight 2 · round to Retell AI
   Vapi—none0/10

   The evidence pack documents assistant creation, tools, testing (Evals, Voice Test Suites), CLI/MCP management, and voice library features, but contains no mention of a dashboard showing call success rates, durations, outcomes, or sentiment analytics — the core of this story.

   ⚑ flag

   Retell AI✓fullclaimed8/10

   Retell docs describe a built-in Analytics dashboard with charts, filters, and breakdowns for success rate, latency, cost, and concurrency, plus AI QA scoring for sentiment, hallucination, and tool usage—directly matching the founder's need for out-of-box call analytics. Missing for 10: independent/hands-on confirmation of dashboard usability and no evidence of exportable reports or deeper outcome breakdowns beyond what's documented.

   • [claimed-docs] “Build custom Retell analytics dashboards to track call and chat metrics like success rate, latency, cost, and concurrency, with charts, filt…”
   • [claimed-docs] “The Analytics dashboard charts your call and chat data so you can see how your agents are performing over time.”
   • [claimed-docs] “AI QA scores Retell calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage to surface quality trends and issues…”
   • [claimed-docs] “AI QA automatically evaluates a sampled set of your calls against rules and metrics you configure.”
   • [claimed-docs] “Built-in call success and sentiment scoring, plus custom fields synced to your CRM.”

   proof ↗⚑ flag

Monitoring

1. platform-engineerMonitor live calls in production and get alerts when agents misbehave or error rates spike

   weight 1 · round to Retell AI
   Vapi—none0/10

   The evidence pack covers pre-production testing (Voice Test Suites, Evals) and webhook debugging via the CLI, but contains no mention of live call monitoring dashboards, real-time alerting, or error-rate-spike detection for production traffic. This is a fair axis for a voice AI platform, but nothing in the pack demonstrates it.

   ⚑ flag

   Retell AI~partialclaimed6/10

   Retell provides real building blocks for production monitoring — a customizable analytics dashboard tracking success rate, latency, cost and concurrency (retell-docs-13/42), AI QA that scores calls for hallucination, sentiment and tool-usage issues (retell-docs-14/43), and webhooks that push real-time event notifications (retell-docs-12/24) which a platform engineer could wire into an alerting pipeline. However there is no documented native alerting/threshold system (e.g., automatic notification when error rates spike or an agent misbehaves) or a live in-call monitoring view — engineers must build that themselves on top of webhooks/dashboard APIs. Missing for 10: built-in threshold-based alerts or anomaly detection, a real-time 'in-progress calls' monitoring view, and any independent evidence that alerting/monitoring works reliably in production.

   • [claimed-docs] “Webhooks allow your application to receive real-time notifications about events that occur in your Retell AI account.”
   • [claimed-docs] “Build custom Retell analytics dashboards to track call and chat metrics like success rate, latency, cost, and concurrency, with charts, filt…”
   • [claimed-docs] “AI QA scores Retell calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage to surface quality trends and issues…”
   • [claimed-docs] “webhooks push data to your application as events happen, making your integrations more efficient and responsive.”
   • [claimed-docs] “The Analytics dashboard charts your call and chat data so you can see how your agents are performing over time.”
   • [claimed-docs] “AI QA automatically evaluates a sampled set of your calls against rules and metrics you configure.”

   proof ↗⚑ flag

Testing

1. developerTest agents with simulated conversations or evals before putting them on real phone calls

   weight 2 · round to Vapi
   Vapi✓fullclaimed9/10

   Vapi documents two dedicated testing features directly matching the story: Voice Test Suites for simulated phone conversations via an AI tester following scripted customer behavior, and Evals, a testing framework for mock conversations with automated validation before production. Missing for 10: independent/hands-on developer corroboration of these specific testing features beyond vendor docs.

   • [claimed-docs] “Voice Test Suites enable you to test your AI voice agents through simulated phone conversations.”
   • [claimed-docs] “Our AI tester calls your voice agent and follows a script that simulates real customer behavior.”
   • [claimed-docs] “Evals is Vapi's AI agent testing framework that enables you to systematically test assistants and squads using mock conversations with autom…”
   • [claimed-docs] “you'll create mock conversations, define expected behaviors, and validate your agents work correctly before production”

   proof ↗⚑ flag

   Retell AI~partialclaimed5/10

   Retell's dashboard has a free 'Test' web-call button that lets developers try an agent before assigning it to a real phone number, and AI QA can score calls on hallucination, accuracy, sentiment, etc., which supports some testing/analytics workflow. However there's no documented feature for automated simulated-conversation test suites or eval scripts run pre-deployment — AI QA appears to operate on sampled real calls rather than synthetic scripted evals. Missing for 10: dedicated simulation/eval framework for scripted test conversations, batch eval tooling, and any independent verification that pre-call testing catches issues before production use.

   • [claimed-docs] “Click the "Test" button to start a web call with your agent... This step is free and doesn't need a phone number or payment method.”
   • [claimed-docs] “Click the "Test" button to start a web call with your agent”
   • [claimed-docs] “AI QA scores Retell calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage to surface quality trends and issues…”
   • [claimed-docs] “AI QA automatically evaluates a sampled set of your calls against rules and metrics you configure.”
   • [claimed-docs] “Build custom Retell analytics dashboards to track call and chat metrics like success rate, latency, cost, and concurrency, with charts, filt…”

   proof ↗⚑ flag

Post call

1. developerExtract structured data from every call — outcomes, entities, dispositions — delivered via API or webhook after the call

   weight 2 · round to Retell AI
   Vapi—none0/10

   The evidence pack shows Vapi's webhook/tool-calling system for live in-call actions (tool-calls messages, custom webhook tools, function calling) but contains no mention of a post-call structured-data/analysis feature (outcomes, entities, dispositions) delivered via API or webhook after the call ends. Missing for 10: any docs on end-of-call reports, structured data extraction schemas, call analysis/summary webhooks, or an API endpoint returning call outcome/entity data.

   • [claimed-docs] “When tools are triggered, your Server URL receives a `tool-calls` message”
   • [claimed-docs] “Vapi supports OpenAI-style tool/function calling. Assistants can ping your server to perform actions.”
   • [claimed-docs] “Custom tools that you create... interact with your systems via webhooks”
   • [claimed-docs] “Create your own webhook-based tools to extend assistant capabilities”

   proof ↗⚑ flag

   Retell AI✓fullclaimed7/10

   Retell explicitly supports post-call structured extraction: built-in call success/sentiment scoring plus custom fields synced to CRM, delivered via real-time webhooks and viewable/aggregated in the analytics/AI QA dashboards. This directly covers outcomes (success rate), dispositions (sentiment), and entities (custom fields) delivered via API/webhook as the story requires. Missing for 10: no independent/hands-on confirmation of the specific post-call-analysis JSON schema or webhook payload structure, and no detail on how custom entity fields are defined/configured.

   • [claimed-docs] “Built-in call success and sentiment scoring, plus custom fields synced to your CRM.”
   • [claimed-docs] “Webhooks allow your application to receive real-time notifications about events that occur in your Retell AI account.”
   • [claimed-docs] “webhooks push data to your application as events happen, making your integrations more efficient and responsive.”
   • [claimed-docs] “Build custom Retell analytics dashboards to track call and chat metrics like success rate, latency, cost, and concurrency, with charts, filt…”
   • [claimed-docs] “The Analytics dashboard charts your call and chat data so you can see how your agents are performing over time.”
   • [claimed-docs] “AI QA scores Retell calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage to surface quality trends and issues…”
   • [claimed-docs] “AI QA automatically evaluates a sampled set of your calls against rules and metrics you configure.”

   proof ↗⚑ flag

Tools

1. ai-native userMy voice agent can plug in MCP servers as tool sources so one integration grants it whole toolsets mid-call

   weight 2 · round to Retell AI
   Vapi—none0/10

   Evidence only shows Vapi exposing its own APIs as an MCP *server* for external AI assistants (e.g., Claude Desktop) to manage calls/assistants — the reverse direction of the story. There is no documentation or probe showing a Vapi voice assistant can itself act as an MCP *client*, plugging in external MCP servers as tool sources mid-call; tool integration is instead described only via webhook-based custom tools and OpenAI-style function calling.

   • [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
   • [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
   • [claimed-docs] “Create your own webhook-based tools to extend assistant capabilities”
   • [claimed-docs] “Vapi supports OpenAI-style tool/function calling. Assistants can ping your server to perform actions.”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”

   proof ↗⚑ flag

   Retell AI✓fullclaimed8/10

   Retell documents connecting single/multi-prompt voice agents to remote MCP servers so the agent can call the server's tools mid-call (retell-docs-6/23/36), directly matching the story, and this is distinct from Retell's own client-side MCP server for building agents. missing for 10: independent third-party corroboration of mid-call MCP tool invocation in production (only first-party docs and a probe of the client-facing MCP server, not the agent-as-MCP-client tool-call path, are available).

   • [claimed-docs] “Connect a Retell single- or multi-prompt agent to a remote MCP server so it can call the server's tools during a live voice or chat conversa…”
   • [claimed-docs] “Connect your single- or multi-prompt agent to a remote Model Context Protocol (MCP) server, and the agent can call that server's tools durin…”
   • [claimed-docs] “Connect your single- or multi-prompt agent to a remote [Model Context Protocol (MCP)] server, and the agent can call that server's tools dur…”
   • [claimed-docs] “Function calling lets Retell single or multi-prompt agents take real actions — transfer calls, end calls, book appointments, send SMS, and c…”
   • [claimed-docs] “Function calling transforms your AI agent from a conversational interface into an action-oriented assistant.”

   proof ↗⚑ flag

2. developerMy agent can call external APIs and custom functions mid-conversation and speak the result without awkward dead air

   weight 3 · round drawn
   Vapi~partialclaimed6/10

   Vapi's docs clearly show mid-call function/tool calling via webhook Server URLs and OpenAI-style tool-calls messages, letting the assistant fetch external API results and use them in conversation (vapi-docs-6, vapi-docs-7, vapi-docs-17, vapi-docs-20, vapi-docs-23, vapi-docs-34). However, the pack lacks explicit evidence about mechanisms for avoiding 'dead air' during the API call latency (e.g., async tool config, filler phrases, or interim speech) — only general latency-tuning docs for model selection are present (vapi-docs-15). Missing for 10: explicit documentation/demo of filler/interim speech or async tool handling during function execution, and independent hands-on confirmation that the conversation flow feels seamless during a live tool call.

   • [claimed-docs] “Create your own webhook-based tools to extend assistant capabilities”
   • [claimed-docs] “Server URL: The endpoint where your function is hosted”
   • [claimed-docs] “When tools are triggered, your Server URL receives a `tool-calls` message”
   • [claimed-docs] “This guide shows you how to create custom tools, including Function Tools, for your Vapi assistants.”
   • [claimed-docs] “Vapi supports OpenAI-style tool/function calling. Assistants can ping your server to perform actions.”
   • [claimed-docs] “Custom tools that you create... interact with your systems via webhooks”
   • [claimed-docs] “Ultra Fast uses speed-optimized models for lower latency. High Intelligence accepts higher latency for greater capability.”

   proof ↗⚑ flag

   Retell AI~partialprobed6/10

   Retell's docs clearly document function calling that lets agents call external APIs mid-conversation (retell-docs-5, retell-docs-35) and even connect to remote MCP servers to invoke tools live during a call (retell-docs-6, retell-docs-23, retell-docs-36), confirmed by a keyless runtime probe showing the hosted MCP server exposing invoke_api_endpoint tools (retell-probe-rt-2). However, no evidence specifically addresses the 'no awkward dead air' requirement — there's no documented filler-speech, streaming-response, or latency-masking mechanism specifically tied to function-call execution; transcription-mode docs (retell-docs-9/39/48) discuss general STT latency tradeoffs, not function-call dead-air handling. missing for 10: explicit filler/backchannel speech during function execution, hands-on demonstration of seamless mid-call tool latency, independent confirmation of no dead-air behavior.

   • [claimed-docs] “Function calling lets Retell single or multi-prompt agents take real actions — transfer calls, end calls, book appointments, send SMS, and c…”
   • [claimed-docs] “Function calling transforms your AI agent from a conversational interface into an action-oriented assistant.”
   • [claimed-docs] “Connect a Retell single- or multi-prompt agent to a remote MCP server so it can call the server's tools during a live voice or chat conversa…”
   • [claimed-docs] “Connect your single- or multi-prompt agent to a remote Model Context Protocol (MCP) server, and the agent can call that server's tools durin…”
   • [claimed-docs] “Connect your single- or multi-prompt agent to a remote [Model Context Protocol (MCP)] server, and the agent can call that server's tools dur…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”

   proof ↗⚑ flag

Recording

1. platform-engineerRetrieve full call recordings and transcripts programmatically for every call

   weight 2 · round to Retell AI
   Vapi—none0/10

   The evidence pack contains no documentation, API reference, or example demonstrating retrieval of call recordings or transcripts, despite this being a natural, expected capability for a voice AI platform; only tangential tooling (CLI, MCP server, custom tools, testing/evals) is covered.

   ⚑ flag

   Retell AI~partialclaimed6/10

   Docs confirm that Retell stores per-call transcripts and recordings (referenced in data-retention docs) and that full voice/chat API endpoint coverage exists via official SDKs, implying programmatic retrieval, but no evidence pack item explicitly documents a 'get call' or 'list calls' API endpoint returning recording URLs/transcript text, nor examples of pulling them via SDK/CLI. Missing for 10: explicit API reference for call/recording/transcript retrieval endpoints, CLI/SDK code samples showing recording download or transcript fetch, and any independent confirmation of this working end-to-end.

   • [claimed-docs] “Configure per-agent data retention to automatically delete call and chat data — transcripts, recordings, and logs — after a set period for c…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent. After the retention period expires, call and chat data associated with tha…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent.”
   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Retell provides official SDKs for Node.js and Python to simplify integration with our platform.”
   • [claimed-docs] “Webhooks allow your application to receive real-time notifications about events that occur in your Retell AI account.”
   • [claimed-docs] “webhooks push data to your application as events happen, making your integrations more efficient and responsive.”

   proof ↗⚑ flag

Transcription

1. developerGet accurate real-time transcription with control over the STT provider, language models, or key terms

   weight 2 · round to Retell AI
   Vapi—none0/10

   The evidence pack contains no documentation about selecting/configuring an STT provider, choosing language models for transcription, or defining custom key terms/vocabulary for real-time transcription — topics like 'transcriber', 'Deepgram', or keyword boosting are absent. While Vapi is clearly a voice AI platform where such controls are a fair axis, none of the provided docs, community items, or probes address it.

   ⚑ flag

   Retell AI~partialcommunity4/10

   Retell docs confirm real-time transcription with a documented latency/accuracy trade-off (interim vs context-aware endpointing), giving developers some control over accuracy tuning, and community notes hint at underlying provider choices (e.g., Deepgram) for voice pipelines. However, there is no documented ability to select or swap STT providers, choose an ASR language model, or configure custom vocabulary/key-term boosting for transcription accuracy. Missing for 10: explicit STT provider selection API, language model choice for transcription, custom vocabulary/key-term boosting support, and independent accuracy benchmarks.

   • [claimed-docs] “Pick the right Retell transcription mode to balance accuracy against latency — interim results for speed or context-aware results for higher…”
   • [claimed-docs] “Real time transcription is often a trade off between latency and accuracy.”
   • [claimed-docs] “optimize for speed: uses the latest interim results with a low endpointing setting... optimize for accuracy: uses the results with a higher …”
   • [community] “Founder (yanyan_evie) noted voice provider costs differ: 'The different providers have different prices. openai tts & deepgram are cheaper, …”

   proof ↗⚑ flag

Voices

1. founderClone a custom brand voice and use it for my agents, with a documented consent process

   weight 2 · round to Retell AI
   Vapi—none0/10

   Vapi docs show you can plug in a 'custom voice' by setting the voice property with a provider ID (vapi-docs-9, vapi-docs-26) and browse a Voice Library (vapi-docs-8, vapi-docs-35), but nothing describes an actual voice-cloning workflow or any documented consent/verification process required before cloning a brand voice. Missing for 10: a described voice-cloning feature/flow, a documented consent or identity-verification process, and any policy language governing voice cloning.

   • [claimed-docs] “You can use your own custom voice with any supported provider by setting the `voice` property in your assistant configuration”
   • [claimed-docs] “You can use your own custom voice with any supported provider by setting the voice property in your assistant configuration”
   • [claimed-docs] “The Voice Library in the Vapi Dashboard lists every voice available to your organization. Browse and preview voices there”
   • [claimed-docs] “The **Voice Library** in the [Vapi Dashboard] lists every voice available to your organization. Browse and preview voices there, then copy a…”

   proof ↗⚑ flag

   Retell AI~partialclaimed5/10

   Retell documents voice cloning from uploaded audio files (up to 25 files) and lets you attach the cloned voice to an agent via the voice selector, covering the core 'clone and use a custom voice' capability. However, none of the evidence describes any documented consent-verification step (e.g., consent recording, rights attestation) as part of the cloning flow. Missing for 10: an explicit consent-collection/verification mechanism in the clone-voice API or dashboard docs, and any compliance guidance tying voice cloning to consent requirements.

   • [claimed-docs] “Add custom voices to your Retell agent — search ElevenLabs community voices, import a voice clone, or train a clone for a unique brand-speci…”
   • [claimed-docs] “Clone a voice from audio files”
   • [claimed-docs] “Audio files to use for voice cloning. Up to 25 files allowed.”
   • [claimed-docs] “You can also add a voice clone by clicking "Add custom voice" in the voice selector.”

   proof ↗⚑ flag

2. developerChoose from a broad voice library or plug in multiple TTS providers to get the voice I want

   weight 2 · round to Retell AI
   Vapi~partialclaimed6/10

   Docs confirm a Voice Library with many previewable voices and support for custom voices with any supported provider, indicating multi-provider TTS flexibility, but the evidence never names or lists specific TTS providers (e.g., ElevenLabs, PlayHT, Azure) or details plugging in third-party/custom TTS engines beyond voice ID selection. missing for 10: explicit list of supported TTS providers, documentation of custom/BYO TTS provider integration mechanics, independent hands-on confirmation of voice quality/variety.

   • [claimed-docs] “The Voice Library in the Vapi Dashboard lists every voice available to your organization. Browse and preview voices there”
   • [claimed-docs] “You can use your own custom voice with any supported provider by setting the `voice` property in your assistant configuration”
   • [claimed-docs] “Browse and preview voices there, then copy a voice's ID to use on an assistant.”
   • [claimed-docs] “You can use your own custom voice with any supported provider by setting the voice property in your assistant configuration”
   • [claimed-docs] “The **Voice Library** in the [Vapi Dashboard] lists every voice available to your organization. Browse and preview voices there, then copy a…”

   proof ↗⚑ flag

   Retell AI✓fullcommunity7/10

   Docs confirm a searchable voice library (ElevenLabs community voices) plus voice cloning options (retell-docs-7, retell-docs-37, retell-docs-38), and a founder community comment independently confirms multiple underlying TTS providers (OpenAI TTS, Deepgram, ElevenLabs) with different pricing tiers (retell-comm-10), showing developers can indeed pick across providers/voices. Missing for 10: a first-party docs page enumerating all supported TTS providers and API-level provider-switching parameters beyond the voice-selector UI, and independent hands-on comparison of voice quality across providers.

   • [claimed-docs] “Add custom voices to your Retell agent — search ElevenLabs community voices, import a voice clone, or train a clone for a unique brand-speci…”
   • [claimed-docs] “You can also add a voice clone by clicking "Add custom voice" in the voice selector.”
   • [claimed-docs] “In the voice selector, you can click "Add custom voice" to search and add publicly available community voices.”
   • [community] “Founder (yanyan_evie) noted voice provider costs differ: 'The different providers have different prices. openai tts & deepgram are cheaper, …”

   proof ↗⚑ flag