Retell AI vs ElevenLabs Agents

Agent ops

1. ai-native userMy coding agent can provision a complete voice agent end to end — create the agent, attach a number, and place a call — through the API, CLI, or MCP without touching the dashboard

   weight 3 · round drawn
   Retell AI~partialprobed6/10

   Retell ships an official Node/Python SDK with full voice endpoint coverage, a CLI for managing agents and phone numbers, and an MCP server whose meta-tools (list/get/invoke_api_endpoint) expose the entire API — including agent create/update/publish — to MCP clients like Cursor or Claude Code, and a keyless runtime probe confirms the MCP handshake works end-to-end. However, the flagship quick-start walkthrough is dashboard-centric (create in UI, assign number in 'configuration settings', test/call via dashboard button), and no evidence explicitly shows a CLI/API/MCP call sequence that attaches a number and places a live call without touching the UI. Missing for 10: an explicit end-to-end CLI/MCP example showing number-attach and call-placement commands, and independent confirmation that non-dashboard number provisioning/outbound calling works in practice.

   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”
   • [claimed-docs] “Use Retell's MCP server to build and manage voice agents from MCP-capable clients like Cursor, Claude Desktop, and Claude Code via Retell AP…”
   • [claimed-docs] “Retell supports the Model Context Protocol (MCP) so you can build Retell AI voice agents directly from MCP-capable clients (Cursor, Claude D…”
   • [claimed-docs] “Agents: create, update, publish, list, and fetch agent versions.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”
   • [claimed-docs] “Assign your agent to the number in the configuration settings”

   proof ↗⚑ flag

   ElevenLabs Agents~partialprobed6/10

   Docs and runtime probes confirm agents can be created and managed via API, a real CLI (@elevenlabs/cli, verified live on npm), and a hosted MCP server (verified live with OAuth), all without the dashboard (docs-1,3,4,27,28,38; probe-rt-1,2). Phone-number attachment is also supported via SIP trunking to route calls to agents (docs-12,36), but there is no explicit documentation or evidence of actually placing/initiating an outbound call end-to-end through the API/CLI/MCP — only routing inbound infrastructure integration is described. Missing for 10: explicit API/CLI/MCP call to place an outbound call, and a concrete end-to-end walkthrough combining create+attach+call in one non-dashboard flow.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “Use the ElevenLabs agents skill to build and manage voice agents from your AI coding assistant: npx skills add elevenlabs/skills --skill age…”
   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
   • [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”
   • [claimed-docs] “Route calls to AI agents without changing your existing phone infrastructure”
   • [claimed-docs] “connect the hosted MCP server to Claude or any other MCP client to create and manage agents through natural language”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”

   proof ↗⚑ flag

2. ai-native userThe platform's own AI helps me author agents — generating or improving prompts, flows, and test cases from a description

   weight 1 · round drawn
   Retell AI~partialclaimed5/10

   Retell docs mention a 'Generate from prompt' feature where Conductor drafts an agent from a plain-English description (retell-docs-30), directly supporting AI-assisted agent authoring. However, there's no evidence of AI-driven improvement of existing prompts/flows or automatic test-case generation, and no independent/hands-on confirmation of Conductor's output quality. Missing for 10: documentation on iterative prompt/flow refinement by the platform's AI, automated test-case generation, and community or hands-on verification of Conductor's generated agents.

   • [claimed-docs] “click **Generate from prompt** (marked Suggested) to let [Conductor](/conductor/create-agent) draft an agent from a plain-English descriptio…”
   • [claimed-docs] “Retell agents are node-based flows or single prompts, with call handling, a knowledge base, and integrations built in.”
   • [claimed-docs] “Drag-and-drop, node-based flows for structured, high-stakes calls.”
   • [claimed-docs] “Single prompt agent... Define your whole agent with one prompt.”

   proof ↗⚑ flag

   ElevenLabs Agents~partialprobed5/10

   ElevenLabs documents that an external AI assistant (e.g., Claude) can create, configure, and manage agents through natural language via the hosted MCP server, and agent-testing docs describe turning real conversations into test cases for regression testing — both are AI-assisted authoring flows. However, this relies on a third-party AI client (Claude) rather than a built-in, platform-native 'describe your agent and we generate it' copilot for prompts/flows within the dashboard itself. Missing for 10: evidence of a first-party generative wizard that authors prompts/flows purely from a natural-language description inside the ElevenLabs UI, and evidence of AI-generated (not just conversation-derived) test-case authoring.

   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
   • [claimed-docs] “connect the hosted MCP server to Claude or any other MCP client to create and manage agents through natural language”
   • [claimed-docs] “Transform real conversations into test cases when you find an interaction where the agent underperformed.”
   • [claimed-docs] “Agent testing lets you verify conversational responses, tool usage, and full multi-turn outcomes before you deploy.”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”

   proof ↗⚑ flag

Build

1. developerBuild a working phone voice agent — prompt, voice, and phone number — and take my first live call within an hour

   weight 3 · round to Retell AI
   Retell AI✓fullcommunity8/10

   Docs walk through the exact flow described: create account, pick/generate a template, set prompt and voice, test in dashboard, assign a phone number, and make a live call, with a 15-minute quickstart and free trial credits removing payment friction. Community feedback confirms the demo/agent works end-to-end for real calls, though it also shows occasional conversational glitches (contradictions, confusion) that are quality issues rather than build-flow blockers. missing for 10: independent third-party timing confirmation that a full build-to-first-call truly completes within an hour, and evidence of phone-number provisioning speed in practice.

   • [claimed-docs] “Build your first Retell AI phone agent in 15 minutes: create an account, pick a template, test in the dashboard, deploy to a phone number, a…”
   • [claimed-docs] “Click the "Test" button to start a web call with your agent... This step is free and doesn't need a phone number or payment method.”
   • [claimed-docs] “click **Generate from prompt** (marked Suggested) to let [Conductor](/conductor/create-agent) draft an agent from a plain-English descriptio…”
   • [claimed-docs] “Click the "Test" button to start a web call with your agent”
   • [claimed-docs] “Assign your agent to the number in the configuration settings”
   • [claimed-docs] “New accounts start with \$10 in free trial credits, no payment method required.”
   • [claimed-docs] “Add custom voices to your Retell agent — search ElevenLabs community voices, import a voice clone, or train a clone for a unique brand-speci…”
   • [community] “Curious what model the dentist bot is running on? Tried it out, was surprisingly good, though eventually it contradicted itself (booked a sl…”
   • [community] “I tried to fool the Dental Office demo trying to book on Sunday or outside of the slots it had indicated, and it did a better job than many …”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed7/10

   Docs show a clear quickstart path — create an agent via dashboard/API, pick from 5k+ voices, and connect phone numbers via SIP trunking to route calls to the agent without changing existing phone infrastructure — which together cover the prompt+voice+phone number pieces of the story. However, there's no explicit end-to-end 'first call in under an hour' walkthrough or independent timing evidence, and phone-number provisioning specifics (vs. just SIP integration) are thin. Missing for 10: an explicit timed/complete quickstart-to-live-call walkthrough, first-party or independent evidence of the actual time-to-first-call, and clearer native phone-number purchase/setup docs beyond SIP trunking.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “Select from 5k+ voices across 31 languages with customization options”
   • [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”
   • [claimed-docs] “Route calls to AI agents without changing your existing phone infrastructure”
   • [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”
   • [claimed-docs] “This assistant can be embedded into your website or app to provide real-time support to your customers”

   proof ↗⚑ flag

2. developerRun conversations in multiple languages, including detecting and switching language mid-call

   weight 2 · round to ElevenLabs Agents
   Retell AI—none0/10

   No documentation in the evidence pack describes multi-language conversation support, language detection, or mid-call language switching; the only related evidence is a founder comment stating multilingual support is merely 'on our roadmap' rather than shipped, indicating the capability does not currently exist.

   • [community] “Founder said multilingual support is on roadmap: 'It's definitely in our roadmap. After the core product—the voice AI part—becomes humanlike…”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed4/10

   Docs confirm broad multi-language support (70+ languages, language-specific voice settings) and voice-switching for multi-character use, but there is no explicit documentation of automatic language detection or switching languages mid-call. Missing for 10: explicit language-detection/mid-call switching feature docs, and any hands-on demonstration of a live language switch during a single conversation.

   • [claimed-docs] “Configure, deploy and monitor natural, human-sounding agents in 70+ languages with leading accuracy and ultra-low latency across voice or ch…”
   • [claimed-docs] “You can customize various aspects of your AI agent's voice to create a more natural and engaging conversation experience. This includes cont…”
   • [claimed-docs] “Enable your agent to switch between different voices for multi-character conversations, storytelling, and language tutoring.”
   • [claimed-docs] “Select from 5k+ voices across 31 languages with customization options”

   proof ↗⚑ flag

3. founderDesign multi-step conversation flows in a visual builder with branching, states, and handoffs without writing code

   weight 2 · round to Retell AI
   Retell AI✓fullclaimed7/10

   Retell's docs describe node-based, drag-and-drop conversation flows for 'structured, high-stakes calls' as an alternative to single-prompt agents, with function calling enabling transfers, call-ending, and API calls that serve as handoff points — directly matching the branching/states/handoff story without requiring code. The quick-start flow (pick template, test in dashboard, deploy) reinforces a no-code workflow. Missing for 10: independent/hands-on evidence of the visual builder's branching UI itself (community evidence only covers conversational behavior, not the builder), and no detail on how 'states' are represented/connected beyond the general node-based description.

   • [claimed-docs] “Retell agents are node-based flows or single prompts, with call handling, a knowledge base, and integrations built in.”
   • [claimed-docs] “Drag-and-drop, node-based flows for structured, high-stakes calls.”
   • [claimed-docs] “Single prompt agent... Define your whole agent with one prompt.”
   • [claimed-docs] “Function calling lets Retell single or multi-prompt agents take real actions — transfer calls, end calls, book appointments, send SMS, and c…”
   • [claimed-docs] “Function calling transforms your AI agent from a conversational interface into an action-oriented assistant.”
   • [claimed-docs] “Build your first Retell AI phone agent in 15 minutes: create an account, pick a template, test in the dashboard, deploy to a phone number, a…”
   • [claimed-docs] “click **Generate from prompt** (marked Suggested) to let [Conductor](/conductor/create-agent) draft an agent from a plain-English descriptio…”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed6/10

   Docs explicitly claim a 'visual workflow builder' for multi-step workflows, plus dashboard-based agent creation, conversation-flow settings (turn-taking, interruptions, timeouts), and tools/handoff-like actions (webhooks, client tools) — supporting the no-code, branching, multi-step story. However, evidence never details branching logic, explicit 'states' or agent-to-agent 'handoffs' within the builder, nor independent hands-on confirmation of the visual builder's depth. Missing for 10: concrete detail/screenshots of branching and state logic in the builder, explicit handoff-between-agents feature description, and independent/third-party corroboration of the builder's no-code sufficiency.

   • [claimed-docs] “Build multi-step workflows with visual workflow builder”
   • [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”
   • [claimed-docs] “Configure turn-taking, interruptions, and timeout settings”
   • [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”
   • [claimed-docs] “Enable agents to call clients & APIs to perform actions”

   proof ↗⚑ flag

Personalization

1. developerInject dynamic variables and per-caller context at call time so each conversation is personalized

   weight 2 · round to ElevenLabs Agents
   Retell AI—none0/10

   The evidence pack covers function calling, webhooks, custom telephony, knowledge base retrieval, CRM field sync, and SDK/CLI/MCP tooling, but nowhere describes injecting dynamic variables or per-caller context (e.g., a startCall/create-call parameter for passing caller-specific data into the prompt/LLM at runtime). This is a fair, plausible capability for a voice-agent platform, so absence of evidence yields 'none' rather than 'na'.

   ⚑ flag

   ElevenLabs Agents✓fullclaimed8/10

   Docs explicitly state 'Use dynamic variables and overrides for per-conversation customization' as a core build feature, directly matching the story of injecting per-caller context at call time. Missing for 10: no independent/hands-on example showing a real dynamic-variable injection call in code, and no detail on variable scoping/limits beyond the single doc mention.

   • [claimed-docs] “Use dynamic variables and overrides for per-conversation customization”

   proof ↗⚑ flag

2. developerGround the agent on my documents with a built-in knowledge base or RAG so it answers from my content

   weight 2 · round to ElevenLabs Agents
   Retell AI~partialclaimed6/10

   Retell explicitly ships a knowledge base feature where you can crawl websites or upload documents for the agent to retrieve from, and AI QA scores 'knowledge base accuracy' as a call metric, confirming grounding is a first-class capability. However, evidence is thin — only brief homepage-level mentions rather than a dedicated deep-dive doc on KB architecture/RAG internals, and there's no independent/hands-on confirmation of retrieval quality. Missing for 10: dedicated knowledge-base documentation page detailing chunking/retrieval mechanics, and independent verification that answers are accurately grounded in uploaded content.

   • [claimed-docs] “Retell agents are node-based flows or single prompts, with call handling, a knowledge base, and integrations built in.”
   • [claimed-docs] “Crawl a website or upload documents your agent retrieves from.”
   • [claimed-docs] “AI QA scores Retell calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage to surface quality trends and issues…”

   proof ↗⚑ flag

   ElevenLabs Agents✓fullclaimed8/10

   Docs explicitly state agents support uploading documents and enabling RAG for grounded responses, directly matching the story's built-in knowledge base/RAG capability. Missing for 10: independent/hands-on corroboration of RAG quality and details on knowledge base management (chunking, size limits, sources).

   • [claimed-docs] “Upload documents and enable RAG for grounded responses”

   proof ↗⚑ flag

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round drawn
   Retell AI✓fullprobed9/10

   A direct probe confirms an llms.txt file exists at docs.retellai.com/llms.txt returning HTTP 200 with a structured index of the docs, and Retell's docs are also agent-oriented enough to support MCP-based discovery/management of resources. This directly satisfies the story of pointing an agent at llms.txt or agent-oriented docs. Missing for 10: no independent third-party corroboration of llms.txt usage in the wild.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.retellai.com/llms.txt # Retell AI > Explore Retell AI docs to learn how to build, test, deploy, an…”
   • [claimed-docs] “Use Retell's MCP server to build and manage voice agents from MCP-capable clients like Cursor, Claude Desktop, and Claude Code via Retell AP…”

   proof ↗⚑ flag

   ElevenLabs Agents✓fullprobed9/10

   ElevenLabs publishes a working llms.txt at the root (HTTP 200) and structures its docs as machine-readable .md pages with an explicit pointer to a full docs/llms.txt index, exactly matching the ai-native 'point an agent at llms.txt' pattern; this was independently verified via live probes rather than just claimed docs. Missing for 10: no evidence of a hands-on agent actually consuming the file end-to-end to complete a task (only that the endpoint/format is live and documented).

   • [probe] “PROBE llms.txt: HTTP 200 at https://elevenlabs.io/llms.txt # ElevenLabs ElevenLabs is an AI research and product company transforming how w…”
   • [probe] “PROBE docs-md: HTTP 200 at https://elevenlabs.io/docs/eleven-agents/overview.md > This is a page from the ElevenLabs documentation. For a co…”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to Retell AI
   Retell AI~partialprobed7/10

   Retell ships official Node.js/Python SDKs with API-key auth and a terminal CLI for managing agents, phone numbers, and resources, both of which are scriptable outside the dashboard, and a runtime probe confirms the CLI installs and runs keylessly via npm — all consistent with headless/CI use. However, there is no explicit CI/automation documentation (e.g., GitHub Actions example, testing-in-pipeline guide) confirming an officially supported headless workflow. Missing for 10: explicit CI/pipeline documentation or example, confirmation of non-interactive auth flow for CI secrets, and independent evidence of real-world CI usage.

   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”
   • [claimed-docs] “The Retell CLI lets you manage Retell resources from your terminal.”
   • [claimed-docs] “Retell provides official SDKs for Node.js and Python to simplify integration with our platform.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”
   • [probe] “official CLI documented at https://docs.retellai.com/get-started/cli”

   proof ↗⚑ flag

   ElevenLabs Agents~partialprobed6/10

   ElevenLabs Agents can be managed via a full REST API and an official CLI (verified runtime install/run), enabling scripted/headless creation and configuration of agents outside the dashboard, which supports CI-style automation. However, evidence is docs/probe-based with no CI pipeline examples, no first-class 'run in CI' guidance, and Agent testing is described as pre-deploy validation without confirmation it can be triggered non-interactively/headlessly. Missing for 10: explicit CI/CD integration examples, headless testing invocation docs, and independent hands-on confirmation of automated pipelines.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”
   • [claimed-docs] “Agent testing lets you verify conversational responses, tool usage, and full multi-turn outcomes before you deploy.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round drawn
   Retell AI✓fullclaimed8/10

   Retell explicitly documents connecting a live voice/chat agent to a remote MCP server so it can invoke that server's tools during calls, directly matching the story of plugging in external MCP servers for tool use. This is a first-party, well-specified capability (not just Retell exposing its own API as MCP) covering both single- and multi-prompt agents. Missing for 10: independent/hands-on verification that a third-party MCP server's tools work reliably mid-call in production.

   • [claimed-docs] “Connect a Retell single- or multi-prompt agent to a remote MCP server so it can call the server's tools during a live voice or chat conversa…”
   • [claimed-docs] “Connect your single- or multi-prompt agent to a remote Model Context Protocol (MCP) server, and the agent can call that server's tools durin…”
   • [claimed-docs] “Connect your single- or multi-prompt agent to a remote [Model Context Protocol (MCP)] server, and the agent can call that server's tools dur…”
   • [claimed-docs] “Function calling transforms your AI agent from a conversational interface into an action-oriented assistant.”

   proof ↗⚑ flag

   ElevenLabs Agents✓fullclaimed8/10

   Docs explicitly describe connecting conversational agents to external MCP servers so agents can access data sources and tools via MCP, directly matching the story, and this is corroborated by dedicated documentation pages (mcp.md) rather than just passing mention. Missing for 10: independent/hands-on verification of an actual third-party MCP server being plugged in and tool-calls succeeding in production (only docs and a runtime probe on the separate hosted-server feature are present).

   • [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers. This enables your agents to: Access and process informa…”
   • [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers.”
   • [claimed-docs] “Access and process information from various data sources via the MCP server”

   proof ↗⚑ flag

4. ai-native userConnect an agent via an official MCP server

   weight 3 · round drawn
   Retell AI✓fullprobed9/10

   Retell documents and hosts an official MCP server (mcp.retellai.com) that lets MCP-capable clients like Cursor, Claude Desktop, and Claude Code build and manage Retell voice agents via the Retell API, exposing meta-tools for listing/invoking endpoints. A runtime probe confirms the server completes a full keyless JSON-RPC initialize handshake and exposes its tool set, corroborating the vendor docs. Missing for 10: independent/community usage reports of the MCP server in practice beyond vendor docs and the probe.

   • [claimed-docs] “Use Retell's MCP server to build and manage voice agents from MCP-capable clients like Cursor, Claude Desktop, and Claude Code via Retell AP…”
   • [claimed-docs] “Retell supports the Model Context Protocol (MCP) so you can build Retell AI voice agents directly from MCP-capable clients (Cursor, Claude D…”
   • [claimed-docs] “Retell supports the [Model Context Protocol (MCP)] so you can build Retell AI voice agents directly from MCP-capable clients (Cursor, Claude…”
   • [claimed-docs] “Agents: create, update, publish, list, and fetch agent versions.”
   • [probe] “official MCP server documented at https://docs.retellai.com/get-started/mcp-server”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”

   proof ↗⚑ flag

   ElevenLabs Agents✓fullprobed9/10

   ElevenLabs explicitly documents and operates a hosted MCP server (api.elevenlabs.io/v1/mcp) that lets external MCP clients like Claude connect to create, configure, and manage agents through natural language, with a runtime probe confirming the live endpoint correctly responds with the MCP OAuth challenge. This is a first-party 'official MCP server' offering, not just client-side MCP consumption, matching the story precisely. Missing for 10: independent/community hands-on confirmation beyond ElevenLabs' own docs and the single runtime probe.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
   • [claimed-docs] “connect the hosted MCP server to Claude or any other MCP client to create and manage agents through natural language”
   • [claimed-docs] “create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally”
   • [probe] “official MCP server documented at https://elevenlabs.io/docs/eleven-agents/operate/hosted-mcp”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”

   proof ↗⚑ flag

5. ai-native userUse an official CLI

   weight 2 · round drawn
   Retell AI✓fullprobed9/10

   Retell ships an official CLI (`@retell-ai/retell-cli`) documented for managing agents, phone numbers, and knowledge bases from the terminal, and a runtime probe confirms it installs and runs keylessly via npx. Missing for 10: independent/community usage reports of the CLI itself (only docs and one probe run corroborate it).

   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”
   • [claimed-docs] “The Retell CLI lets you manage Retell resources from your terminal.”
   • [probe] “official CLI documented at https://docs.retellai.com/get-started/cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”

   proof ↗⚑ flag

   ElevenLabs Agents✓fullprobed9/10

   ElevenLabs documents an official Agents CLI for managing agents, and runtime probing confirms `npx @elevenlabs/cli` installs and runs keylessly from npm, corroborating the docs claim with hands-on evidence. Missing for 10: deeper documentation of full CLI command surface/subcommands beyond basic install confirmation.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “Use the ElevenLabs agents skill to build and manage voice agents from your AI coding assistant: npx skills add elevenlabs/skills --skill age…”
   • [claimed-docs] “Use the ElevenLabs agents skill to build and manage voice agents from your AI coding assistant”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”

   proof ↗⚑ flag

6. ai-native userDrive the product through a documented public API

   weight 3 · round drawn
   Retell AI✓fullprobed9/10

   Retell documents official Node.js/Python SDKs with typed clients and full endpoint coverage, a CLI for managing resources from the terminal, and API references (e.g., clone-voice endpoint) — and a runtime probe confirms the CLI installs/runs keylessly and the hosted MCP server exposes the entire API via meta-tools, proving the API is genuinely agent-drivable. Missing for 10: a publicly discoverable OpenAPI/swagger spec (all standard paths returned 404 in the probe).

   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”
   • [claimed-docs] “Retell provides official SDKs for Node.js and Python to simplify integration with our platform.”
   • [claimed-docs] “Type safety: Full TypeScript support with autocomplete ... Error handling: Structured error responses with detailed messages”
   • [probe] “official MCP server documented at https://docs.retellai.com/get-started/mcp-server”
   • [probe] “official CLI documented at https://docs.retellai.com/get-started/cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.retellai.com/openapi.json, https://docs.retellai.com/swagger.json, https://docs.retella…”

   proof ↗⚑ flag

   ElevenLabs Agents✓fullprobed9/10

   ElevenLabs Agents ships a documented public API alongside a dashboard, CLI, and hosted MCP server for programmatic agent creation/management, with quickstart docs explicitly noting 'creating an agent via the API' and full customization (tools, voice, analysis, testing) accessible programmatically. Runtime probes confirm the CLI installs keylessly and the hosted MCP endpoint is live and functional. Missing for 10: no independent third-party API reference/SDK usage examples or community corroboration beyond first-party docs.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”
   • [claimed-docs] “Enable agents to call clients & APIs to perform actions”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”

   proof ↗⚑ flag

7. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round drawn
   Retell AI—none0/10

   The evidence pack documents API key auth, SDKs, CLI, and an MCP server, but nowhere describes scoped, role-based, or least-privilege API key/credential issuance (e.g., per-agent or per-permission keys) for Retell. This is a fair axis for an API/voice-agent platform, but no docs or probes show scoped credential support, so it is unproven.

   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Retell provides official SDKs for Node.js and Python to simplify integration with our platform.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”
   • [claimed-docs] “Use Retell's MCP server to build and manage voice agents from MCP-capable clients like Cursor, Claude Desktop, and Claude Code via Retell AP…”

   proof ↗⚑ flag

   ElevenLabs Agents—none0/10

   Evidence covers agent-side authentication (protecting inbound access to a deployed agent) and OAuth for the hosted MCP server, but there is no documentation of issuing scoped or least-privilege API keys/tokens that limit what an agent itself can do via the ElevenLabs API. Missing for 10: any docs on API key permission scopes, workspace role-based tokens, or restricted-credential issuance for agents.

   • [claimed-docs] “Secure agent access | Authentication | Implement custom authentication for protected agent access”
   • [claimed-docs] “Implement custom authentication for protected agent access”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”

   proof ↗⚑ flag

8. ai-native userBuild against official SDKs

   weight 2 · round to Retell AI
   Retell AI✓fullprobed9/10

   Retell documents and ships official typed SDKs for Node.js and Python with API key auth, full endpoint coverage, structured errors, and TypeScript autocomplete, plus an official CLI and MCP server for programmatic/agentic access — all independently confirmed by runtime probes (CLI installs and runs, MCP server completes handshake). Missing for 10: no independent third-party review of SDK quality/DX beyond vendor docs and no public OpenAPI spec discovered.

   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Retell provides official SDKs for Node.js and Python to simplify integration with our platform.”
   • [claimed-docs] “Type safety: Full TypeScript support with autocomplete ... Error handling: Structured error responses with detailed messages”
   • [probe] “official CLI documented at https://docs.retellai.com/get-started/cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”
   • [probe] “official MCP server documented at https://docs.retellai.com/get-started/mcp-server”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”

   proof ↗⚑ flag

   ElevenLabs Agents~partialprobed4/10

   Docs confirm programmatic access via the ElevenLabs API and an official CLI (`@elevenlabs/cli`, verified runtime) for building agents, but no evidence pack item explicitly documents a first-party language SDK (Python/Node/etc.) for Agents beyond the generic API reference. Missing for 10: explicit official SDK docs/repos, SDK code examples, and independent confirmation of SDK feature parity with the API.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”

   proof ↗⚑ flag

9. ai-native userSubscribe to events via webhooks

   weight 2 · round to Retell AI
   Retell AI✓fullclaimed8/10

   Retell has dedicated first-party documentation for webhooks describing real-time event notifications pushed to your application as events occur, enabling event-driven integrations rather than polling — directly matching the story of subscribing to events via webhooks. Missing for 10: no independent/hands-on corroboration of webhook reliability or event catalog completeness beyond the docs themselves.

   • [claimed-docs] “Webhooks allow your application to receive real-time notifications about events that occur in your Retell AI account.”
   • [claimed-docs] “webhooks push data to your application as events happen, making your integrations more efficient and responsive.”

   proof ↗⚑ flag

   ElevenLabs Agents—none0/10

   The evidence describes 'webhook tools' that let an agent make outbound calls to external endpoints during a conversation (docs-14, docs-29, docs-31), which is the opposite of subscribing to platform-emitted events via webhooks. There is no documented mechanism for a developer to register a webhook URL to receive event notifications (e.g., conversation-ended, call-completed) from ElevenLabs Agents.

   • [claimed-docs] “Tools give your assistant the ability to make external function calls to third party apps so you can get real-time information.”
   • [claimed-docs] “Tools allow ElevenLabs agents to perform actions beyond generating text responses.”
   • [claimed-docs] “allow an assistant to trigger authenticated actions based on the conversation, like scheduling meetings or initiating order returns”

   proof ↗⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round to Retell AI
   Retell AI~partialclaimed6/10

   Retell's AI QA feature automatically scores calls on hallucination, knowledge-base accuracy, latency, sentiment, and tool usage to surface quality trends and issues, and custom analytics dashboards let users chart/filter call and chat metrics — both are AI-derived insights from the product's own call data. However, this is scoring/quality analytics rather than proactive generative 'suggestions' (e.g., recommended actions, next-best-response, or coaching tips), and there's no independent/hands-on evidence validating the accuracy or usefulness of these AI-generated insights. Missing for 10: evidence of proactive suggestion/recommendation generation beyond scoring, and third-party corroboration of AI QA insight quality.

   • [claimed-docs] “AI QA scores Retell calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage to surface quality trends and issues…”
   • [claimed-docs] “AI QA automatically evaluates a sampled set of your calls against rules and metrics you configure.”
   • [claimed-docs] “Build custom Retell analytics dashboards to track call and chat metrics like success rate, latency, cost, and concurrency, with charts, filt…”
   • [claimed-docs] “The Analytics dashboard charts your call and chat data so you can see how your agents are performing over time.”
   • [claimed-docs] “Built-in call success and sentiment scoring, plus custom fields synced to your CRM.”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed5/10

   Agent-analysis features (sentiment analysis, success evaluation, structured data extraction, semantic conversation search) generate AI-derived insights from the user's own conversation data, which partially satisfies the story. However, these are evaluation/reporting metrics rather than proactive AI-generated suggestions or recommendations for next actions, and there is no independent/hands-on corroboration of this analytics experience. Missing for 10: evidence of proactive suggestion generation (e.g., recommended actions/optimizations), a dashboard summarization view, and independent verification beyond first-party docs.

   • [claimed-docs] “Success evaluation: Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Data collection: Extract specific data points from conversations such as contact information, issue details, or any structured information”
   • [claimed-docs] “Sentiment analysis: Understand user sentiment across completed conversations”
   • [claimed-docs] “Searching conversations: Find messages by keyword or meaning across your conversation history.”
   • [claimed-docs] “Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Extract structured information from conversations such as contact details and business data”

   proof ↗⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Retell AI
   Retell AI~partialcommunity6/10

   Retell agents act autonomously during live calls — function calling lets them independently book appointments, send SMS, transfer calls, and call external APIs without human intervention (retell-docs-5, retell-docs-35), and webhooks push real-time events to downstream systems (retell-docs-12, retell-docs-24). This shows in-call autonomous action-taking, which is a form of background automation once a call is triggered. However, there's no evidence of standalone scheduled/triggered automations running independent of a live voice/chat session (e.g., cron-like outbound campaigns or autonomous multi-step workflows outside conversation context). Missing for 10: evidence of scheduled or event-triggered background automations outside live calls, and independent/hands-on confirmation that autonomous function-calling reliably completes tasks unattended (community reports show some confusion/looping during autonomous task execution, retell-comm-1, retell-comm-3, retell-comm-5).

   • [claimed-docs] “Function calling lets Retell single or multi-prompt agents take real actions — transfer calls, end calls, book appointments, send SMS, and c…”
   • [claimed-docs] “Function calling transforms your AI agent from a conversational interface into an action-oriented assistant.”
   • [claimed-docs] “Webhooks allow your application to receive real-time notifications about events that occur in your Retell AI account.”
   • [claimed-docs] “webhooks push data to your application as events happen, making your integrations more efficient and responsive.”
   • [community] “The AI contradicted itself when layering conditionals - it got confused about morning vs afternoon time, kept asking to repeat despite corre…”
   • [community] “Voice sounds great, but: told it unavailable until next year, it confirmed Feb 4th 'next year' but thought current year was 2022. Also got s…”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed4/10

   ElevenLabs Agents supports tool-calling, webhook actions, and multi-step workflows that let a voice agent perform actions (scheduling, API calls, MCP access) during a conversation, and agents can be created/managed via API for programmatic triggering, which give some automation flavor. But all documented action-taking is tied to a live conversational turn (voice/chat) rather than an autonomous background process that runs without a triggering conversation or human interaction, so the 'runs autonomously in the background' framing is only weakly supported. Missing for 10: evidence of scheduled/cron-triggered autonomous runs, evidence of multi-step tasks executing without a live conversation session, and independent confirmation of background automation outside the demoed conversational/tool-call flow.

   • [claimed-docs] “Build multi-step workflows with visual workflow builder”
   • [claimed-docs] “Enable agents to call clients & APIs to perform actions”
   • [claimed-docs] “Tools give your assistant the ability to make external function calls to third party apps so you can get real-time information.”
   • [claimed-docs] “allow an assistant to trigger authenticated actions based on the conversation, like scheduling meetings or initiating order returns”
   • [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers. This enables your agents to: Access and process informa…”
   • [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”

   proof ↗⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round to ElevenLabs Agents
   Retell AI~partialclaimed3/10

   Retell's dashboard includes 'Conductor', a built-in AI assistant that can draft an entire agent from a plain-English prompt (docs-30), which is a genuine instance of delegating a task to an in-product AI assistant. However this is a single thin mention with no further detail on scope, limits, or other delegable tasks beyond initial agent creation. Missing for 10: documentation of Conductor's full capabilities/limits, evidence of delegating other tasks (not just agent drafting) to a built-in assistant, and independent/hands-on corroboration of Conductor actually working.

   • [claimed-docs] “click **Generate from prompt** (marked Suggested) to let [Conductor](/conductor/create-agent) draft an agent from a plain-English descriptio…”

   proof ↗⚑ flag

   ElevenLabs Agents~partialprobed5/10

   ElevenLabs documents a first-party hosted MCP server letting an assistant like Claude create, configure, and manage agents in the workspace via natural language, with nothing to install locally — a form of task delegation to an AI assistant for managing the product. However this is not a native in-product chat/assistant UI; it requires an external MCP client (e.g., Claude) to act as the assistant, and delegation is scoped to agent-management tasks rather than general product tasks. Missing for 10: a built-in assistant surface inside the ElevenLabs dashboard itself, evidence of broader task delegation beyond agent CRUD, and independent hands-on confirmation of the natural-language workflow.

   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language”
   • [claimed-docs] “connect the hosted MCP server to Claude or any other MCP client to create and manage agents through natural language”
   • [claimed-docs] “create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”

   proof ↗⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round to ElevenLabs Agents
   Retell AI✓fullprobed7/10

   Retell supports natural-language operation via Conductor's 'Generate from prompt' feature that drafts an agent from a plain-English description, and via its official MCP server which exposes the full API as agent-callable tools so AI-native clients (Cursor, Claude Desktop, Claude Code) can build/manage agents through natural-language MCP tool calls — confirmed live by a keyless JSON-RPC probe against the hosted MCP endpoint. missing for 10: independent/hands-on evaluation of Conductor's prompt-to-agent quality, and no evidence of NL commands for other everyday operations (e.g., dashboard chat-based control) beyond agent creation/management.

   • [claimed-docs] “click **Generate from prompt** (marked Suggested) to let [Conductor](/conductor/create-agent) draft an agent from a plain-English descriptio…”
   • [claimed-docs] “Use Retell's MCP server to build and manage voice agents from MCP-capable clients like Cursor, Claude Desktop, and Claude Code via Retell AP…”
   • [claimed-docs] “Retell supports the Model Context Protocol (MCP) so you can build Retell AI voice agents directly from MCP-capable clients (Cursor, Claude D…”
   • [claimed-docs] “Retell supports the [Model Context Protocol (MCP)] so you can build Retell AI voice agents directly from MCP-capable clients (Cursor, Claude…”
   • [claimed-docs] “Agents: create, update, publish, list, and fetch agent versions.”
   • [probe] “official MCP server documented at https://docs.retellai.com/get-started/mcp-server”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”

   proof ↗⚑ flag

   ElevenLabs Agents✓fullprobed8/10

   ElevenLabs Agents ships a hosted MCP server explicitly designed so assistants like Claude can 'create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally,' and a runtime probe confirms the endpoint is live and speaks the MCP OAuth flow. This is a direct, first-party natural-language control surface, not just a client-side integration. Missing for 10: independent/community hands-on confirmation of a full natural-language agent-management session (only endpoint liveness was probed, not an actual successful NL task).

   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language”
   • [claimed-docs] “connect the hosted MCP server to Claude or any other MCP client to create and manage agents through natural language”
   • [claimed-docs] “create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”
   • [probe] “official MCP server documented at https://elevenlabs.io/docs/eleven-agents/operate/hosted-mcp”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   Retell AI—none0/10

   Evidence shows Retell has API reference doc pages (e.g., api-references/clone-voice.md) and SDKs/CLI/MCP tooling, but no evidence of an interactive, runnable API reference (e.g., embedded 'try it' console or Swagger UI). The probe explicitly found no OpenAPI/Swagger spec exposed (all candidate paths 404), undermining any claim of an interactive reference.

   • [claimed-docs] “Clone a voice from audio files”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.retellai.com/openapi.json, https://docs.retellai.com/swagger.json, https://docs.retella…”

   proof ↗⚑ flag

   ElevenLabs Agents—none0/10

   The evidence pack contains extensive markdown documentation for ElevenLabs Agents (quickstart, customization, tools, etc.) but nothing describes an interactive API reference page with runnable/'try it' examples — no mention of a Swagger/OpenAPI explorer, live code sandbox, or embedded runnable snippets. Since ElevenLabs Agents exposes an API, this axis is applicable, but no evidence supports the specific capability.

   ⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   Retell AI—none0/10

   A direct probe for standard OpenAPI/Swagger spec locations (openapi.json, swagger.json, etc.) on Retell's docs domain returned 404 for all candidates, and no evidence pack item shows a downloadable, machine-readable API spec being published or linked from docs. The CLI's version string mentions an internal 'OpenAPI 3.0.0' schema, but this is not shown to be an artifact users can download or fetch programmatically.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.retellai.com/openapi.json, https://docs.retellai.com/swagger.json, https://docs.retella…”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”

   proof ↗⚑ flag

   ElevenLabs Agents—none0/10

   The evidence pack documents the API, CLI, dashboard, and hosted MCP server for ElevenLabs Agents, but nowhere mentions a downloadable OpenAPI/Swagger spec or machine-readable schema for the API. This axis clearly applies to an API-driven product, but no evidence shows it being delivered.

   ⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round drawn
   Retell AI~partialclaimed4/10

   Retell's quick-start lets you click 'Test' to run a free web call with your agent before it ever touches a phone number or payment method, and new accounts get $10 trial credit to experiment risk-free — a lightweight way to try an agent without production telephony traffic. However, there's no documented dedicated sandbox environment, test-mode API flag, or explicit data isolation guarantee separating test calls from production records/analytics. Missing for 10: explicit sandbox/staging environment concept, test vs prod data isolation guarantees, and independent confirmation that test-call data doesn't mix with production analytics/QA.

   • [claimed-docs] “Click the "Test" button to start a web call with your agent... This step is free and doesn't need a phone number or payment method.”
   • [claimed-docs] “Click the "Test" button to start a web call with your agent”
   • [claimed-docs] “New accounts start with \$10 in free trial credits, no payment method required.”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed4/10

   Docs describe an 'agent testing' feature to verify conversational responses, tool usage, and multi-turn outcomes 'before you deploy', implying a pre-production test workflow, and tool execution can run in a 'sandboxed environment' on ElevenLabs' infrastructure. However there is no explicit documentation of a separate sandbox/staging environment or test data isolation from production conversation/analytics data. Missing for 10: explicit sandbox/staging environment separate from production data, documentation of test-vs-prod data isolation, independent confirmation that test runs don't affect production analytics or logs.

   • [claimed-docs] “Agent testing lets you verify conversational responses, tool usage, and full multi-turn outcomes before you deploy.”
   • [claimed-docs] “Agent testing lets you verify conversational responses, tool usage, and full multi-turn outcomes before you deploy”
   • [claimed-docs] “Custom JavaScript executed in a sandboxed environment on ElevenLabs' infrastructure.”
   • [claimed-docs] “Custom JavaScript executed in a sandboxed environment on ElevenLabs' infrastructure”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Retell AI—none0/10

   The evidence pack covers SDKs, CLI, MCP server, and API endpoints extensively, but there is no documentation of API versioning scheme or a deprecation policy for breaking changes. The probe notes an OpenAPI version string (3.0.0, catalog v4) but this is not evidence of a documented deprecation/versioning policy for API consumers.

   ⚑ flag

   ElevenLabs Agents—none0/10

   No evidence in the pack mentions API versioning scheme, version numbers, or any documented deprecation policy for ElevenLabs Agents' API/CLI/MCP interfaces. missing for 10: versioned API endpoints or version headers, changelog/release notes, explicit deprecation policy or sunset timelines, migration guides for breaking changes.

   ⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Retell AI
   Retell AI~partialprobed4/10

   Retell exposes a full-coverage SDK, CLI, and MCP server that let an AI agent script repeated single-item operations (create/update/list agents, phone numbers, knowledge bases) programmatically, enabling scripted bulk-like loops, but there is no documented native batch/bulk endpoint operating on many items in a single call. missing for 10: explicit batch/bulk API endpoints or CLI commands operating on multiple items in one call, evidence of rate-limit-safe bulk workflows, and hands-on confirmation of bulk usage at scale

   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”
   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Use Retell's MCP server to build and manage voice agents from MCP-capable clients like Cursor, Claude Desktop, and Claude Code via Retell AP…”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”
   • [claimed-docs] “Agents: create, update, publish, list, and fetch agent versions.”

   proof ↗⚑ flag

   ElevenLabs Agents—none0/10

   The evidence documents API, CLI, and MCP-based management of agents, but nothing describes bulk/batch operations (e.g., batch-create, batch-update, or batch-delete across many agents or conversations at once). This axis is fair to ask given the product exposes an API and CLI, but no evidence confirms bulk-operation support.

   ⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round drawn
   Retell AI~partialclaimed6/10

   Retell supports event-driven automation via webhooks that fire on account events (retell-docs-12, retell-docs-24), function calling that lets agents automatically transfer calls, end calls, book appointments, or call external APIs based on conversation logic (retell-docs-5, retell-docs-35), and node-based flows with conditional branching (retell-docs-25) plus AI QA rules evaluated against configured metrics (retell-docs-43). However, this is scattered across call-flow logic and webhook notifications rather than a unified 'if event then action' rule-definition interface. Missing for 10: a dedicated rules/trigger engine UI for arbitrary account-wide events, documentation of webhook-to-action automation chains, and independent verification that rule-based automation works reliably (community reports note conversational logic errors, e.g. retell-comm-1, retell-comm-3, retell-comm-7).

   • [claimed-docs] “Function calling lets Retell single or multi-prompt agents take real actions — transfer calls, end calls, book appointments, send SMS, and c…”
   • [claimed-docs] “Webhooks allow your application to receive real-time notifications about events that occur in your Retell AI account.”
   • [claimed-docs] “webhooks push data to your application as events happen, making your integrations more efficient and responsive.”
   • [claimed-docs] “Drag-and-drop, node-based flows for structured, high-stakes calls.”
   • [claimed-docs] “Function calling transforms your AI agent from a conversational interface into an action-oriented assistant.”
   • [claimed-docs] “AI QA automatically evaluates a sampled set of your calls against rules and metrics you configure.”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed6/10

   ElevenLabs Agents supports tool-calling and webhook actions that are triggered automatically based on conversation events (e.g., 'trigger authenticated actions based on the conversation, like scheduling meetings or initiating order returns'), plus multi-step visual workflows and MCP-based external actions. This covers event-triggered automation within a conversation but the evidence doesn't show general rule-based triggers on arbitrary system events (e.g., scheduled/external triggers outside a live conversation) — missing for 10: documentation of standalone event/rule engine independent of a live conversation, examples of conditional branching logic beyond tool calls, and independent verification of automation reliability.

   • [claimed-docs] “Enable agents to call clients & APIs to perform actions”
   • [claimed-docs] “Tools give your assistant the ability to make external function calls to third party apps so you can get real-time information.”
   • [claimed-docs] “allow an assistant to trigger authenticated actions based on the conversation, like scheduling meetings or initiating order returns”
   • [claimed-docs] “Build multi-step workflows with visual workflow builder”
   • [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers. This enables your agents to: Access and process informa…”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round drawn
   Retell AI—none0/10

   Retell's docs cover webhooks, function calling, MCP tool-calling, and telephony integration, but nothing describes scheduling recurring jobs, campaigns, or workflows (e.g., cron-like recurring outbound call batches or repeating automations). This is a fair question for a voice-agent platform (buyers often want scheduled/recurring outbound campaigns), so the axis applies, but no evidence in the pack shows this capability. missing for 10: any mention of a scheduler, recurring campaign/job feature, or cron-style automation trigger.

   ⚑ flag

   ElevenLabs Agents—none0/10

   The evidence pack covers multi-step workflow building, tool calling, MCP integration, and analysis, but no documentation mentions scheduling recurring jobs, cron-like triggers, or automated recurring workflow execution. This is a fair capability to expect from an automation-focused agent platform, but no evidence supports it.

   ⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round to Retell AI
   Retell AI~partialclaimed4/10

   Retell supports 'agent versions' with create/update/publish/list/fetch operations via CLI and MCP (retell-docs-47), implying some versioning and possibly rollback via republishing an older version, but there is no documented review workflow (diffs, approvals, changelogs) or explicit rollback command/UI. missing for 10: explicit rollback mechanism, version diff/review UI, changelog or audit trail, and any independent/hands-on confirmation that version history and rollback actually work as described.

   • [claimed-docs] “Agents: create, update, publish, list, and fetch agent versions.”
   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”
   • [claimed-docs] “Use Retell's MCP server to build and manage voice agents from MCP-capable clients like Cursor, Claude Desktop, and Claude Code via Retell AP…”

   proof ↗⚑ flag

   ElevenLabs Agents—none0/10

   The docs describe building, testing, and analyzing agents (agent-testing, agent-analysis) but there is no mention of version history, change review workflows, or rollback/revert capability for agent configurations. Missing for 10: version history UI, rollback/restore of prior agent configs, change-review/approval workflow, audit trail of edits.

   ⚑ flag

Compliance

1. founderMeet call-recording consent and disclosure obligations with per-call recording controls and configurable data retention

   weight 2 · round to Retell AI
   Retell AI~partialclaimed5/10

   Retell documents per-agent configurable data retention that auto-deletes transcripts, recordings, and logs after a set period, directly addressing retention obligations. However, there is no evidence of explicit per-call recording enable/disable controls or built-in consent-disclosure features (e.g., automated recording announcements) that founders would need for two-party consent compliance. Missing for 10: per-call recording on/off toggle, consent-disclosure/announcement mechanism, and any documentation tying these to legal consent requirements.

   • [claimed-docs] “Configure per-agent data retention to automatically delete call and chat data — transcripts, recordings, and logs — after a set period for c…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent. After the retention period expires, call and chat data associated with tha…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent.”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed4/10

   Docs confirm configurable data-retention settings for conversation transcripts and audio recordings, which addresses the retention half of the story, plus HIPAA/BAA support suggesting compliance posture. However, there is no evidence of per-call recording enable/disable controls or built-in consent/disclosure messaging mechanisms for calls. Missing for 10: per-call recording toggle, consent/disclosure prompt configuration, and any independent verification of retention enforcement.

   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings.”
   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings”
   • [claimed-docs] “ElevenLabs allows such customers to develop AI-powered voice agents for the handling Protected Health Information (PHI)”
   • [claimed-docs] “ElevenLabs Agents is one of ElevenLabs' HIPAA-eligible services, and we offer Business Associate Agreements (BAAs) to eligible customers”

   proof ↗⚑ flag

2. platform-engineerRun regulated workloads with HIPAA/BAA support, SOC 2, and data-residency options

   weight 2 · round to ElevenLabs Agents
   Retell AI—none0/10

   Evidence shows only generic data-retention controls (docs-15/27/44) but no mention of HIPAA, BAA, SOC 2 certification, or data-residency options anywhere in the docs or community pack, despite this being a fair compliance axis for a voice-AI platform handling call recordings/transcripts.

   • [claimed-docs] “Configure per-agent data retention to automatically delete call and chat data — transcripts, recordings, and logs — after a set period for c…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent. After the retention period expires, call and chat data associated with tha…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent.”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed5/10

   ElevenLabs Agents documents HIPAA eligibility and BAA availability for eligible customers, directly supporting regulated PHI workloads. However, there is no evidence in the pack of SOC 2 certification or data-residency options, both explicitly named in the story. Missing for 10: SOC 2 attestation/report evidence, data-residency region selection or documentation, independent audit corroboration of HIPAA/BAA claims.

   • [claimed-docs] “ElevenLabs allows such customers to develop AI-powered voice agents for the handling Protected Health Information (PHI)”
   • [claimed-docs] “ElevenLabs allows such customers to develop AI-powered voice agents for the handling Protected Health Information (PHI).”
   • [claimed-docs] “ElevenLabs Agents is one of ElevenLabs' HIPAA-eligible services, and we offer Business Associate Agreements (BAAs) to eligible customers”

   proof ↗⚑ flag

Scale

1. platform-engineerSee documented concurrency limits and scale to many simultaneous calls without manual capacity begging

   weight 2 · round to Retell AI
   Retell AI~partialclaimed5/10

   Retell documents a concurrency configuration feature (reserved inbound concurrency vs. outbound limit) showing platform-level concurrency controls exist, but there is no documented default/numeric concurrency limits per plan, no described self-service process to raise limits without contacting support, and no evidence (docs or community) confirming scaling to many simultaneous calls happens without manual requests. missing for 10: published default concurrency limits, self-service limit-increase mechanism, and independent confirmation of large-scale concurrent call handling without support intervention.

   • [claimed-docs] “Reserved inbound concurrency protects inbound calls from being crowded out by outbound traffic.”
   • [claimed-docs] “When `reserved_inbound_concurrency` is configured, outbound calls can use at most your concurrency limit minus the reserved amount.”

   proof ↗⚑ flag

   ElevenLabs Agents—none0/10

   No evidence pack item documents concurrency limits, per-account call caps, autoscaling behavior, or a process for scaling to many simultaneous calls without contacting sales/support; docs cover features like tools, voice customization, SIP trunking, HIPAA, and MCP but nothing on capacity/scale numbers or self-service scaling guarantees.

   ⚑ flag

Self host

1. platform-engineerSelf-host the voice agent runtime from open-source code on my own infrastructure

   weight 3 · round drawn
   Retell AI—none0/10

   Retell AI is a hosted SaaS voice-agent platform; all evidence points to cloud-hosted APIs, SDKs, CLI, and MCP server that call Retell's own servers, with no mention of open-source runtime code or self-hosting instructions. There is no evidence a platform-engineer can download and run the voice agent runtime on their own infrastructure.

   • [claimed-docs] “Build your first Retell AI phone agent in 15 minutes: create an account, pick a template, test in the dashboard, deploy to a phone number, a…”
   • [claimed-docs] “Retell starts at $0, pay only for what you use.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”

   proof ↗⚑ flag

   ElevenLabs Agents—none0/10

   ElevenLabs Agents is entirely a managed/hosted service — the CLI and MCP server are clients/interfaces to ElevenLabs' cloud infrastructure, not open-source runtime code that can be deployed on a platform-engineer's own servers. Nothing in the evidence pack mentions open-sourcing the runtime, self-hosting instructions, or on-prem deployment options; all references (dashboard, API, CLI, hosted MCP, SIP trunking) point to ElevenLabs-operated infrastructure.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
   • [claimed-docs] “create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”

   proof ↗⚑ flag

Latency

1. platform-engineerSee documented end-to-end voice latency numbers or tuning guidance backing the platform's speed claims

   weight 3 · round to Retell AI
   Retell AI~partialclaimed5/10

   Retell docs offer tuning guidance for latency (transcription mode trade-offs, endpointing settings) and let customers monitor latency via analytics dashboards and AI QA scoring, but there are no documented end-to-end latency benchmarks or numbers substantiating speed claims. missing for 10: published end-to-end latency figures/benchmarks, and independent verification of speed claims.

   • [claimed-docs] “Pick the right Retell transcription mode to balance accuracy against latency — interim results for speed or context-aware results for higher…”
   • [claimed-docs] “Real time transcription is often a trade off between latency and accuracy.”
   • [claimed-docs] “optimize for speed: uses the latest interim results with a low endpointing setting... optimize for accuracy: uses the results with a higher …”
   • [claimed-docs] “Build custom Retell analytics dashboards to track call and chat metrics like success rate, latency, cost, and concurrency, with charts, filt…”
   • [claimed-docs] “The Analytics dashboard charts your call and chat data so you can see how your agents are performing over time.”
   • [claimed-docs] “AI QA scores Retell calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage to surface quality trends and issues…”
   • [claimed-docs] “AI QA automatically evaluates a sampled set of your calls against rules and metrics you configure.”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed4/10

   ElevenLabs markets 'ultra-low latency' (docs-37) and does document conversation-flow tuning knobs like turn-taking, interruption handling and timeouts (docs-33, docs-42, docs-19/49/54), which counts as tuning guidance for latency/turn-taking behavior. However there are no documented end-to-end latency benchmark numbers (e.g., ms figures, percentile breakdowns) anywhere in the evidence pack. Missing for 10: concrete latency benchmark numbers/metrics, methodology for measuring end-to-end latency, and any independent corroboration of the 'ultra-low latency' marketing claim.

   • [claimed-docs] “Configure, deploy and monitor natural, human-sounding agents in 70+ languages with leading accuracy and ultra-low latency across voice or ch…”
   • [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”
   • [claimed-docs] “Configure turn-taking, interruptions, and timeout settings”
   • [claimed-docs] “The Max conversation duration setting limits the total time a conversation can remain active. ... The default is 600 seconds (10 minutes).”

   proof ↗⚑ flag

Turn taking

1. developerRely on the agent to handle interruptions (barge-in) gracefully — stopping speech, updating context, and recovering the turn

   weight 3 · round to ElevenLabs Agents
   Retell AI—none0/10

   The evidence pack covers transcription latency/accuracy trade-offs (retell-docs-9, retell-docs-39, retell-docs-48) but nowhere describes explicit barge-in/interruption handling — stopping agent speech, updating context, and recovering the turn when a caller interrupts. Community threads show turn-taking confusion (contradictions, loops, disconnects) but do not address interruption handling specifically. Missing for full/partial credit: any documentation or hands-on report confirming barge-in detection, speech-stop behavior, or turn recovery mechanics.

   ⚑ flag

   ElevenLabs Agents~partialclaimed6/10

   Docs explicitly document configuring turn-taking, interruptions, and timeout settings, and conversation-flow settings that 'determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavior,' confirming barge-in handling is a first-class configurable feature. However, evidence lacks detail on how context is updated after an interruption or how turn recovery specifically behaves, and there's no hands-on/independent validation of graceful recovery in practice. Missing for 10: technical detail on context truncation/update after interruption, independent or hands-on testing confirming graceful recovery, and latency benchmarks for barge-in specifically.

   • [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”
   • [claimed-docs] “Configure turn-taking, interruptions, and timeout settings”
   • [claimed-docs] “The Max conversation duration setting limits the total time a conversation can remain active. ... The default is 600 seconds (10 minutes).”

   proof ↗⚑ flag

2. developerEnable noise suppression or audio filtering so the agent stays coherent on noisy real-world calls

   weight 1 · round drawn
   Retell AI—none0/10

   No evidence pack item mentions noise suppression, background-noise filtering, or audio-preprocessing features for handling noisy real-world call environments; the closest related feature (transcription-mode trade-off between latency and accuracy) addresses turn-taking speed, not noise robustness. This is a plausible axis for a voice-agent platform, so absence of evidence yields 'none' rather than 'na'.

   ⚑ flag

   ElevenLabs Agents—none0/10

   The evidence pack covers voice customization, turn-taking/interruption settings, tools, testing, and telephony (SIP) integration, but nowhere mentions noise suppression, background noise handling, or audio filtering for noisy real-world calls. This is a fair capability to expect for a voice-agent platform handling phone calls, but no documentation or feature reference confirms it exists.

   • [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”
   • [claimed-docs] “Configure turn-taking, interruptions, and timeout settings”
   • [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”

   proof ↗⚑ flag

3. developerUse model-based end-of-turn detection beyond simple VAD silence timeouts so the agent doesn't talk over slow speakers

   weight 2 · round drawn
   Retell AI—none0/10

   The transcription-mode docs only describe tuning an 'endpointing setting' (low vs high) as a speed/accuracy tradeoff for VAD-style silence detection, not a distinct model-based semantic end-of-turn detector that understands incomplete utterances from slow speakers. No evidence pack item describes a dedicated turn-detection model or algorithm beyond adjusting silence-timeout thresholds.

   • [claimed-docs] “Pick the right Retell transcription mode to balance accuracy against latency — interim results for speed or context-aware results for higher…”
   • [claimed-docs] “Real time transcription is often a trade off between latency and accuracy.”
   • [claimed-docs] “optimize for speed: uses the latest interim results with a low endpointing setting... optimize for accuracy: uses the results with a higher …”

   proof ↗⚑ flag

   ElevenLabs Agents—none0/10

   Docs mention configurable turn-taking, interruption handling, and timeout settings (docs-33, docs-42), but nothing in the evidence describes a model-based end-of-turn detector that goes beyond a VAD/silence-timeout mechanism — no mention of semantic or ML-based turn-completion detection to avoid interrupting slow speakers. Missing for 10: any documentation of a dedicated end-of-turn prediction model, benchmarks or comparisons versus simple VAD timeouts, or hands-on evidence that the agent waits appropriately for slow speakers.

   • [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”
   • [claimed-docs] “Configure turn-taking, interruptions, and timeout settings”
   • [claimed-docs] “The Max conversation duration setting limits the total time a conversation can remain active. ... The default is 600 seconds (10 minutes).”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Retell AI
   Retell AI✓fullprobed8/10

   Retell exposes full API coverage via official Node/Python SDKs, a CLI, and a hosted MCP server that dynamically exposes the entire API (list/get/invoke endpoint tools), confirmed by keyless runtime probes actually reaching the CLI and MCP handshake — meaning nearly anything doable in the dashboard (agents, phone numbers, knowledge bases, function calling, voice cloning, analytics) is API/CLI/MCP accessible. missing for 10: no public OpenAPI spec was found (404s across candidate paths), and no independent hands-on confirmation that every single UI feature (e.g., analytics dashboards, AI QA config) has a 1:1 API equivalent.

   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”
   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Use Retell's MCP server to build and manage voice agents from MCP-capable clients like Cursor, Claude Desktop, and Claude Code via Retell AP…”
   • [claimed-docs] “Retell supports the Model Context Protocol (MCP) so you can build Retell AI voice agents directly from MCP-capable clients (Cursor, Claude D…”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.retellai.com/openapi.json, https://docs.retellai.com/swagger.json, https://docs.retella…”
   • [claimed-docs] “Agents: create, update, publish, list, and fetch agent versions.”

   proof ↗⚑ flag

   ElevenLabs Agents~partialprobed7/10

   Docs show agents can be created, configured, and managed via the API, CLI, dashboard, or hosted MCP server, and quickstart explicitly says agents can be created 'via the API or the web dashboard,' implying broad parity across surfaces (elevenlabs-agents-docs-1, docs-39, docs-44). However, there's no explicit feature-by-feature confirmation that every dashboard capability (e.g., visual workflow builder, analytics dashboards, testing UI) has a full API equivalent, nor independent verification of parity. Missing for 10: an explicit parity statement or changelog confirming all dashboard features (workflow builder, analytics, testing suite) are fully scriptable via API, and independent/hands-on confirmation of that parity.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”
   • [claimed-docs] “create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally”
   • [claimed-docs] “Build multi-step workflows with visual workflow builder”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round drawn
   Retell AI—none0/10

   Docs describe API/SDK/CLI access, data retention (auto-deletion) and analytics dashboards, but nowhere document a data-export or portability feature that would let a user pull all call/chat data, transcripts, and recordings in an open format to leave the platform; retention docs only cover automatic deletion, not export.

   • [claimed-docs] “Configure per-agent data retention to automatically delete call and chat data — transcripts, recordings, and logs — after a set period for c…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent. After the retention period expires, call and chat data associated with tha…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent.”
   • [claimed-docs] “Retell provides official SDKs for Node.js and Python to simplify integration with our platform.”
   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”

   proof ↗⚑ flag

   ElevenLabs Agents—none0/10

   No evidence describes a data export feature, open-format data dumps, or account deletion/portability tooling; only conversation retention settings are documented, not export mechanisms. Missing for 10: any documented export API/CLI command, supported open export formats (e.g., JSON/CSV), and confirmation of full data portability/account closure process.

   ⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round drawn
   Retell AI—none0/10

   Retell AI is a closed, proprietary SaaS platform; the evidence pack shows SDKs, CLI, MCP server and docs but no mention of source code being published under any open license, nor any GitHub repo for the core platform. Missing for 10: any open-source license grant, public source repository, or licensing terms for the core agent/voice engine.

   ⚑ flag

   ElevenLabs Agents—none0/10

   ElevenLabs Agents is a closed, proprietary SaaS product; the CLI is open on npm but no evidence indicates the core Agents platform/backend source is available under an open license. No license file, GitHub repo, or open-source statement is present in the evidence pack.

   ⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round drawn
   Retell AI—none0/10

   Retell AI is presented entirely as a hosted SaaS (cloud dashboard, hosted APIs, hosted MCP server, usage-based pricing) with no mention anywhere in docs of a self-hosted or on-prem deployment option; all evidence points to a fully managed cloud product.

   • [claimed-docs] “Retell starts at $0, pay only for what you use.”
   • [claimed-docs] “Build your first Retell AI phone agent in 15 minutes: create an account, pick a template, test in the dashboard, deploy to a phone number, a…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”
   • [claimed-docs] “Integrate Retell voice agents with your own telephony provider using elastic SIP trunking or imported numbers from Twilio, Telnyx, and Vonag…”

   proof ↗⚑ flag

   ElevenLabs Agents—none0/10

   ElevenLabs Agents is entirely a hosted SaaS product—managed via dashboard, API, CLI, and hosted MCP server, all pointing to ElevenLabs' own infrastructure; there is no evidence of an open-source release, downloadable server binary, or self-hosting option for the core voice-agent runtime. missing for 10: any open-source repo or self-hosted deployment package, docs describing running the core service on one's own infrastructure, independent confirmation of self-hosting.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
   • [claimed-docs] “create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally”

   proof ↗⚑ flag

Pricing

1. founderSee published per-minute or usage pricing and estimate cost per call before committing

   weight 2 · round to Retell AI
   Retell AI✓fullcommunity8/10

   Retell's pricing page explicitly states pay-as-you-go pricing starting at $0 with no annual contracts, offers an interactive calculator to estimate approximate monthly cost by adjusting key parameters, and new accounts get $10 free trial credit to test before paying — directly enabling a founder to estimate call costs before committing. Community discussion also confirms a concrete per-minute rate (17¢/min) is being charged in practice, corroborating usage-based billing exists. missing for 10: the per-minute rate itself is not shown in first-party docs (only inferred from a community comment), and there's no independent verification of the calculator's accuracy.

   • [claimed-docs] “Retell starts at $0, pay only for what you use.”
   • [claimed-docs] “Adjust key parameters to see an approximate monthly cost for your Retell AI voice agents.”
   • [claimed-docs] “Most voice AI platforms require annual contracts before you write a single line of code. Retell starts at $0, pay only for what you use.”
   • [claimed-docs] “New accounts start with \$10 in free trial credits, no payment method required.”
   • [community] “Founder (yanyan_evie) noted voice provider costs differ: 'The different providers have different prices. openai tts & deepgram are cheaper, …”
   • [community] “A commenter noted per-minute pricing (17¢/min) could be exploited: a malicious caller could keep the bot on the line for hours across many p…”

   proof ↗⚑ flag

   ElevenLabs Agents—none0/10

   No evidence item in the pack references pricing, per-minute rates, usage-based billing, or cost calculators for ElevenLabs Agents; all citations are product/docs feature descriptions. Since founders evaluating a voice-agent SaaS could reasonably expect published pricing, the axis applies but is entirely unaddressed in this evidence pack.

   ⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round drawn
   Retell AI—none0/10

   No evidence in the pack mentions data residency, region selection, or geographic storage options; the closest related feature is per-agent data retention/deletion (retell-docs-15/27/44), which addresses retention duration, not location of storage. This axis is applicable to a cloud AI voice platform handling call/chat data, so absence of evidence yields 'none'. Missing for 10: any documentation of selectable regions, data residency guarantees, or geo-specific hosting.

   • [claimed-docs] “Configure per-agent data retention to automatically delete call and chat data — transcripts, recordings, and logs — after a set period for c…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent. After the retention period expires, call and chat data associated with tha…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent.”

   proof ↗⚑ flag

   ElevenLabs Agents—none0/10

   Evidence covers data retention duration settings, HIPAA/BAA eligibility, and infrastructure details, but nothing addresses selectable data region/residency options for storage location. missing for 10: any mention of region selection, data residency controls, or geographic storage options.

   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings.”
   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings”
   • [claimed-docs] “ElevenLabs Agents is one of ElevenLabs' HIPAA-eligible services, and we offer Business Associate Agreements (BAAs) to eligible customers”

   proof ↗⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   Retell AI—none0/10

   The evidence only covers configurable data retention (auto-deleting transcripts/recordings/logs after a set period) for compliance, not an explicit opt-out or guarantee against using customer data for AI model training. No docs mention training-data usage policy, opt-out toggles, or contractual no-train clauses.

   • [claimed-docs] “Configure per-agent data retention to automatically delete call and chat data — transcripts, recordings, and logs — after a set period for c…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent. After the retention period expires, call and chat data associated with tha…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent.”

   proof ↗⚑ flag

   ElevenLabs Agents—none0/10

   The evidence pack covers data retention settings (how long transcripts/audio are stored) and HIPAA/BAA compliance, but nowhere mentions an explicit opt-out or policy regarding use of customer data/conversations for AI model training. missing for 10: explicit training-data opt-out policy or setting, terms-of-service language on model training use, any statement distinguishing enterprise vs free-tier data usage for training.

   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings.”
   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings”
   • [claimed-docs] “ElevenLabs Agents is one of ElevenLabs' HIPAA-eligible services, and we offer Business Associate Agreements (BAAs) to eligible customers”

   proof ↗⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round to ElevenLabs Agents
   Retell AI!disputedcontradicted5/10

   Retell's docs describe per-agent configurable data retention with automatic, permanent deletion of call/chat transcripts, recordings, and logs after a set period (retell-docs-15, retell-docs-27, retell-docs-44), which is solid first-party evidence for the retention-control axis. However, an independent community report describes the company refusing to delete a user's stored data (credit card) on request, forcing the user to escalate to bank disputes/consumer-affairs complaints — a concrete real-world case where deletion did not work as a customer expected (retell-comm-14). Missing for 10: independent verification that call/chat data deletion itself (not just billing data) works as documented, and no public response/resolution to the deletion complaint.

   • [claimed-docs] “Configure per-agent data retention to automatically delete call and chat data — transcripts, recordings, and logs — after a set period for c…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent. After the retention period expires, call and chat data associated with tha…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent.”
   • [community] “Thread titled 'Retellai won't delete my credit card' - a user complained the company would not delete their stored credit card; commenters s…”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed6/10

   Docs explicitly document configurable retention settings for conversation transcripts and audio recordings, giving users control over how long data is stored. However, evidence does not detail a deletion mechanism (e.g., data subject deletion requests, API/CLI commands to purge stored data) beyond retention duration settings. Missing for 10: explicit deletion/export controls, granular per-conversation deletion API, and independent verification of retention enforcement.

   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings.”
   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings”

   proof ↗⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Retell AI—none0/10

   The evidence pack covers data retention for call/chat data (transcripts, recordings, logs) but contains no mention of telemetry or usage-tracking opt-out controls for the product itself (e.g., CLI/SDK/dashboard analytics collection). Data retention (retell-docs-15/27/44) addresses deletion of customer call data, not opting out of Retell's own telemetry collection.

   ⚑ flag

   ElevenLabs Agents—none0/10

   The evidence pack covers conversation transcript/audio retention settings (docs-26, docs-52) but nothing about opting out of product telemetry or usage tracking (e.g., CLI analytics, dashboard usage metrics) for the AI-native user. Retention configuration is about customer conversation data, not the vendor's own telemetry collection, so this axis has no supporting evidence.

   ⚑ flag

Call control

1. developerEscalate a live call to a human with warm or blind transfer, passing context along

   weight 2 · round to Retell AI
   Retell AI~partialclaimed4/10

   Retell's function-calling docs list 'transfer calls' as a built-in agent action alongside ending calls, booking appointments, etc., which supports the general concept of escalating a call to a human. However, the evidence never distinguishes warm vs. blind transfer modes nor describes passing conversational context/metadata to the receiving human agent. missing for 10: explicit warm-transfer vs blind-transfer configuration, evidence of context/data hand-off to the human agent, and any hands-on confirmation of this feature working in practice.

   • [claimed-docs] “Function calling lets Retell single or multi-prompt agents take real actions — transfer calls, end calls, book appointments, send SMS, and c…”
   • [claimed-docs] “Function calling transforms your AI agent from a conversational interface into an action-oriented assistant.”

   proof ↗⚑ flag

   ElevenLabs Agents—none0/10

   The evidence pack covers telephony features like SIP trunking, tools, webhooks, and MCP integrations, but nowhere documents call transfer (warm or blind) to a human agent or passing conversation context during handoff. Missing for 10: any mention of transfer_to_number/transfer_to_agent tool types, warm/blind transfer configuration, or context-passing during escalation.

   ⚑ flag

2. developerMy agent can send DTMF keypresses, navigate IVR menus, and detect or leave voicemail

   weight 1 · round drawn
   Retell AI—none0/10

   The evidence pack covers function calling, custom telephony/SIP trunking, transcription, and MCP tool integration, but nowhere mentions DTMF keypress sending, IVR menu navigation, or voicemail detection/leaving capabilities. This is a fair capability question for a telephony voice-agent platform, but no evidence confirms it is supported.

   ⚑ flag

   ElevenLabs Agents—none0/10

   The evidence pack confirms ElevenLabs Agents supports telephony integration via SIP trunking and phone numbers, so the telephony axis clearly applies, but nothing in the docs mentions DTMF tone sending, IVR menu navigation, or voicemail detection/leaving capabilities.

   • [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”
   • [claimed-docs] “Route calls to AI agents without changing your existing phone infrastructure”

   proof ↗⚑ flag

Campaigns

1. founderRun batch outbound call campaigns with scheduling and throughput controls

   weight 2 · round drawn
   Retell AI—none0/10

   Docs describe outbound vs inbound concurrency reservation (retell-docs-11, retell-docs-41) and custom telephony/SIP integration, but there is no evidence of a batch outbound campaign feature — no mention of scheduling calls, uploading contact lists, or campaign-level throughput controls beyond general concurrency limits. Axis is plausible for a voice-agent platform but unsupported by the evidence pack.

   • [claimed-docs] “Reserved inbound concurrency protects inbound calls from being crowded out by outbound traffic.”
   • [claimed-docs] “When `reserved_inbound_concurrency` is configured, outbound calls can use at most your concurrency limit minus the reserved amount.”
   • [claimed-docs] “Integrate Retell voice agents with your own telephony provider using elastic SIP trunking or imported numbers from Twilio, Telnyx, and Vonag…”

   proof ↗⚑ flag

   ElevenLabs Agents—none0/10

   The evidence pack documents phone-number/SIP trunking integration for routing calls to agents, but contains no mention of batch outbound calling, campaign scheduling, or throughput/rate controls for outbound call volume. Since telephony campaign management is a fair capability to expect from a voice-agent telephony product, absence of evidence here is 'none' rather than 'na'.

   • [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”
   • [claimed-docs] “Route calls to AI agents without changing your existing phone infrastructure”

   proof ↗⚑ flag

Numbers

1. developerProvision phone numbers and run both inbound and outbound calls through the platform's API

   weight 3 · round to Retell AI
   Retell AI✓fullprobed8/10

   Retell's docs show phone-number provisioning via native assignment (docs-32) or importing/SIP-trunking your own numbers from Twilio/Telnyx/Vonage (docs-10,40,49), and both inbound and outbound calling are explicitly supported and distinguished (reserved inbound concurrency vs. outbound traffic in docs-11/41). Full API/SDK coverage for voice endpoints (docs-2,33,46) and a CLI to manage phone numbers (docs-3,21) round out programmatic control, with a working keyless CLI/MCP probe corroborating API-level access (retell-probe-rt-1/2). Missing for 10: explicit documentation of a native 'buy a new number' provisioning endpoint (as opposed to importing existing numbers) and a concrete outbound-call code example.

   • [claimed-docs] “Build your first Retell AI phone agent in 15 minutes: create an account, pick a template, test in the dashboard, deploy to a phone number, a…”
   • [claimed-docs] “Integrate Retell voice agents with your own telephony provider using elastic SIP trunking or imported numbers from Twilio, Telnyx, and Vonag…”
   • [claimed-docs] “Reserved inbound concurrency protects inbound calls from being crowded out by outbound traffic.”
   • [claimed-docs] “This guide shows how to integrate Retell agents with your telephony provider and use your own numbers.”
   • [claimed-docs] “When `reserved_inbound_concurrency` is configured, outbound calls can use at most your concurrency limit minus the reserved amount.”
   • [claimed-docs] “You set up a SIP trunk, configure your number to point to it, and import that number to Retell.”
   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Retell provides official SDKs for Node.js and Python to simplify integration with our platform.”
   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”
   • [claimed-docs] “The Retell CLI lets you manage Retell resources from your terminal.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed6/10

   Docs confirm phone/telephony integration via SIP trunking, letting agents route inbound/outbound calls over existing phone infrastructure, and agents can be created/configured via the API. However, evidence does not explicitly show a documented API endpoint for provisioning new phone numbers or explicitly triggering outbound calls programmatically vs. via SIP trunk setup. Missing for 10: explicit phone-number provisioning API endpoint docs, explicit outbound-call-initiation API example, and independent/hands-on confirmation of inbound+outbound call flows.

   • [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”
   • [claimed-docs] “Route calls to AI agents without changing your existing phone infrastructure”
   • [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”

   proof ↗⚑ flag

Sip

1. platform-engineerConnect my own carrier or PBX via SIP trunking (or import Twilio/Telnyx numbers) instead of being locked to bundled telephony

   weight 2 · round to Retell AI
   Retell AI✓fullclaimed8/10

   Official docs explicitly describe elastic SIP trunking and importing numbers from Twilio, Telnyx, and Vonage, walking through setting up a SIP trunk and pointing/importing numbers into Retell — directly matching the story. This is first-party documentation without independent hands-on corroboration of the SIP flow itself. Missing for 10: independent/community verification of a real SIP trunk setup working end-to-end, and detail on carrier-specific edge cases (codecs, failover, latency).

   • [claimed-docs] “Integrate Retell voice agents with your own telephony provider using elastic SIP trunking or imported numbers from Twilio, Telnyx, and Vonag…”
   • [claimed-docs] “This guide shows how to integrate Retell agents with your telephony provider and use your own numbers.”
   • [claimed-docs] “You set up a SIP trunk, configure your number to point to it, and import that number to Retell.”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed7/10

   Docs explicitly describe a SIP trunking integration that lets customers connect their existing phone systems/carrier infrastructure to ElevenLabs agents without changing carriers, directly addressing the core ask of bringing your own PBX/carrier. However, the pack has no explicit mention of importing existing Twilio/Telnyx numbers or detailed carrier onboarding steps, and no independent/hands-on confirmation of a live SIP trunk setup working end-to-end. Missing for 10: explicit Twilio/Telnyx number import instructions, and third-party/hands-on validation of the SIP trunking flow.

   • [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”
   • [claimed-docs] “Route calls to AI agents without changing your existing phone infrastructure”

   proof ↗⚑ flag

Analytics

1. ai-native userThe platform's AI reviews my calls for me — scoring quality, flagging failures, and analyzing resolution automatically

   weight 2 · round to ElevenLabs Agents
   Retell AI~partialclaimed6/10

   Retell's AI QA feature automatically scores sampled calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage, and the Analytics dashboard tracks success rate and other metrics, directly matching the story's core ask of automated call scoring and quality flagging. However, evidence shows AI QA only evaluates a 'sampled set' of calls (not all calls) against user-configured rules, and there's no documented automatic 'resolution analysis' narrative beyond metrics/dashboards, nor independent hands-on validation of AI QA accuracy. missing for 10: evidence of full (non-sampled) coverage, independent/third-party validation of AI QA scoring accuracy, and detail on automated resolution/failure-cause analysis beyond configured metrics.

   • [claimed-docs] “AI QA scores Retell calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage to surface quality trends and issues…”
   • [claimed-docs] “AI QA automatically evaluates a sampled set of your calls against rules and metrics you configure.”
   • [claimed-docs] “Build custom Retell analytics dashboards to track call and chat metrics like success rate, latency, cost, and concurrency, with charts, filt…”
   • [claimed-docs] “The Analytics dashboard charts your call and chat data so you can see how your agents are performing over time.”
   • [claimed-docs] “Built-in call success and sentiment scoring, plus custom fields synced to your CRM.”

   proof ↗⚑ flag

   ElevenLabs Agents✓fullclaimed7/10

   Docs explicitly describe automated post-call analysis: 'Success evaluation' scores conversation quality/goal achievement, 'Sentiment analysis' flags user sentiment, and 'Data collection' extracts structured resolution details — directly matching the story's scoring/flagging/resolution-analysis ask. Missing for 10: no independent/hands-on validation of analysis accuracy, and no explicit 'failure flagging' terminology beyond success-criteria evaluation.

   • [claimed-docs] “Success evaluation: Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Data collection: Extract specific data points from conversations such as contact information, issue details, or any structured information”
   • [claimed-docs] “Sentiment analysis: Understand user sentiment across completed conversations”
   • [claimed-docs] “Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Extract structured information from conversations such as contact details and business data.”

   proof ↗⚑ flag

2. founderSee call analytics — success rates, durations, outcomes, sentiment — in dashboards without building my own

   weight 2 · round to Retell AI
   Retell AI✓fullclaimed8/10

   Retell docs describe a built-in Analytics dashboard with charts, filters, and breakdowns for success rate, latency, cost, and concurrency, plus AI QA scoring for sentiment, hallucination, and tool usage—directly matching the founder's need for out-of-box call analytics. Missing for 10: independent/hands-on confirmation of dashboard usability and no evidence of exportable reports or deeper outcome breakdowns beyond what's documented.

   • [claimed-docs] “Build custom Retell analytics dashboards to track call and chat metrics like success rate, latency, cost, and concurrency, with charts, filt…”
   • [claimed-docs] “The Analytics dashboard charts your call and chat data so you can see how your agents are performing over time.”
   • [claimed-docs] “AI QA scores Retell calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage to surface quality trends and issues…”
   • [claimed-docs] “AI QA automatically evaluates a sampled set of your calls against rules and metrics you configure.”
   • [claimed-docs] “Built-in call success and sentiment scoring, plus custom fields synced to your CRM.”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed6/10

   Docs confirm built-in agent-analysis features—success evaluation, sentiment analysis, data extraction, and conversation search—managed via the ElevenAgents dashboard, which covers most of the story's asked-for metrics without custom-building analytics. However, there's no explicit evidence of a dashboard showing aggregated success-rate percentages or call-duration analytics/rollups across calls, only per-conversation criteria and settings. Missing for 10: dashboard screenshots/evidence of aggregate success-rate metrics, call-duration reporting/rollups, and independent corroboration that these appear in a unified analytics view.

   • [claimed-docs] “Success evaluation: Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Data collection: Extract specific data points from conversations such as contact information, issue details, or any structured information”
   • [claimed-docs] “Sentiment analysis: Understand user sentiment across completed conversations”
   • [claimed-docs] “Searching conversations: Find messages by keyword or meaning across your conversation history.”
   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “Extract structured information from conversations such as contact details and business data.”

   proof ↗⚑ flag

Monitoring

1. platform-engineerMonitor live calls in production and get alerts when agents misbehave or error rates spike

   weight 1 · round to Retell AI
   Retell AI~partialclaimed6/10

   Retell provides real building blocks for production monitoring — a customizable analytics dashboard tracking success rate, latency, cost and concurrency (retell-docs-13/42), AI QA that scores calls for hallucination, sentiment and tool-usage issues (retell-docs-14/43), and webhooks that push real-time event notifications (retell-docs-12/24) which a platform engineer could wire into an alerting pipeline. However there is no documented native alerting/threshold system (e.g., automatic notification when error rates spike or an agent misbehaves) or a live in-call monitoring view — engineers must build that themselves on top of webhooks/dashboard APIs. Missing for 10: built-in threshold-based alerts or anomaly detection, a real-time 'in-progress calls' monitoring view, and any independent evidence that alerting/monitoring works reliably in production.

   • [claimed-docs] “Webhooks allow your application to receive real-time notifications about events that occur in your Retell AI account.”
   • [claimed-docs] “Build custom Retell analytics dashboards to track call and chat metrics like success rate, latency, cost, and concurrency, with charts, filt…”
   • [claimed-docs] “AI QA scores Retell calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage to surface quality trends and issues…”
   • [claimed-docs] “webhooks push data to your application as events happen, making your integrations more efficient and responsive.”
   • [claimed-docs] “The Analytics dashboard charts your call and chat data so you can see how your agents are performing over time.”
   • [claimed-docs] “AI QA automatically evaluates a sampled set of your calls against rules and metrics you configure.”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed4/10

   Docs show post-call analytics (sentiment analysis, success evaluation, data extraction, conversation search) and a marketing claim to 'monitor' agents, but there is no evidence of live/real-time call monitoring dashboards, error-rate metrics, or alerting/notification mechanisms for misbehaving agents. Missing for 10: real-time monitoring dashboard, error-rate/latency metrics, alerting/notification system, on-call integration (e.g., PagerDuty/Slack alerts).

   • [claimed-docs] “Success evaluation: Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Sentiment analysis: Understand user sentiment across completed conversations”
   • [claimed-docs] “Searching conversations: Find messages by keyword or meaning across your conversation history.”
   • [claimed-docs] “Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Configure, deploy and monitor natural, human-sounding agents in 70+ languages with leading accuracy and ultra-low latency across voice or ch…”

   proof ↗⚑ flag

Testing

1. developerTest agents with simulated conversations or evals before putting them on real phone calls

   weight 2 · round to ElevenLabs Agents
   Retell AI~partialclaimed5/10

   Retell's dashboard has a free 'Test' web-call button that lets developers try an agent before assigning it to a real phone number, and AI QA can score calls on hallucination, accuracy, sentiment, etc., which supports some testing/analytics workflow. However there's no documented feature for automated simulated-conversation test suites or eval scripts run pre-deployment — AI QA appears to operate on sampled real calls rather than synthetic scripted evals. Missing for 10: dedicated simulation/eval framework for scripted test conversations, batch eval tooling, and any independent verification that pre-call testing catches issues before production use.

   • [claimed-docs] “Click the "Test" button to start a web call with your agent... This step is free and doesn't need a phone number or payment method.”
   • [claimed-docs] “Click the "Test" button to start a web call with your agent”
   • [claimed-docs] “AI QA scores Retell calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage to surface quality trends and issues…”
   • [claimed-docs] “AI QA automatically evaluates a sampled set of your calls against rules and metrics you configure.”
   • [claimed-docs] “Build custom Retell analytics dashboards to track call and chat metrics like success rate, latency, cost, and concurrency, with charts, filt…”

   proof ↗⚑ flag

   ElevenLabs Agents✓fullclaimed8/10

   ElevenLabs documents an explicit Agent Testing feature that verifies conversational responses, tool usage, and full multi-turn outcomes before deployment, including converting real conversations into simulated test cases, plus success-evaluation criteria for assessing conversation quality/goal achievement — directly matching the pre-deployment simulated testing story. Missing for 10: no independent/hands-on report validating test accuracy or CI-integration details, and no explicit mention of automated eval scoring pipelines beyond docs description.

   • [claimed-docs] “Agent testing lets you verify conversational responses, tool usage, and full multi-turn outcomes before you deploy.”
   • [claimed-docs] “Transform real conversations into test cases when you find an interaction where the agent underperformed.”
   • [claimed-docs] “Agent testing lets you verify conversational responses, tool usage, and full multi-turn outcomes before you deploy”
   • [claimed-docs] “Success evaluation: Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”

   proof ↗⚑ flag

Post call

1. developerExtract structured data from every call — outcomes, entities, dispositions — delivered via API or webhook after the call

   weight 2 · round to Retell AI
   Retell AI✓fullclaimed7/10

   Retell explicitly supports post-call structured extraction: built-in call success/sentiment scoring plus custom fields synced to CRM, delivered via real-time webhooks and viewable/aggregated in the analytics/AI QA dashboards. This directly covers outcomes (success rate), dispositions (sentiment), and entities (custom fields) delivered via API/webhook as the story requires. Missing for 10: no independent/hands-on confirmation of the specific post-call-analysis JSON schema or webhook payload structure, and no detail on how custom entity fields are defined/configured.

   • [claimed-docs] “Built-in call success and sentiment scoring, plus custom fields synced to your CRM.”
   • [claimed-docs] “Webhooks allow your application to receive real-time notifications about events that occur in your Retell AI account.”
   • [claimed-docs] “webhooks push data to your application as events happen, making your integrations more efficient and responsive.”
   • [claimed-docs] “Build custom Retell analytics dashboards to track call and chat metrics like success rate, latency, cost, and concurrency, with charts, filt…”
   • [claimed-docs] “The Analytics dashboard charts your call and chat data so you can see how your agents are performing over time.”
   • [claimed-docs] “AI QA scores Retell calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage to surface quality trends and issues…”
   • [claimed-docs] “AI QA automatically evaluates a sampled set of your calls against rules and metrics you configure.”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed6/10

   Docs show structured post-call outputs (data collection with contact/business data extraction, success-evaluation criteria, sentiment analysis) available via 'agent-analysis' features, and the platform is manageable via API. However, the pack never explicitly documents a webhook payload/callback delivering this structured analysis after a call, only webhook *tools* for in-call actions — missing for 10: explicit post-call webhook delivery mechanism/schema for outcomes and dispositions, and confirmation these analysis fields are retrievable via a documented API endpoint.

   • [claimed-docs] “Success evaluation: Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Data collection: Extract specific data points from conversations such as contact information, issue details, or any structured information”
   • [claimed-docs] “Sentiment analysis: Understand user sentiment across completed conversations”
   • [claimed-docs] “Extract structured information from conversations such as contact details and business data”
   • [claimed-docs] “Extract structured information from conversations such as contact details and business data.”
   • [claimed-docs] “Tools give your assistant the ability to make external function calls to third party apps so you can get real-time information.”
   • [claimed-docs] “allow an assistant to trigger authenticated actions based on the conversation, like scheduling meetings or initiating order returns”

   proof ↗⚑ flag

Tools

1. ai-native userMy voice agent can plug in MCP servers as tool sources so one integration grants it whole toolsets mid-call

   weight 2 · round drawn
   Retell AI✓fullclaimed8/10

   Retell documents connecting single/multi-prompt voice agents to remote MCP servers so the agent can call the server's tools mid-call (retell-docs-6/23/36), directly matching the story, and this is distinct from Retell's own client-side MCP server for building agents. missing for 10: independent third-party corroboration of mid-call MCP tool invocation in production (only first-party docs and a probe of the client-facing MCP server, not the agent-as-MCP-client tool-call path, are available).

   • [claimed-docs] “Connect a Retell single- or multi-prompt agent to a remote MCP server so it can call the server's tools during a live voice or chat conversa…”
   • [claimed-docs] “Connect your single- or multi-prompt agent to a remote Model Context Protocol (MCP) server, and the agent can call that server's tools durin…”
   • [claimed-docs] “Connect your single- or multi-prompt agent to a remote [Model Context Protocol (MCP)] server, and the agent can call that server's tools dur…”
   • [claimed-docs] “Function calling lets Retell single or multi-prompt agents take real actions — transfer calls, end calls, book appointments, send SMS, and c…”
   • [claimed-docs] “Function calling transforms your AI agent from a conversational interface into an action-oriented assistant.”

   proof ↗⚑ flag

   ElevenLabs Agents✓fullclaimed8/10

   Docs explicitly describe connecting conversational agents to external MCP servers as a tool source, enabling access to data/processing capabilities via that server mid-call, directly matching the story. Missing for 10: independent hands-on confirmation of mid-call multi-tool retrieval from a connected MCP server (only vendor docs, no third-party validation) and detail on how many tools/toolsets a single MCP connection exposes.

   • [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers. This enables your agents to: Access and process informa…”
   • [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers.”
   • [claimed-docs] “Access and process information from various data sources via the MCP server”

   proof ↗⚑ flag

2. developerMy agent can call external APIs and custom functions mid-conversation and speak the result without awkward dead air

   weight 3 · round to ElevenLabs Agents
   Retell AI~partialprobed6/10

   Retell's docs clearly document function calling that lets agents call external APIs mid-conversation (retell-docs-5, retell-docs-35) and even connect to remote MCP servers to invoke tools live during a call (retell-docs-6, retell-docs-23, retell-docs-36), confirmed by a keyless runtime probe showing the hosted MCP server exposing invoke_api_endpoint tools (retell-probe-rt-2). However, no evidence specifically addresses the 'no awkward dead air' requirement — there's no documented filler-speech, streaming-response, or latency-masking mechanism specifically tied to function-call execution; transcription-mode docs (retell-docs-9/39/48) discuss general STT latency tradeoffs, not function-call dead-air handling. missing for 10: explicit filler/backchannel speech during function execution, hands-on demonstration of seamless mid-call tool latency, independent confirmation of no dead-air behavior.

   • [claimed-docs] “Function calling lets Retell single or multi-prompt agents take real actions — transfer calls, end calls, book appointments, send SMS, and c…”
   • [claimed-docs] “Function calling transforms your AI agent from a conversational interface into an action-oriented assistant.”
   • [claimed-docs] “Connect a Retell single- or multi-prompt agent to a remote MCP server so it can call the server's tools during a live voice or chat conversa…”
   • [claimed-docs] “Connect your single- or multi-prompt agent to a remote Model Context Protocol (MCP) server, and the agent can call that server's tools durin…”
   • [claimed-docs] “Connect your single- or multi-prompt agent to a remote [Model Context Protocol (MCP)] server, and the agent can call that server's tools dur…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”

   proof ↗⚑ flag

   ElevenLabs Agents✓fullclaimed7/10

   Docs clearly describe multiple tool-calling mechanisms — webhook tools for external API calls, client-side tools, sandboxed custom JS, and MCP server integration — explicitly for making external function calls mid-conversation to fetch real-time info and trigger authenticated actions. Conversation-flow settings (turn-taking, interruption handling, ultra-low latency) address smooth conversational pacing, though not explicitly framed as avoiding dead air during tool execution. Missing for 10: explicit documentation of filler/hold-speech behavior while awaiting API results, and independent/hands-on verification that tool calls don't introduce audible pauses.

   • [claimed-docs] “Enable agents to call clients & APIs to perform actions”
   • [claimed-docs] “Tools executed directly on the client-side application (e.g., web browser, mobile app).”
   • [claimed-docs] “Tools give your assistant the ability to make external function calls to third party apps so you can get real-time information.”
   • [claimed-docs] “Custom JavaScript executed in a sandboxed environment on ElevenLabs' infrastructure.”
   • [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers. This enables your agents to: Access and process informa…”
   • [claimed-docs] “Tools allow ElevenLabs agents to perform actions beyond generating text responses.”
   • [claimed-docs] “allow an assistant to trigger authenticated actions based on the conversation, like scheduling meetings or initiating order returns”
   • [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”
   • [claimed-docs] “Configure, deploy and monitor natural, human-sounding agents in 70+ languages with leading accuracy and ultra-low latency across voice or ch…”

   proof ↗⚑ flag

Recording

1. platform-engineerRetrieve full call recordings and transcripts programmatically for every call

   weight 2 · round to Retell AI
   Retell AI~partialclaimed6/10

   Docs confirm that Retell stores per-call transcripts and recordings (referenced in data-retention docs) and that full voice/chat API endpoint coverage exists via official SDKs, implying programmatic retrieval, but no evidence pack item explicitly documents a 'get call' or 'list calls' API endpoint returning recording URLs/transcript text, nor examples of pulling them via SDK/CLI. Missing for 10: explicit API reference for call/recording/transcript retrieval endpoints, CLI/SDK code samples showing recording download or transcript fetch, and any independent confirmation of this working end-to-end.

   • [claimed-docs] “Configure per-agent data retention to automatically delete call and chat data — transcripts, recordings, and logs — after a set period for c…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent. After the retention period expires, call and chat data associated with tha…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent.”
   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Retell provides official SDKs for Node.js and Python to simplify integration with our platform.”
   • [claimed-docs] “Webhooks allow your application to receive real-time notifications about events that occur in your Retell AI account.”
   • [claimed-docs] “webhooks push data to your application as events happen, making your integrations more efficient and responsive.”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed5/10

   Docs confirm that conversation transcripts and audio recordings are stored with configurable retention, and that conversations can be searched, analyzed, and have structured data extracted (docs-26/52, docs-21/23/56), implying underlying storage of call recordings/transcripts. However, there is no explicit documentation of a specific API endpoint or method for programmatically retrieving full call recordings and transcripts on a per-call basis. Missing for 10: an explicit 'get conversation' or 'get recording/transcript' API reference, example request/response showing programmatic export of audio + transcript per call.

   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings.”
   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings”
   • [claimed-docs] “Data collection: Extract specific data points from conversations such as contact information, issue details, or any structured information”
   • [claimed-docs] “Searching conversations: Find messages by keyword or meaning across your conversation history.”
   • [claimed-docs] “Extract structured information from conversations such as contact details and business data.”

   proof ↗⚑ flag

Transcription

1. developerGet accurate real-time transcription with control over the STT provider, language models, or key terms

   weight 2 · round drawn
   Retell AI~partialcommunity4/10

   Retell docs confirm real-time transcription with a documented latency/accuracy trade-off (interim vs context-aware endpointing), giving developers some control over accuracy tuning, and community notes hint at underlying provider choices (e.g., Deepgram) for voice pipelines. However, there is no documented ability to select or swap STT providers, choose an ASR language model, or configure custom vocabulary/key-term boosting for transcription accuracy. Missing for 10: explicit STT provider selection API, language model choice for transcription, custom vocabulary/key-term boosting support, and independent accuracy benchmarks.

   • [claimed-docs] “Pick the right Retell transcription mode to balance accuracy against latency — interim results for speed or context-aware results for higher…”
   • [claimed-docs] “Real time transcription is often a trade off between latency and accuracy.”
   • [claimed-docs] “optimize for speed: uses the latest interim results with a low endpointing setting... optimize for accuracy: uses the results with a higher …”
   • [community] “Founder (yanyan_evie) noted voice provider costs differ: 'The different providers have different prices. openai tts & deepgram are cheaper, …”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed4/10

   Docs confirm agents let developers choose or bring custom LLMs (docs-41) and mention real-time voice interaction (docs-37), but there is no evidence of selectable STT providers, transcription accuracy tuning, or 'key terms'/vocabulary boosting controls anywhere in the pack. Missing for 10: STT provider selection, transcription-specific accuracy settings, key-term/glossary support, and any independent verification of transcription accuracy claims.

   • [claimed-docs] “Choose from supported LLMs or bring your own custom model”
   • [claimed-docs] “Configure, deploy and monitor natural, human-sounding agents in 70+ languages with leading accuracy and ultra-low latency across voice or ch…”
   • [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”

   proof ↗⚑ flag

Voices

1. founderClone a custom brand voice and use it for my agents, with a documented consent process

   weight 2 · round to Retell AI
   Retell AI~partialclaimed5/10

   Retell documents voice cloning from uploaded audio files (up to 25 files) and lets you attach the cloned voice to an agent via the voice selector, covering the core 'clone and use a custom voice' capability. However, none of the evidence describes any documented consent-verification step (e.g., consent recording, rights attestation) as part of the cloning flow. Missing for 10: an explicit consent-collection/verification mechanism in the clone-voice API or dashboard docs, and any compliance guidance tying voice cloning to consent requirements.

   • [claimed-docs] “Add custom voices to your Retell agent — search ElevenLabs community voices, import a voice clone, or train a clone for a unique brand-speci…”
   • [claimed-docs] “Clone a voice from audio files”
   • [claimed-docs] “Audio files to use for voice cloning. Up to 25 files allowed.”
   • [claimed-docs] “You can also add a voice clone by clicking "Add custom voice" in the voice selector.”

   proof ↗⚑ flag

   ElevenLabs Agents—none0/10

   The evidence pack describes selecting from 5k+ pre-built voices and voice customization (speed, multi-voice switching) but contains no mention of a voice-cloning feature for creating a custom brand voice, nor any documented consent process for cloning (e.g., voice verification, spoken consent statement) required by ElevenLabs' broader platform. Missing for 10: any docs on voice cloning/Instant or Professional Voice Cloning for Agents, and any consent-verification workflow.

   • [claimed-docs] “Select from 5k+ voices across 31 languages with customization options”
   • [claimed-docs] “Enable your agent to switch between different voices for multi-character conversations, storytelling, and language tutoring.”
   • [claimed-docs] “You can customize various aspects of your AI agent's voice to create a more natural and engaging conversation experience. This includes cont…”

   proof ↗⚑ flag

2. developerChoose from a broad voice library or plug in multiple TTS providers to get the voice I want

   weight 2 · round to Retell AI
   Retell AI✓fullcommunity7/10

   Docs confirm a searchable voice library (ElevenLabs community voices) plus voice cloning options (retell-docs-7, retell-docs-37, retell-docs-38), and a founder community comment independently confirms multiple underlying TTS providers (OpenAI TTS, Deepgram, ElevenLabs) with different pricing tiers (retell-comm-10), showing developers can indeed pick across providers/voices. Missing for 10: a first-party docs page enumerating all supported TTS providers and API-level provider-switching parameters beyond the voice-selector UI, and independent hands-on comparison of voice quality across providers.

   • [claimed-docs] “Add custom voices to your Retell agent — search ElevenLabs community voices, import a voice clone, or train a clone for a unique brand-speci…”
   • [claimed-docs] “You can also add a voice clone by clicking "Add custom voice" in the voice selector.”
   • [claimed-docs] “In the voice selector, you can click "Add custom voice" to search and add publicly available community voices.”
   • [community] “Founder (yanyan_evie) noted voice provider costs differ: 'The different providers have different prices. openai tts & deepgram are cheaper, …”

   proof ↗⚑ flag

   ElevenLabs Agents~partialclaimed5/10

   Docs confirm a broad in-house voice library (5k+ voices across 31 languages, docs-6) with customization like speed, pronunciation, and multi-voice switching (docs-17/32/48), satisfying the first half of the story. However, there is no evidence of plugging in third-party/alternative TTS providers — the docs only mention bring-your-own LLM (docs-41), not bring-your-own TTS engine. Missing for 10: any documented support for external/third-party TTS provider integration, evidence of provider-agnostic voice pipeline, independent confirmation of using non-ElevenLabs voices.

   • [claimed-docs] “Select from 5k+ voices across 31 languages with customization options”
   • [claimed-docs] “Enable your agent to switch between different voices for multi-character conversations, storytelling, and language tutoring.”
   • [claimed-docs] “You can customize various aspects of your AI agent's voice to create a more natural and engaging conversation experience. This includes cont…”
   • [claimed-docs] “Enable your agent to switch between different voices for multi-character conversations, storytelling, and language tutoring”
   • [claimed-docs] “Choose from supported LLMs or bring your own custom model”

   proof ↗⚑ flag