Retell AI vs Pipecat

Agent ops

1. ai-native userMy coding agent can provision a complete voice agent end to end — create the agent, attach a number, and place a call — through the API, CLI, or MCP without touching the dashboard

   weight 3 · round to Retell AI
   Retell AI~partialprobed6/10

   Retell ships an official Node/Python SDK with full voice endpoint coverage, a CLI for managing agents and phone numbers, and an MCP server whose meta-tools (list/get/invoke_api_endpoint) expose the entire API — including agent create/update/publish — to MCP clients like Cursor or Claude Code, and a keyless runtime probe confirms the MCP handshake works end-to-end. However, the flagship quick-start walkthrough is dashboard-centric (create in UI, assign number in 'configuration settings', test/call via dashboard button), and no evidence explicitly shows a CLI/API/MCP call sequence that attaches a number and places a live call without touching the UI. Missing for 10: an explicit end-to-end CLI/MCP example showing number-attach and call-placement commands, and independent confirmation that non-dashboard number provisioning/outbound calling works in practice.

   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”
   • [claimed-docs] “Use Retell's MCP server to build and manage voice agents from MCP-capable clients like Cursor, Claude Desktop, and Claude Code via Retell AP…”
   • [claimed-docs] “Retell supports the Model Context Protocol (MCP) so you can build Retell AI voice agents directly from MCP-capable clients (Cursor, Claude D…”
   • [claimed-docs] “Agents: create, update, publish, list, and fetch agent versions.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”
   • [claimed-docs] “Assign your agent to the number in the configuration settings”

   proof ↗⚑ flag

   Pipecat~partialprobed3/10

   Pipecat provides a CLI (`pipecat init`, `pipecat cloud`, `pipecat context-hub`) and code-level telephony hooks (`python bot.py -t twilio -x ngrok...`) that let a coding agent scaffold and run a bot, and the context-hub can register an MCP server for coding-agent tooling, but there is no documented single API/CLI/MCP flow that creates the agent, provisions/attaches a phone number, and places a call — telephony setup still relies on the carrier's (Twilio/Telnyx) own console/API for number purchase and webhook config. missing for 10: a unified provisioning API or CLI command that attaches a phone number and places a call, MCP-driven agent creation (context-hub only builds a docs index, not agent provisioning), evidence of number attachment without touching a carrier dashboard.

   • [claimed-docs] “Telephony bots have a different shape than WebRTC bots. The session doesn't start because a client sent your dispatcher an HTTP request — it…”
   • [claimed-docs] “the carrier (Twilio, Telnyx, Plivo, Exotel, or your SIP provider) is calling your webhook to tell you there's an inbound call”
   • [claimed-docs] “python bot.py -t twilio -x your-name.ngrok.io”
   • [claimed-docs] “pipecat init quickstart # Change to the project directory cd pipecat-quickstart”
   • [claimed-docs] “pipecat init quickstart”
   • [claimed-docs] “pipecat context-hub install registers the hub as an MCP server with your coding agent and builds the local index”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Pipecat CLI (pypi pipecat-ai[cli]) ran keylessly via uvx — `pipecat --help` lists init (pr…”

   proof ↗⚑ flag

2. ai-native userThe platform's own AI helps me author agents — generating or improving prompts, flows, and test cases from a description

   weight 1 · round to Retell AI
   Retell AI~partialclaimed5/10

   Retell docs mention a 'Generate from prompt' feature where Conductor drafts an agent from a plain-English description (retell-docs-30), directly supporting AI-assisted agent authoring. However, there's no evidence of AI-driven improvement of existing prompts/flows or automatic test-case generation, and no independent/hands-on confirmation of Conductor's output quality. Missing for 10: documentation on iterative prompt/flow refinement by the platform's AI, automated test-case generation, and community or hands-on verification of Conductor's generated agents.

   • [claimed-docs] “click **Generate from prompt** (marked Suggested) to let [Conductor](/conductor/create-agent) draft an agent from a plain-English descriptio…”
   • [claimed-docs] “Retell agents are node-based flows or single prompts, with call handling, a knowledge base, and integrations built in.”
   • [claimed-docs] “Drag-and-drop, node-based flows for structured, high-stakes calls.”
   • [claimed-docs] “Single prompt agent... Define your whole agent with one prompt.”

   proof ↗⚑ flag

   Pipecat—none0/10

   Evidence shows Pipecat has a context-hub that registers as an MCP server so external coding agents (e.g. Claude) can get context about the framework, and an Evals system for testing agent behavior — but these rely on the user writing prompts/flows/tests and connecting third-party AI tools, not Pipecat's own AI generating or improving prompts, flows, or test cases from a description. No evidence of an in-platform AI assistant that authors agent configs.

   • [claimed-docs] “pipecat context-hub install registers the hub as an MCP server with your coding agent and builds the local index”
   • [claimed-docs] “pipecat context-hub install` registers the hub as an MCP server with your coding agent and builds the local index”
   • [claimed-docs] “Pipecat Evals is the framework's built-in system for testing agent behavior. You describe a conversation and the behavior you expect, and Pi…”
   • [claimed-docs] “You describe a conversation and the behavior you expect, and Pipecat runs it against your real agent (the same pipeline, the same services, …”

   proof ↗⚑ flag

Build

1. developerBuild a working phone voice agent — prompt, voice, and phone number — and take my first live call within an hour

   weight 3 · round to Retell AI
   Retell AI✓fullcommunity8/10

   Docs walk through the exact flow described: create account, pick/generate a template, set prompt and voice, test in dashboard, assign a phone number, and make a live call, with a 15-minute quickstart and free trial credits removing payment friction. Community feedback confirms the demo/agent works end-to-end for real calls, though it also shows occasional conversational glitches (contradictions, confusion) that are quality issues rather than build-flow blockers. missing for 10: independent third-party timing confirmation that a full build-to-first-call truly completes within an hour, and evidence of phone-number provisioning speed in practice.

   • [claimed-docs] “Build your first Retell AI phone agent in 15 minutes: create an account, pick a template, test in the dashboard, deploy to a phone number, a…”
   • [claimed-docs] “Click the "Test" button to start a web call with your agent... This step is free and doesn't need a phone number or payment method.”
   • [claimed-docs] “click **Generate from prompt** (marked Suggested) to let [Conductor](/conductor/create-agent) draft an agent from a plain-English descriptio…”
   • [claimed-docs] “Click the "Test" button to start a web call with your agent”
   • [claimed-docs] “Assign your agent to the number in the configuration settings”
   • [claimed-docs] “New accounts start with \$10 in free trial credits, no payment method required.”
   • [claimed-docs] “Add custom voices to your Retell agent — search ElevenLabs community voices, import a voice clone, or train a clone for a unique brand-speci…”
   • [community] “Curious what model the dentist bot is running on? Tried it out, was surprisingly good, though eventually it contradicted itself (booked a sl…”
   • [community] “I tried to fool the Dental Office demo trying to book on Sunday or outside of the slots it had indicated, and it did a better job than many …”

   proof ↗⚑ flag

   Pipecat~partialprobed7/10

   Pipecat's quickstart gets a browser voice bot running in ~5 minutes with prompt+voice configurable across 200+ providers, and the telephony guide shows the same bot file working with real carriers (Twilio/Telnyx/Plivo) via a one-line command with ngrok tunneling, so all three pieces (prompt, voice, phone number) are documented and connectable to the same codebase. However, the telephony path is a separate 'production' doc requiring carrier account setup/ngrok tunneling not folded into the 5-minute quickstart, and there's no first-party or independent account of someone going from zero to a live phone call within an hour. Missing for 10: an end-to-end timed walkthrough combining phone-number provisioning with the quickstart, and independent hands-on confirmation of the 'first live call within an hour' claim.

   • [claimed-docs] “Build and run your first Pipecat voice AI bot in under 5 minutes”
   • [claimed-docs] “Build and run your first Pipecat voice AI bot in under 5 minutes: a simple conversational agent you can talk to in a browser.”
   • [claimed-docs] “python bot.py -t twilio -x your-name.ngrok.io”
   • [claimed-docs] “the carrier (Twilio, Telnyx, Plivo, Exotel, or your SIP provider) is calling your webhook to tell you there's an inbound call”
   • [claimed-docs] “Telephony bots have a different shape than WebRTC bots. The session doesn't start because a client sent your dispatcher an HTTP request — it…”
   • [claimed-docs] “Swap speech, language, and vision services from any of 200+ integrated providers and services — usually one line of code.”
   • [claimed-docs] “Any Transport: WebRTC, SIP, PSTN”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Pipecat CLI (pypi pipecat-ai[cli]) ran keylessly via uvx — `pipecat --help` lists init (pr…”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi pipecat-ai 1.8.1 (BSD-2 OSS) installs and `import pipecat` succeeds with no key, printing its star…”

   proof ↗⚑ flag

2. developerRun conversations in multiple languages, including detecting and switching language mid-call

   weight 2 · round drawn
   Retell AI—none0/10

   No documentation in the evidence pack describes multi-language conversation support, language detection, or mid-call language switching; the only related evidence is a founder comment stating multilingual support is merely 'on our roadmap' rather than shipped, indicating the capability does not currently exist.

   • [community] “Founder said multilingual support is on roadmap: 'It's definitely in our roadmap. After the core product—the voice AI part—becomes humanlike…”

   proof ↗⚑ flag

   Pipecat—none0/10

   The evidence mentions swapping speech/language services across providers (one line of code) but nothing about detecting a spoken language mid-call or dynamically switching STT/TTS/LLM language settings within an active conversation. No docs, community reports, or runtime probes address multilingual or language-switching behavior. missing for 10: language detection mechanism, mid-call language switching mechanism, any docs/demo showing multilingual conversation handling.

   • [claimed-docs] “Swap speech, language, and vision services from any of 200+ integrated providers and services — usually one line of code.”

   proof ↗⚑ flag

3. founderDesign multi-step conversation flows in a visual builder with branching, states, and handoffs without writing code

   weight 2 · round to Retell AI
   Retell AI✓fullclaimed7/10

   Retell's docs describe node-based, drag-and-drop conversation flows for 'structured, high-stakes calls' as an alternative to single-prompt agents, with function calling enabling transfers, call-ending, and API calls that serve as handoff points — directly matching the branching/states/handoff story without requiring code. The quick-start flow (pick template, test in dashboard, deploy) reinforces a no-code workflow. Missing for 10: independent/hands-on evidence of the visual builder's branching UI itself (community evidence only covers conversational behavior, not the builder), and no detail on how 'states' are represented/connected beyond the general node-based description.

   • [claimed-docs] “Retell agents are node-based flows or single prompts, with call handling, a knowledge base, and integrations built in.”
   • [claimed-docs] “Drag-and-drop, node-based flows for structured, high-stakes calls.”
   • [claimed-docs] “Single prompt agent... Define your whole agent with one prompt.”
   • [claimed-docs] “Function calling lets Retell single or multi-prompt agents take real actions — transfer calls, end calls, book appointments, send SMS, and c…”
   • [claimed-docs] “Function calling transforms your AI agent from a conversational interface into an action-oriented assistant.”
   • [claimed-docs] “Build your first Retell AI phone agent in 15 minutes: create an account, pick a template, test in the dashboard, deploy to a phone number, a…”
   • [claimed-docs] “click **Generate from prompt** (marked Suggested) to let [Conductor](/conductor/create-agent) draft an agent from a plain-English descriptio…”

   proof ↗⚑ flag

   Pipecat—none0/10

   Pipecat Flows lets you define branching conversation states via Python functions/config (docs-6, docs-22, docs-39), but this is a code-first framework with a CLI and Python SDK, not a visual drag-and-drop builder — no evidence pack item mentions any GUI/no-code flow designer. Since competing voice-agent platforms do offer visual builders, this is a fair axis to ask about, and Pipecat's evidence shows the opposite (functions/config authored in code).

   • [claimed-docs] “Functions in Pipecat Flows serve two key purposes: 1. Process data ... 2. Progress the conversation by transitioning between nodes”
   • [claimed-docs] “Functions in Pipecat Flows serve two key purposes: Process data by interfacing with external systems and APIs...Progress the conversation by…”
   • [claimed-docs] “Flows auto-derives the function's metadata — name, description, parameter properties (with their descriptions), and which parameters are req…”
   • [claimed-docs] “Hand off to subagents for long-running tools and complex tasks, and use Pipecat Flows when a conversation needs to follow a defined path.”

   proof ↗⚑ flag

Personalization

1. developerInject dynamic variables and per-caller context at call time so each conversation is personalized

   weight 2 · round drawn
   Retell AI—none0/10

   The evidence pack covers function calling, webhooks, custom telephony, knowledge base retrieval, CRM field sync, and SDK/CLI/MCP tooling, but nowhere describes injecting dynamic variables or per-caller context (e.g., a startCall/create-call parameter for passing caller-specific data into the prompt/LLM at runtime). This is a fair, plausible capability for a voice-agent platform, so absence of evidence yields 'none' rather than 'na'.

   ⚑ flag

   Pipecat—none0/10

   Evidence shows Pipecat Flows functions can process data and transition conversation state, and telephony bots receive caller info via webhook, but nothing documents a mechanism for injecting per-caller dynamic variables/context into the bot's prompt or state at call start for personalization. This is a fair capability to expect from a voice-agent framework, but no citation directly demonstrates it.

   • [claimed-docs] “Functions in Pipecat Flows serve two key purposes: 1. Process data ... 2. Progress the conversation by transitioning between nodes”
   • [claimed-docs] “Telephony bots have a different shape than WebRTC bots. The session doesn't start because a client sent your dispatcher an HTTP request — it…”
   • [claimed-docs] “Flows auto-derives the function's metadata — name, description, parameter properties (with their descriptions), and which parameters are req…”

   proof ↗⚑ flag

2. developerGround the agent on my documents with a built-in knowledge base or RAG so it answers from my content

   weight 2 · round to Retell AI
   Retell AI~partialclaimed6/10

   Retell explicitly ships a knowledge base feature where you can crawl websites or upload documents for the agent to retrieve from, and AI QA scores 'knowledge base accuracy' as a call metric, confirming grounding is a first-class capability. However, evidence is thin — only brief homepage-level mentions rather than a dedicated deep-dive doc on KB architecture/RAG internals, and there's no independent/hands-on confirmation of retrieval quality. Missing for 10: dedicated knowledge-base documentation page detailing chunking/retrieval mechanics, and independent verification that answers are accurately grounded in uploaded content.

   • [claimed-docs] “Retell agents are node-based flows or single prompts, with call handling, a knowledge base, and integrations built in.”
   • [claimed-docs] “Crawl a website or upload documents your agent retrieves from.”
   • [claimed-docs] “AI QA scores Retell calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage to surface quality trends and issues…”

   proof ↗⚑ flag

   Pipecat—none0/10

   Pipecat's docs describe voice/multimodal pipeline orchestration, function calling via Flows, telephony, evals, and a context-hub for coding-agent tooling — but nothing about a built-in knowledge base or RAG mechanism for grounding the agent on user documents. The closest hit (context-hub) indexes Pipecat's own docs/API for coding assistants, not user content for the voice agent.

   • [claimed-docs] “Functions in Pipecat Flows serve two key purposes: 1. Process data ... 2. Progress the conversation by transitioning between nodes”
   • [claimed-docs] “pipecat context-hub install registers the hub as an MCP server with your coding agent and builds the local index”
   • [claimed-docs] “pipecat context-hub install` registers the hub as an MCP server with your coding agent and builds the local index”

   proof ↗⚑ flag

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round drawn
   Retell AI✓fullprobed9/10

   A direct probe confirms an llms.txt file exists at docs.retellai.com/llms.txt returning HTTP 200 with a structured index of the docs, and Retell's docs are also agent-oriented enough to support MCP-based discovery/management of resources. This directly satisfies the story of pointing an agent at llms.txt or agent-oriented docs. Missing for 10: no independent third-party corroboration of llms.txt usage in the wild.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.retellai.com/llms.txt # Retell AI > Explore Retell AI docs to learn how to build, test, deploy, an…”
   • [claimed-docs] “Use Retell's MCP server to build and manage voice agents from MCP-capable clients like Cursor, Claude Desktop, and Claude Code via Retell AP…”

   proof ↗⚑ flag

   Pipecat✓fullprobed9/10

   Pipecat provides a working llms.txt endpoint (confirmed HTTP 200 with agent-oriented summary) plus an explicit context-hub CLI feature that registers docs as an MCP server for coding agents and builds a local index, directly enabling agents to consume its docs programmatically. missing for 10: no independent/community confirmation of an agent successfully using llms.txt or context-hub end-to-end.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.pipecat.ai/llms.txt # Pipecat > Pipecat is an open source ecosystem for building voice and multimo…”
   • [claimed-docs] “pipecat context-hub install registers the hub as an MCP server with your coding agent and builds the local index”
   • [claimed-docs] “pipecat context-hub install` registers the hub as an MCP server with your coding agent and builds the local index”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Pipecat CLI (pypi pipecat-ai[cli]) ran keylessly via uvx — `pipecat --help` lists init (pr…”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to Retell AI
   Retell AI~partialprobed7/10

   Retell ships official Node.js/Python SDKs with API-key auth and a terminal CLI for managing agents, phone numbers, and resources, both of which are scriptable outside the dashboard, and a runtime probe confirms the CLI installs and runs keylessly via npm — all consistent with headless/CI use. However, there is no explicit CI/automation documentation (e.g., GitHub Actions example, testing-in-pipeline guide) confirming an officially supported headless workflow. Missing for 10: explicit CI/pipeline documentation or example, confirmation of non-interactive auth flow for CI secrets, and independent evidence of real-world CI usage.

   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”
   • [claimed-docs] “The Retell CLI lets you manage Retell resources from your terminal.”
   • [claimed-docs] “Retell provides official SDKs for Node.js and Python to simplify integration with our platform.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”
   • [probe] “official CLI documented at https://docs.retellai.com/get-started/cli”

   proof ↗⚑ flag

   Pipecat~partialprobed6/10

   Pipecat is a Python framework that is pip-installable with no key required and runs bots as scripts (bot.py) with a CLI supporting init, cloud deploy, and eval subcommands, which supports headless/CI-friendly automation; the Evals system explicitly runs conversations against the real agent programmatically, suitable for CI. However there is no explicit CI/CD example, no documented headless flag or non-interactive mode guidance, and the development runner is framed around local dev/demo UI rather than CI. missing for 10: explicit CI pipeline documentation/example, confirmation that pipecat.runner.run or bot.py can run fully headless without any UI/browser dependency, independent hands-on report of running Pipecat in an automated CI environment.

   • [claimed-docs] “Pipecat Evals is the framework's built-in system for testing agent behavior. You describe a conversation and the behavior you expect, and Pi…”
   • [claimed-docs] “You describe a conversation and the behavior you expect, and Pipecat runs it against your real agent (the same pipeline, the same services, …”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Pipecat CLI (pypi pipecat-ai[cli]) ran keylessly via uvx — `pipecat --help` lists init (pr…”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi pipecat-ai 1.8.1 (BSD-2 OSS) installs and `import pipecat` succeeds with no key, printing its star…”
   • [claimed-docs] “Pipecat ships a built-in development runner (pipecat.runner.run) that handles the server-side glue most bots need during development: creati…”
   • [claimed-docs] “python bot.py -t twilio -x your-name.ngrok.io”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round to Retell AI
   Retell AI✓fullclaimed8/10

   Retell explicitly documents connecting a live voice/chat agent to a remote MCP server so it can invoke that server's tools during calls, directly matching the story of plugging in external MCP servers for tool use. This is a first-party, well-specified capability (not just Retell exposing its own API as MCP) covering both single- and multi-prompt agents. Missing for 10: independent/hands-on verification that a third-party MCP server's tools work reliably mid-call in production.

   • [claimed-docs] “Connect a Retell single- or multi-prompt agent to a remote MCP server so it can call the server's tools during a live voice or chat conversa…”
   • [claimed-docs] “Connect your single- or multi-prompt agent to a remote Model Context Protocol (MCP) server, and the agent can call that server's tools durin…”
   • [claimed-docs] “Connect your single- or multi-prompt agent to a remote [Model Context Protocol (MCP)] server, and the agent can call that server's tools dur…”
   • [claimed-docs] “Function calling transforms your AI agent from a conversational interface into an action-oriented assistant.”

   proof ↗⚑ flag

   Pipecat—none0/10

   The evidence only shows Pipecat's context-hub CLI *exposing itself* as an MCP server to a coding agent (docs-11/38) — the reverse of what the story asks (Pipecat consuming external MCP servers to gain their tools inside its voice-agent pipelines). No evidence shows Pipecat's Flows/function-calling system connecting to arbitrary MCP servers as a client.

   • [claimed-docs] “pipecat context-hub install registers the hub as an MCP server with your coding agent and builds the local index”
   • [claimed-docs] “pipecat context-hub install` registers the hub as an MCP server with your coding agent and builds the local index”

   proof ↗⚑ flag

4. ai-native userConnect an agent via an official MCP server

   weight 3 · round to Retell AI
   Retell AI✓fullprobed9/10

   Retell documents and hosts an official MCP server (mcp.retellai.com) that lets MCP-capable clients like Cursor, Claude Desktop, and Claude Code build and manage Retell voice agents via the Retell API, exposing meta-tools for listing/invoking endpoints. A runtime probe confirms the server completes a full keyless JSON-RPC initialize handshake and exposes its tool set, corroborating the vendor docs. Missing for 10: independent/community usage reports of the MCP server in practice beyond vendor docs and the probe.

   • [claimed-docs] “Use Retell's MCP server to build and manage voice agents from MCP-capable clients like Cursor, Claude Desktop, and Claude Code via Retell AP…”
   • [claimed-docs] “Retell supports the Model Context Protocol (MCP) so you can build Retell AI voice agents directly from MCP-capable clients (Cursor, Claude D…”
   • [claimed-docs] “Retell supports the [Model Context Protocol (MCP)] so you can build Retell AI voice agents directly from MCP-capable clients (Cursor, Claude…”
   • [claimed-docs] “Agents: create, update, publish, list, and fetch agent versions.”
   • [probe] “official MCP server documented at https://docs.retellai.com/get-started/mcp-server”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”

   proof ↗⚑ flag

   Pipecat~partialprobed6/10

   Pipecat ships an official `pipecat context-hub install` CLI command that registers itself as an MCP server with a coding agent, building a local docs/examples index — confirmed both in docs and via a runtime CLI probe showing the `context-hub` subcommand exists. This is a genuine official MCP server for agent connection, but it's scoped narrowly to indexing Pipecat's own docs/examples for coding assistants rather than exposing Pipecat's runtime voice-agent capabilities to arbitrary MCP clients. Missing for 10: independent/hands-on confirmation that the MCP server actually functions end-to-end (only CLI help text was probed, not live MCP protocol behavior), and broader documentation of what tools/resources the MCP server exposes.

   • [claimed-docs] “pipecat context-hub install registers the hub as an MCP server with your coding agent and builds the local index”
   • [claimed-docs] “pipecat context-hub install` registers the hub as an MCP server with your coding agent and builds the local index”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Pipecat CLI (pypi pipecat-ai[cli]) ran keylessly via uvx — `pipecat --help` lists init (pr…”

   proof ↗⚑ flag

5. ai-native userUse an official CLI

   weight 2 · round drawn
   Retell AI✓fullprobed9/10

   Retell ships an official CLI (`@retell-ai/retell-cli`) documented for managing agents, phone numbers, and knowledge bases from the terminal, and a runtime probe confirms it installs and runs keylessly via npx. Missing for 10: independent/community usage reports of the CLI itself (only docs and one probe run corroborate it).

   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”
   • [claimed-docs] “The Retell CLI lets you manage Retell resources from your terminal.”
   • [probe] “official CLI documented at https://docs.retellai.com/get-started/cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”

   proof ↗⚑ flag

   Pipecat✓fullprobed9/10

   Pipecat ships an official CLI (pip-installable pipecat-ai[cli], confirmed via runtime probe) supporting init/scaffolding, cloud deploy, eval, and context-hub (MCP registration for coding agents), well documented at docs.pipecat.ai/api-reference/cli/overview. This directly supports AI-native agentic workflows (context-hub install registers as MCP server, writes AGENTS.md/CLAUDE.md for coding agents). missing for 10: independent/third-party review of the CLI's robustness beyond vendor docs and a single runtime probe.

   • [probe] “PROBE runtime (recorded 2026-09-05): the official Pipecat CLI (pypi pipecat-ai[cli]) ran keylessly via uvx — `pipecat --help` lists init (pr…”
   • [probe] “official CLI documented at https://docs.pipecat.ai/api-reference/cli/overview”
   • [claimed-docs] “pipecat context-hub install registers the hub as an MCP server with your coding agent and builds the local index”
   • [claimed-docs] “Install the Pipecat CLI and scaffold the quickstart project (also writes AGENTS.md + CLAUDE.md)”
   • [claimed-docs] “pipecat context-hub install` registers the hub as an MCP server with your coding agent and builds the local index”

   proof ↗⚑ flag

6. ai-native userDrive the product through a documented public API

   weight 3 · round to Retell AI
   Retell AI✓fullprobed9/10

   Retell documents official Node.js/Python SDKs with typed clients and full endpoint coverage, a CLI for managing resources from the terminal, and API references (e.g., clone-voice endpoint) — and a runtime probe confirms the CLI installs/runs keylessly and the hosted MCP server exposes the entire API via meta-tools, proving the API is genuinely agent-drivable. Missing for 10: a publicly discoverable OpenAPI/swagger spec (all standard paths returned 404 in the probe).

   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”
   • [claimed-docs] “Retell provides official SDKs for Node.js and Python to simplify integration with our platform.”
   • [claimed-docs] “Type safety: Full TypeScript support with autocomplete ... Error handling: Structured error responses with detailed messages”
   • [probe] “official MCP server documented at https://docs.retellai.com/get-started/mcp-server”
   • [probe] “official CLI documented at https://docs.retellai.com/get-started/cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.retellai.com/openapi.json, https://docs.retellai.com/swagger.json, https://docs.retella…”

   proof ↗⚑ flag

   Pipecat~partialprobed7/10

   Pipecat exposes a well-documented programmatic surface — a Python framework API (frame processors, Flows functions with auto-derived metadata), a runner API, and an official CLI (init/cloud/eval/context-hub) confirmed to work keylessly via runtime probe — giving AI-native users a documented, scriptable way to drive the product. However, probing shows no REST/OpenAPI spec (all openapi.json candidates 404'd), so there's no conventional network-callable public API, only the code-level Python/CLI surface. Missing for 10: a formal REST/OpenAPI-documented HTTP API, and independent hands-on confirmation of driving a full pipeline purely via that public API surface.

   • [claimed-docs] “Functions in Pipecat Flows serve two key purposes: 1. Process data ... 2. Progress the conversation by transitioning between nodes”
   • [claimed-docs] “Pipecat ships a built-in development runner (\`pipecat.runner.run\`) that handles the server-side glue most bots need during development”
   • [claimed-docs] “Pipecat ships a built-in development runner (pipecat.runner.run) that handles the server-side glue most bots need during development: creati…”
   • [claimed-docs] “Flows auto-derives the function's metadata — name, description, parameter properties (with their descriptions), and which parameters are req…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.pipecat.ai/openapi.json, https://docs.pipecat.ai/swagger.json, https://docs.pipecat.ai/…”
   • [probe] “official CLI documented at https://docs.pipecat.ai/api-reference/cli/overview”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Pipecat CLI (pypi pipecat-ai[cli]) ran keylessly via uvx — `pipecat --help` lists init (pr…”

   proof ↗⚑ flag

7. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round drawn
   Retell AI—none0/10

   The evidence pack documents API key auth, SDKs, CLI, and an MCP server, but nowhere describes scoped, role-based, or least-privilege API key/credential issuance (e.g., per-agent or per-permission keys) for Retell. This is a fair axis for an API/voice-agent platform, but no docs or probes show scoped credential support, so it is unproven.

   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Retell provides official SDKs for Node.js and Python to simplify integration with our platform.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”
   • [claimed-docs] “Use Retell's MCP server to build and manage voice agents from MCP-capable clients like Cursor, Claude Desktop, and Claude Code via Retell AP…”

   proof ↗⚑ flag

   Pipecat—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

8. ai-native userBuild against official SDKs

   weight 2 · round drawn
   Retell AI✓fullprobed9/10

   Retell documents and ships official typed SDKs for Node.js and Python with API key auth, full endpoint coverage, structured errors, and TypeScript autocomplete, plus an official CLI and MCP server for programmatic/agentic access — all independently confirmed by runtime probes (CLI installs and runs, MCP server completes handshake). Missing for 10: no independent third-party review of SDK quality/DX beyond vendor docs and no public OpenAPI spec discovered.

   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Retell provides official SDKs for Node.js and Python to simplify integration with our platform.”
   • [claimed-docs] “Type safety: Full TypeScript support with autocomplete ... Error handling: Structured error responses with detailed messages”
   • [probe] “official CLI documented at https://docs.retellai.com/get-started/cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”
   • [probe] “official MCP server documented at https://docs.retellai.com/get-started/mcp-server”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”

   proof ↗⚑ flag

   Pipecat✓fullprobed9/10

   Pipecat is itself an official open-source Python SDK/framework (pip-installable, verified via runtime probe to install and import cleanly), and it also ships a family of official 'Pipecat Client' SDKs for web/mobile integration plus pre-built React components, all documented on the official site. Missing for 10: independent/third-party confirmation of the client SDKs' quality beyond docs (only the core Python framework was hands-on verified).

   • [claimed-docs] “Pipecat Clients are a family of SDKs that connect users to your Pipecat agents through web and mobile applications.”
   • [claimed-docs] “Pipecat Clients are a family of SDKs that connect users to your Pipecat agents through web and mobile applications. They handle real-time au…”
   • [claimed-docs] “Pre-built React components for voice AI interfaces.”
   • [claimed-docs] “Open source Python framework for building voice and multimodal AI pipelines. Orchestrate 150+ AI services with ultra-low latency.”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi pipecat-ai 1.8.1 (BSD-2 OSS) installs and `import pipecat` succeeds with no key, printing its star…”
   • [github] “Build a single voice agent or a full multi-agent system where specialists hand off, fan out in parallel, and coordinate over a shared bus, l…”

   proof ↗⚑ flag

9. ai-native userSubscribe to events via webhooks

   weight 2 · round to Retell AI
   Retell AI✓fullclaimed8/10

   Retell has dedicated first-party documentation for webhooks describing real-time event notifications pushed to your application as events occur, enabling event-driven integrations rather than polling — directly matching the story of subscribing to events via webhooks. Missing for 10: no independent/hands-on corroboration of webhook reliability or event catalog completeness beyond the docs themselves.

   • [claimed-docs] “Webhooks allow your application to receive real-time notifications about events that occur in your Retell AI account.”
   • [claimed-docs] “webhooks push data to your application as events happen, making your integrations more efficient and responsive.”

   proof ↗⚑ flag

   Pipecat—none0/10

   Webhooks appear only as an inbound mechanism (a telephony carrier calling Pipecat's webhook to signal an incoming call), not as an outbound event-subscription system that lets a user register a webhook to receive Pipecat's own events (e.g., call end, transcript ready, errors). No evidence of a webhook subscription API or configurable event notifications for consumers.

   • [claimed-docs] “Telephony bots have a different shape than WebRTC bots. The session doesn't start because a client sent your dispatcher an HTTP request — it…”
   • [claimed-docs] “the carrier (Twilio, Telnyx, Plivo, Exotel, or your SIP provider) is calling your webhook to tell you there's an inbound call”
   • [claimed-docs] “python bot.py -t twilio -x your-name.ngrok.io”

   proof ↗⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round to Retell AI
   Retell AI~partialclaimed6/10

   Retell's AI QA feature automatically scores calls on hallucination, knowledge-base accuracy, latency, sentiment, and tool usage to surface quality trends and issues, and custom analytics dashboards let users chart/filter call and chat metrics — both are AI-derived insights from the product's own call data. However, this is scoring/quality analytics rather than proactive generative 'suggestions' (e.g., recommended actions, next-best-response, or coaching tips), and there's no independent/hands-on evidence validating the accuracy or usefulness of these AI-generated insights. Missing for 10: evidence of proactive suggestion/recommendation generation beyond scoring, and third-party corroboration of AI QA insight quality.

   • [claimed-docs] “AI QA scores Retell calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage to surface quality trends and issues…”
   • [claimed-docs] “AI QA automatically evaluates a sampled set of your calls against rules and metrics you configure.”
   • [claimed-docs] “Build custom Retell analytics dashboards to track call and chat metrics like success rate, latency, cost, and concurrency, with charts, filt…”
   • [claimed-docs] “The Analytics dashboard charts your call and chat data so you can see how your agents are performing over time.”
   • [claimed-docs] “Built-in call success and sentiment scoring, plus custom fields synced to your CRM.”

   proof ↗⚑ flag

   Pipecat—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Retell AI
   Retell AI~partialcommunity6/10

   Retell agents act autonomously during live calls — function calling lets them independently book appointments, send SMS, transfer calls, and call external APIs without human intervention (retell-docs-5, retell-docs-35), and webhooks push real-time events to downstream systems (retell-docs-12, retell-docs-24). This shows in-call autonomous action-taking, which is a form of background automation once a call is triggered. However, there's no evidence of standalone scheduled/triggered automations running independent of a live voice/chat session (e.g., cron-like outbound campaigns or autonomous multi-step workflows outside conversation context). Missing for 10: evidence of scheduled or event-triggered background automations outside live calls, and independent/hands-on confirmation that autonomous function-calling reliably completes tasks unattended (community reports show some confusion/looping during autonomous task execution, retell-comm-1, retell-comm-3, retell-comm-5).

   • [claimed-docs] “Function calling lets Retell single or multi-prompt agents take real actions — transfer calls, end calls, book appointments, send SMS, and c…”
   • [claimed-docs] “Function calling transforms your AI agent from a conversational interface into an action-oriented assistant.”
   • [claimed-docs] “Webhooks allow your application to receive real-time notifications about events that occur in your Retell AI account.”
   • [claimed-docs] “webhooks push data to your application as events happen, making your integrations more efficient and responsive.”
   • [community] “The AI contradicted itself when layering conditionals - it got confused about morning vs afternoon time, kept asking to repeat despite corre…”
   • [community] “Voice sounds great, but: told it unavailable until next year, it confirmed Feb 4th 'next year' but thought current year was 2022. Also got s…”

   proof ↗⚑ flag

   Pipecat~partialclaimed4/10

   Pipecat supports multi-agent systems where 'specialists hand off, fan out in parallel, and coordinate over a shared bus, locally or distributed' and can 'hand off to subagents for long-running tools and complex tasks,' plus telephony bots that start autonomously when a carrier calls a webhook rather than a user request — suggesting some background/autonomous agent operation. But Pipecat's core model is a live, turn-based conversational pipeline tied to an active voice/video session, not a scheduler or cron-like background automation runner independent of a live interaction. Missing for 10: explicit scheduling/triggering mechanisms for unattended background jobs, evidence of agents running without any live session, and independent confirmation of autonomous multi-agent orchestration in production.

   • [github] “Build a single voice agent or a full multi-agent system where specialists hand off, fan out in parallel, and coordinate over a shared bus, l…”
   • [claimed-docs] “Hand off to subagents for long-running tools and complex tasks, and use Pipecat Flows when a conversation needs to follow a defined path.”
   • [claimed-docs] “Telephony bots have a different shape than WebRTC bots. The session doesn't start because a client sent your dispatcher an HTTP request — it…”
   • [claimed-docs] “the carrier (Twilio, Telnyx, Plivo, Exotel, or your SIP provider) is calling your webhook to tell you there's an inbound call”

   proof ↗⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round to Retell AI
   Retell AI~partialclaimed3/10

   Retell's dashboard includes 'Conductor', a built-in AI assistant that can draft an entire agent from a plain-English prompt (docs-30), which is a genuine instance of delegating a task to an in-product AI assistant. However this is a single thin mention with no further detail on scope, limits, or other delegable tasks beyond initial agent creation. Missing for 10: documentation of Conductor's full capabilities/limits, evidence of delegating other tasks (not just agent drafting) to a built-in assistant, and independent/hands-on corroboration of Conductor actually working.

   • [claimed-docs] “click **Generate from prompt** (marked Suggested) to let [Conductor](/conductor/create-agent) draft an agent from a plain-English descriptio…”

   proof ↗⚑ flag

   Pipecat—none0/10

   Pipecat's docs describe a framework/CLI for building voice AI agents, a context-hub that exposes Pipecat's own docs as an MCP server to external coding agents, and dev tooling (runner, evals, Flows) — but none of this is a built-in assistant inside Pipecat's own product surface that a user delegates tasks to. Missing for 10: any first-party 'chat with an assistant' feature in the CLI/dashboard/docs site, evidence of task delegation to an embedded assistant, or independent confirmation of such a feature.

   • [claimed-docs] “pipecat context-hub install` registers the hub as an MCP server with your coding agent and builds the local index”
   • [claimed-docs] “pipecat context-hub install registers the hub as an MCP server with your coding agent and builds the local index”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Pipecat CLI (pypi pipecat-ai[cli]) ran keylessly via uvx — `pipecat --help` lists init (pr…”

   proof ↗⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round to Retell AI
   Retell AI✓fullprobed7/10

   Retell supports natural-language operation via Conductor's 'Generate from prompt' feature that drafts an agent from a plain-English description, and via its official MCP server which exposes the full API as agent-callable tools so AI-native clients (Cursor, Claude Desktop, Claude Code) can build/manage agents through natural-language MCP tool calls — confirmed live by a keyless JSON-RPC probe against the hosted MCP endpoint. missing for 10: independent/hands-on evaluation of Conductor's prompt-to-agent quality, and no evidence of NL commands for other everyday operations (e.g., dashboard chat-based control) beyond agent creation/management.

   • [claimed-docs] “click **Generate from prompt** (marked Suggested) to let [Conductor](/conductor/create-agent) draft an agent from a plain-English descriptio…”
   • [claimed-docs] “Use Retell's MCP server to build and manage voice agents from MCP-capable clients like Cursor, Claude Desktop, and Claude Code via Retell AP…”
   • [claimed-docs] “Retell supports the Model Context Protocol (MCP) so you can build Retell AI voice agents directly from MCP-capable clients (Cursor, Claude D…”
   • [claimed-docs] “Retell supports the [Model Context Protocol (MCP)] so you can build Retell AI voice agents directly from MCP-capable clients (Cursor, Claude…”
   • [claimed-docs] “Agents: create, update, publish, list, and fetch agent versions.”
   • [probe] “official MCP server documented at https://docs.retellai.com/get-started/mcp-server”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”

   proof ↗⚑ flag

   Pipecat~partialprobed6/10

   Pipecat's entire raison d'être is building voice agents that end users control via natural spoken language — turn detection, interruption/barge-in, and Flows functions that progress conversations based on NL input (pipecat-docs-9, pipecat-docs-19, pipecat-docs-6/22/30) all support NL-driven operation of the *bots* it produces. However, the story is ambiguous about whether it's the bots or the Pipecat tooling itself; Pipecat's own developer-facing surfaces (CLI init/cloud/eval/context-hub) are traditional structured commands, not natural-language commands (pipecat-probe-rt-1, pipecat-docs-34/37). Missing for 10: evidence that the Pipecat framework/CLI itself (not just end-user bots) can be operated by an AI-native user via natural-language instructions rather than code/CLI syntax.

   • [claimed-docs] “Smart Turn Detection uses an advanced machine learning model to determine when a user has finished speaking and your bot should respond.”
   • [claimed-docs] “Interruptions (also called barge-in) let the user talk over the bot. When the user starts speaking while the bot is talking, the bot stops i…”
   • [claimed-docs] “Functions in Pipecat Flows serve two key purposes: 1. Process data ... 2. Progress the conversation by transitioning between nodes”
   • [claimed-docs] “Functions in Pipecat Flows serve two key purposes: Process data by interfacing with external systems and APIs...Progress the conversation by…”
   • [claimed-docs] “Functions in Pipecat Flows serve two key purposes: 1. Process data by interfacing with external systems and APIs... 2. Progress the conversa…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Pipecat CLI (pypi pipecat-ai[cli]) ran keylessly via uvx — `pipecat --help` lists init (pr…”
   • [claimed-docs] “pipecat init quickstart # Change to the project directory cd pipecat-quickstart”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   Retell AI—none0/10

   Evidence shows Retell has API reference doc pages (e.g., api-references/clone-voice.md) and SDKs/CLI/MCP tooling, but no evidence of an interactive, runnable API reference (e.g., embedded 'try it' console or Swagger UI). The probe explicitly found no OpenAPI/Swagger spec exposed (all candidate paths 404), undermining any claim of an interactive reference.

   • [claimed-docs] “Clone a voice from audio files”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.retellai.com/openapi.json, https://docs.retellai.com/swagger.json, https://docs.retella…”

   proof ↗⚑ flag

   Pipecat—none0/10

   Pipecat's docs are static markdown pages (get-started, fundamentals, flows, CLI reference) with no evidence of an interactive API reference or runnable code examples; a direct probe for OpenAPI/Swagger endpoints returned 404 on all candidate paths, indicating no interactive API explorer exists.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.pipecat.ai/openapi.json, https://docs.pipecat.ai/swagger.json, https://docs.pipecat.ai/…”
   • [probe] “official CLI documented at https://docs.pipecat.ai/api-reference/cli/overview”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   Retell AI—none0/10

   A direct probe for standard OpenAPI/Swagger spec locations (openapi.json, swagger.json, etc.) on Retell's docs domain returned 404 for all candidates, and no evidence pack item shows a downloadable, machine-readable API spec being published or linked from docs. The CLI's version string mentions an internal 'OpenAPI 3.0.0' schema, but this is not shown to be an artifact users can download or fetch programmatically.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.retellai.com/openapi.json, https://docs.retellai.com/swagger.json, https://docs.retella…”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”

   proof ↗⚑ flag

   Pipecat—none0/10

   A direct probe for OpenAPI/swagger specs at all standard locations (openapi.json, swagger.json, etc.) on docs.pipecat.ai returned 404s, and no evidence pack item shows a downloadable machine-readable API spec despite Pipecat having an api-reference docs section and cloud service.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.pipecat.ai/openapi.json, https://docs.pipecat.ai/swagger.json, https://docs.pipecat.ai/…”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round to Pipecat
   Retell AI~partialclaimed4/10

   Retell's quick-start lets you click 'Test' to run a free web call with your agent before it ever touches a phone number or payment method, and new accounts get $10 trial credit to experiment risk-free — a lightweight way to try an agent without production telephony traffic. However, there's no documented dedicated sandbox environment, test-mode API flag, or explicit data isolation guarantee separating test calls from production records/analytics. Missing for 10: explicit sandbox/staging environment concept, test vs prod data isolation guarantees, and independent confirmation that test-call data doesn't mix with production analytics/QA.

   • [claimed-docs] “Click the "Test" button to start a web call with your agent... This step is free and doesn't need a phone number or payment method.”
   • [claimed-docs] “Click the "Test" button to start a web call with your agent”
   • [claimed-docs] “New accounts start with \$10 in free trial credits, no payment method required.”

   proof ↗⚑ flag

   Pipecat~partialclaimed5/10

   Pipecat provides a local development runner (`pipecat.runner.run`) and CLI scaffolding that let developers run and iterate on bots entirely locally, plus a built-in Evals system that runs described conversations against 'your real agent' pipeline for behavior testing — both usable without deploying to Pipecat Cloud or touching a production instance. However, there's no explicit 'sandbox environment' concept with data isolation from production (e.g. no staging vs prod data separation, no test-data seeding, no mention of production data at all since Pipecat doesn't store conversational data centrally). Missing for 10: explicit sandbox/production data isolation, documented data classification for dev vs prod, hands-on confirmation that Evals or local runner never touch live production resources.

   • [claimed-docs] “Pipecat ships a built-in development runner (\`pipecat.runner.run\`) that handles the server-side glue most bots need during development”
   • [claimed-docs] “Pipecat ships a built-in development runner (pipecat.runner.run) that handles the server-side glue most bots need during development: creati…”
   • [claimed-docs] “Pipecat ships a built-in development runner (`pipecat.runner.run`) that handles the server-side glue most bots need during development: crea…”
   • [claimed-docs] “Pipecat Evals is the framework's built-in system for testing agent behavior. You describe a conversation and the behavior you expect, and Pi…”
   • [claimed-docs] “You describe a conversation and the behavior you expect, and Pipecat runs it against your real agent (the same pipeline, the same services, …”
   • [claimed-docs] “This is the property that makes the same bot file portable across the development runner, Pipecat Cloud, and most production self-hosting se…”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Retell AI—none0/10

   The evidence pack covers SDKs, CLI, MCP server, and API endpoints extensively, but there is no documentation of API versioning scheme or a deprecation policy for breaking changes. The probe notes an OpenAPI version string (3.0.0, catalog v4) but this is not evidence of a documented deprecation/versioning policy for API consumers.

   ⚑ flag

   Pipecat—none0/10

   No evidence pack item discusses API versioning scheme, version numbers surfaced in docs, or any documented deprecation policy/timeline for Pipecat's SDKs or APIs; the only version datum (pip package 1.8.1) shows a version exists but nothing about a deprecation policy is documented.

   ⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Retell AI
   Retell AI~partialprobed4/10

   Retell exposes a full-coverage SDK, CLI, and MCP server that let an AI agent script repeated single-item operations (create/update/list agents, phone numbers, knowledge bases) programmatically, enabling scripted bulk-like loops, but there is no documented native batch/bulk endpoint operating on many items in a single call. missing for 10: explicit batch/bulk API endpoints or CLI commands operating on multiple items in one call, evidence of rate-limit-safe bulk workflows, and hands-on confirmation of bulk usage at scale

   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”
   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Use Retell's MCP server to build and manage voice agents from MCP-capable clients like Cursor, Claude Desktop, and Claude Code via Retell AP…”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”
   • [claimed-docs] “Agents: create, update, publish, list, and fetch agent versions.”

   proof ↗⚑ flag

   Pipecat~partialclaimed3/10

   Pipecat supports 'unlimited concurrency' for running many simultaneous voice sessions and multi-agent fan-out ('fan out in parallel... coordinate over a shared bus'), which could support bulk-style parallel processing of many calls/agents at once. However, there is no explicit bulk-operation tooling (e.g., batch APIs, bulk data import/export, mass-edit of items) documented anywhere in the evidence pack. Missing for 10: explicit bulk/batch operation APIs or workflows, evidence of processing many discrete 'items' (not just concurrent voice sessions) in one operation, and any hands-on confirmation of this at scale.

   • [claimed-docs] “Pipecat Cloud supports unlimited concurrency.”
   • [github] “Build a single voice agent or a full multi-agent system where specialists hand off, fan out in parallel, and coordinate over a shared bus, l…”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round drawn
   Retell AI~partialclaimed6/10

   Retell supports event-driven automation via webhooks that fire on account events (retell-docs-12, retell-docs-24), function calling that lets agents automatically transfer calls, end calls, book appointments, or call external APIs based on conversation logic (retell-docs-5, retell-docs-35), and node-based flows with conditional branching (retell-docs-25) plus AI QA rules evaluated against configured metrics (retell-docs-43). However, this is scattered across call-flow logic and webhook notifications rather than a unified 'if event then action' rule-definition interface. Missing for 10: a dedicated rules/trigger engine UI for arbitrary account-wide events, documentation of webhook-to-action automation chains, and independent verification that rule-based automation works reliably (community reports note conversational logic errors, e.g. retell-comm-1, retell-comm-3, retell-comm-7).

   • [claimed-docs] “Function calling lets Retell single or multi-prompt agents take real actions — transfer calls, end calls, book appointments, send SMS, and c…”
   • [claimed-docs] “Webhooks allow your application to receive real-time notifications about events that occur in your Retell AI account.”
   • [claimed-docs] “webhooks push data to your application as events happen, making your integrations more efficient and responsive.”
   • [claimed-docs] “Drag-and-drop, node-based flows for structured, high-stakes calls.”
   • [claimed-docs] “Function calling transforms your AI agent from a conversational interface into an action-oriented assistant.”
   • [claimed-docs] “AI QA automatically evaluates a sampled set of your calls against rules and metrics you configure.”

   proof ↗⚑ flag

   Pipecat~partialclaimed6/10

   Pipecat's frame-based pipeline is inherently event-driven — interruption events automatically halt bot output when the user speaks, and Pipecat Flows functions can process data and automatically transition conversation nodes based on function-call events. This gives a real automation layer, but it's scoped to conversation/pipeline events rather than a general-purpose rules engine for arbitrary triggers/actions. Missing for 10: a documented generic rule-definition system (arbitrary trigger→action bindings outside conversation flow), and independent/hands-on evidence of custom event-rule automation beyond the built-in interruption/Flows mechanics.

   • [claimed-docs] “When the user starts speaking while the bot is talking, the bot stops immediately, in-flight work is cancelled, and the pipeline is ready fo…”
   • [claimed-docs] “Interruptions (also called barge-in) let the user talk over the bot. When the user starts speaking while the bot is talking, the bot stops i…”
   • [claimed-docs] “Functions in Pipecat Flows serve two key purposes: 1. Process data ... 2. Progress the conversation by transitioning between nodes”
   • [claimed-docs] “Functions in Pipecat Flows serve two key purposes: Process data by interfacing with external systems and APIs...Progress the conversation by…”
   • [claimed-docs] “Functions in Pipecat Flows serve two key purposes: 1. Process data by interfacing with external systems and APIs... 2. Progress the conversa…”
   • [claimed-docs] “Pipecat's turn events make it easy to collect both user and assistant messages as they occur.”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round drawn
   Retell AI—none0/10

   Retell's docs cover webhooks, function calling, MCP tool-calling, and telephony integration, but nothing describes scheduling recurring jobs, campaigns, or workflows (e.g., cron-like recurring outbound call batches or repeating automations). This is a fair question for a voice-agent platform (buyers often want scheduled/recurring outbound campaigns), so the axis applies, but no evidence in the pack shows this capability. missing for 10: any mention of a scheduler, recurring campaign/job feature, or cron-style automation trigger.

   ⚑ flag

   Pipecat—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round to Retell AI
   Retell AI~partialclaimed4/10

   Retell supports 'agent versions' with create/update/publish/list/fetch operations via CLI and MCP (retell-docs-47), implying some versioning and possibly rollback via republishing an older version, but there is no documented review workflow (diffs, approvals, changelogs) or explicit rollback command/UI. missing for 10: explicit rollback mechanism, version diff/review UI, changelog or audit trail, and any independent/hands-on confirmation that version history and rollback actually work as described.

   • [claimed-docs] “Agents: create, update, publish, list, and fetch agent versions.”
   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”
   • [claimed-docs] “Use Retell's MCP server to build and manage voice agents from MCP-capable clients like Cursor, Claude Desktop, and Claude Code via Retell AP…”

   proof ↗⚑ flag

   Pipecat—none0/10

   Pipecat's docs describe building pipelines/Flows in Python code but no evidence of built-in versioning, review workflows, or rollback capabilities for automations (Flows or pipelines) — no changelog, diff, or rollback tooling is mentioned anywhere in the evidence pack.

   ⚑ flag

Compliance

1. founderMeet call-recording consent and disclosure obligations with per-call recording controls and configurable data retention

   weight 2 · round to Retell AI
   Retell AI~partialclaimed5/10

   Retell documents per-agent configurable data retention that auto-deletes transcripts, recordings, and logs after a set period, directly addressing retention obligations. However, there is no evidence of explicit per-call recording enable/disable controls or built-in consent-disclosure features (e.g., automated recording announcements) that founders would need for two-party consent compliance. Missing for 10: per-call recording on/off toggle, consent-disclosure/announcement mechanism, and any documentation tying these to legal consent requirements.

   • [claimed-docs] “Configure per-agent data retention to automatically delete call and chat data — transcripts, recordings, and logs — after a set period for c…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent. After the retention period expires, call and chat data associated with tha…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent.”

   proof ↗⚑ flag

   Pipecat~partialclaimed3/10

   Pipecat provides an AudioBufferProcessor that developers can use to capture call recordings, which is a building block a founder could use to implement recording, but there's no documented feature for per-call consent/disclosure toggles or configurable data-retention policies — those would have to be built entirely by the developer. missing for 10: consent/disclosure mechanisms, retention-policy configuration, compliance-specific controls, any documentation tying recording to legal/consent obligations.

   • [claimed-docs] “Pipecat's \`AudioBufferProcessor\` makes it easy to capture high-quality audio recordings of both the user and bot during interactions.”
   • [claimed-docs] “Pipecat's AudioBufferProcessor makes it easy to capture high-quality audio recordings of both the user and bot during interactions.”
   • [claimed-docs] “Pipecat's `AudioBufferProcessor` makes it easy to capture high-quality audio recordings of both the user and bot during interactions.”

   proof ↗⚑ flag

2. platform-engineerRun regulated workloads with HIPAA/BAA support, SOC 2, and data-residency options

   weight 2 · round drawn
   Retell AI—none0/10

   Evidence shows only generic data-retention controls (docs-15/27/44) but no mention of HIPAA, BAA, SOC 2 certification, or data-residency options anywhere in the docs or community pack, despite this being a fair compliance axis for a voice-AI platform handling call recordings/transcripts.

   • [claimed-docs] “Configure per-agent data retention to automatically delete call and chat data — transcripts, recordings, and logs — after a set period for c…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent. After the retention period expires, call and chat data associated with tha…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent.”

   proof ↗⚑ flag

   Pipecat—none0/10

   No evidence in the pack mentions HIPAA/BAA, SOC 2 certification, or data-residency options for Pipecat or Pipecat Cloud; the pack covers pipeline features, telephony, CLI, and dev tooling but nothing on compliance/regulatory posture. missing for 10: HIPAA/BAA documentation, SOC 2 attestation, data-residency configuration options.

   ⚑ flag

Scale

1. platform-engineerSee documented concurrency limits and scale to many simultaneous calls without manual capacity begging

   weight 2 · round to Retell AI
   Retell AI~partialclaimed5/10

   Retell documents a concurrency configuration feature (reserved inbound concurrency vs. outbound limit) showing platform-level concurrency controls exist, but there is no documented default/numeric concurrency limits per plan, no described self-service process to raise limits without contacting support, and no evidence (docs or community) confirming scaling to many simultaneous calls happens without manual requests. missing for 10: published default concurrency limits, self-service limit-increase mechanism, and independent confirmation of large-scale concurrent call handling without support intervention.

   • [claimed-docs] “Reserved inbound concurrency protects inbound calls from being crowded out by outbound traffic.”
   • [claimed-docs] “When `reserved_inbound_concurrency` is configured, outbound calls can use at most your concurrency limit minus the reserved amount.”

   proof ↗⚑ flag

   Pipecat~partialcommunity4/10

   Pipecat Cloud's pricing page asserts 'unlimited concurrency' and offers multiple agent profiles for compute needs, giving a vendor-level answer to scaling, but there is no documented capacity planning guide, per-instance concurrency limits, or benchmarks for self-hosted deployments. Community commentary (a developer porting Pipecat to avoid Python's GIL for 'real concurrency') suggests self-hosted scaling is not frictionless and may require manual work to reach high concurrency. Missing for 10: documented concurrency ceilings/benchmarks for self-hosted deployments, load-testing data, and detail behind the 'unlimited concurrency' claim beyond a pricing-page bullet.

   • [claimed-docs] “Pipecat Cloud supports unlimited concurrency.”
   • [claimed-docs] “Pipecat Cloud offers different agent profiles, to best support your use case and compute needs.”
   • [community] “I got tired of running Python servers for voice agents, so I started porting Pipecat over. The design is Pipecat's, frames and processors an…”

   proof ↗⚑ flag

Self host

1. platform-engineerSelf-host the voice agent runtime from open-source code on my own infrastructure

   weight 3 · round to Pipecat
   Retell AI—none0/10

   Retell AI is a hosted SaaS voice-agent platform; all evidence points to cloud-hosted APIs, SDKs, CLI, and MCP server that call Retell's own servers, with no mention of open-source runtime code or self-hosting instructions. There is no evidence a platform-engineer can download and run the voice agent runtime on their own infrastructure.

   • [claimed-docs] “Build your first Retell AI phone agent in 15 minutes: create an account, pick a template, test in the dashboard, deploy to a phone number, a…”
   • [claimed-docs] “Retell starts at $0, pay only for what you use.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”

   proof ↗⚑ flag

   Pipecat✓fullprobed9/10

   Pipecat is confirmed as a pip-installable, BSD-2 open-source Python framework that runs keylessly with no vendor lock-in, ships a development runner and CLI, and docs explicitly state bot files are portable across the dev runner, Pipecat Cloud, and 'most production self-hosting setups.' Runtime probes confirm the package installs and imports successfully outside any managed cloud, and community evidence corroborates real-world self-hosted usage (local inference via Ollama/Whisper, custom ports of the framework). Missing for 10: a first-party detailed production self-hosting deployment guide (e.g., Docker/Kubernetes reference architecture) beyond the telephony/local-runner docs.

   • [probe] “PROBE runtime (recorded 2026-09-05): pypi pipecat-ai 1.8.1 (BSD-2 OSS) installs and `import pipecat` succeeds with no key, printing its star…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Pipecat CLI (pypi pipecat-ai[cli]) ran keylessly via uvx — `pipecat --help` lists init (pr…”
   • [claimed-docs] “This is the property that makes the same bot file portable across the development runner, Pipecat Cloud, and most production self-hosting se…”
   • [claimed-docs] “Open source Python framework for building voice and multimodal AI pipelines. Orchestrate 150+ AI services with ultra-low latency.”
   • [community] “Local inference is already supported via Pipecat, you can use ollama or any custom OpenAI endpoint. Local STT is also supported via whisper,…”
   • [community] “Yes, Pipecat already supports that natively, so this can be done easily with ollama... Also, check out any provider they support, and it can…”
   • [github] “Build a single voice agent or a full multi-agent system where specialists hand off, fan out in parallel, and coordinate over a shared bus, l…”

   proof ↗⚑ flag

Latency

1. platform-engineerSee documented end-to-end voice latency numbers or tuning guidance backing the platform's speed claims

   weight 3 · round to Pipecat
   Retell AI~partialclaimed5/10

   Retell docs offer tuning guidance for latency (transcription mode trade-offs, endpointing settings) and let customers monitor latency via analytics dashboards and AI QA scoring, but there are no documented end-to-end latency benchmarks or numbers substantiating speed claims. missing for 10: published end-to-end latency figures/benchmarks, and independent verification of speed claims.

   • [claimed-docs] “Pick the right Retell transcription mode to balance accuracy against latency — interim results for speed or context-aware results for higher…”
   • [claimed-docs] “Real time transcription is often a trade off between latency and accuracy.”
   • [claimed-docs] “optimize for speed: uses the latest interim results with a low endpointing setting... optimize for accuracy: uses the results with a higher …”
   • [claimed-docs] “Build custom Retell analytics dashboards to track call and chat metrics like success rate, latency, cost, and concurrency, with charts, filt…”
   • [claimed-docs] “The Analytics dashboard charts your call and chat data so you can see how your agents are performing over time.”
   • [claimed-docs] “AI QA scores Retell calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage to surface quality trends and issues…”
   • [claimed-docs] “AI QA automatically evaluates a sampled set of your calls against rules and metrics you configure.”

   proof ↗⚑ flag

   Pipecat~partialclaimed6/10

   Pipecat documents a dedicated STT-latency-tuning guide describing measured P99 latency values per service and a TTFS (Time To Final Segment) metric, plus a built-in metrics system for enabling latency instrumentation — real tuning guidance exists. However, this is scoped to STT stage latency rather than a comprehensive end-to-end voice round-trip latency benchmark, and no actual published numbers/tables or independent corroboration of the claimed P99 figures are shown. Missing for 10: published end-to-end latency benchmark numbers (not just STT), tuning guidance across the full voice pipeline (TTS/LLM/turn-taking), and independent verification of the claimed latency figures.

   • [claimed-docs] “Pipecat ships with measured P99 latency values for each supported service so that turn detection can account for this delay automatically.”
   • [claimed-docs] “Time To Final Segment (TTFS) measures how long it takes from the moment a user stops speaking until the STT service delivers the final trans…”
   • [claimed-docs] “Pipecat offers built-in metrics for both, which can be enabled with straightforward configuration options.”

   proof ↗⚑ flag

Turn taking

1. developerRely on the agent to handle interruptions (barge-in) gracefully — stopping speech, updating context, and recovering the turn

   weight 3 · round to Pipecat
   Retell AI—none0/10

   The evidence pack covers transcription latency/accuracy trade-offs (retell-docs-9, retell-docs-39, retell-docs-48) but nowhere describes explicit barge-in/interruption handling — stopping agent speech, updating context, and recovering the turn when a caller interrupts. Community threads show turn-taking confusion (contradictions, loops, disconnects) but do not address interruption handling specifically. Missing for full/partial credit: any documentation or hands-on report confirming barge-in detection, speech-stop behavior, or turn recovery mechanics.

   ⚑ flag

   Pipecat✓fullclaimed8/10

   Pipecat has explicit, dedicated documentation on interruptions/barge-in describing immediate speech stop, cancellation of in-flight work, and pipeline readiness for new input, plus Smart Turn Detection for natural turn-taking and Krisp-based false-interruption suppression in Pipecat Cloud. Missing for 10: independent hands-on validation of graceful context/turn recovery specifically (community evidence is about debugging/latency tooling, not barge-in correctness) and no explicit documentation of how conversation context is updated/rolled back after an interruption.

   • [claimed-docs] “When the user starts speaking while the bot is talking, the bot stops immediately, in-flight work is cancelled, and the pipeline is ready fo…”
   • [claimed-docs] “Interruptions (also called barge-in) let the user talk over the bot. When the user starts speaking while the bot is talking, the bot stops i…”
   • [claimed-docs] “Smart Turn Detection uses an advanced machine learning model to determine when a user has finished speaking and your bot should respond.”
   • [claimed-docs] “Smart Turn Detection recognizes natural conversational cues like intonation patterns and linguistic signals for more natural conversations.”
   • [claimed-docs] “Eliminates background noise and voices that cause false interruptions, powered by Krisp.”

   proof ↗⚑ flag

2. developerEnable noise suppression or audio filtering so the agent stays coherent on noisy real-world calls

   weight 1 · round to Pipecat
   Retell AI—none0/10

   No evidence pack item mentions noise suppression, background-noise filtering, or audio-preprocessing features for handling noisy real-world call environments; the closest related feature (transcription-mode trade-off between latency and accuracy) addresses turn-taking speed, not noise robustness. This is a plausible axis for a voice-agent platform, so absence of evidence yields 'none' rather than 'na'.

   ⚑ flag

   Pipecat✓fullclaimed8/10

   Pipecat Cloud explicitly ships Krisp-powered noise/background-voice suppression to eliminate false interruptions, and the framework's Smart Turn Detection complements this by recognizing genuine speech cues, keeping the agent coherent on noisy calls. Missing for 10: no first-party docs on self-hosted (non-cloud) noise suppression configuration, and no independent hands-on validation of suppression quality in real noisy environments.

   • [claimed-docs] “Eliminates background noise and voices that cause false interruptions, powered by Krisp.”
   • [claimed-docs] “Smart Turn Detection uses an advanced machine learning model to determine when a user has finished speaking and your bot should respond.”
   • [claimed-docs] “Smart Turn Detection recognizes natural conversational cues like intonation patterns and linguistic signals for more natural conversations.”

   proof ↗⚑ flag

3. developerUse model-based end-of-turn detection beyond simple VAD silence timeouts so the agent doesn't talk over slow speakers

   weight 2 · round to Pipecat
   Retell AI—none0/10

   The transcription-mode docs only describe tuning an 'endpointing setting' (low vs high) as a speed/accuracy tradeoff for VAD-style silence detection, not a distinct model-based semantic end-of-turn detector that understands incomplete utterances from slow speakers. No evidence pack item describes a dedicated turn-detection model or algorithm beyond adjusting silence-timeout thresholds.

   • [claimed-docs] “Pick the right Retell transcription mode to balance accuracy against latency — interim results for speed or context-aware results for higher…”
   • [claimed-docs] “Real time transcription is often a trade off between latency and accuracy.”
   • [claimed-docs] “optimize for speed: uses the latest interim results with a low endpointing setting... optimize for accuracy: uses the results with a higher …”

   proof ↗⚑ flag

   Pipecat✓fullclaimed8/10

   Pipecat documents 'Smart Turn Detection' as an ML model that determines end-of-turn based on intonation and linguistic cues, going beyond simple VAD silence timeouts, directly matching the story (pipecat-docs-9, pipecat-docs-31). Additional STT latency-tuning docs show the framework accounts for provider-specific delays to avoid premature cutoffs (pipecat-docs-15, pipecat-docs-20). Missing for 10: independent/hands-on evidence validating the model's accuracy on slow speakers in production, and no third-party benchmark of false-interruption reduction.

   • [claimed-docs] “Smart Turn Detection uses an advanced machine learning model to determine when a user has finished speaking and your bot should respond.”
   • [claimed-docs] “Smart Turn Detection recognizes natural conversational cues like intonation patterns and linguistic signals for more natural conversations.”
   • [claimed-docs] “Pipecat ships with measured P99 latency values for each supported service so that turn detection can account for this delay automatically.”
   • [claimed-docs] “Time To Final Segment (TTFS) measures how long it takes from the moment a user stops speaking until the STT service delivers the final trans…”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Retell AI
   Retell AI✓fullprobed8/10

   Retell exposes full API coverage via official Node/Python SDKs, a CLI, and a hosted MCP server that dynamically exposes the entire API (list/get/invoke endpoint tools), confirmed by keyless runtime probes actually reaching the CLI and MCP handshake — meaning nearly anything doable in the dashboard (agents, phone numbers, knowledge bases, function calling, voice cloning, analytics) is API/CLI/MCP accessible. missing for 10: no public OpenAPI spec was found (404s across candidate paths), and no independent hands-on confirmation that every single UI feature (e.g., analytics dashboards, AI QA config) has a 1:1 API equivalent.

   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”
   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Use Retell's MCP server to build and manage voice agents from MCP-capable clients like Cursor, Claude Desktop, and Claude Code via Retell AP…”
   • [claimed-docs] “Retell supports the Model Context Protocol (MCP) so you can build Retell AI voice agents directly from MCP-capable clients (Cursor, Claude D…”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.retellai.com/openapi.json, https://docs.retellai.com/swagger.json, https://docs.retella…”
   • [claimed-docs] “Agents: create, update, publish, list, and fetch agent versions.”

   proof ↗⚑ flag

   Pipecat~partialprobed4/10

   Pipecat is fundamentally code-first: the CLI covers init/scaffolding, cloud deploy, evals, and context-hub (pipecat-probe-rt-1, pipecat-docs-16/34/37), and the framework itself is the primary interface. But evidence never explicitly confirms that Pipecat Cloud's web dashboard (agent profiles, concurrency settings, monitoring) exposes no exclusive UI-only functionality, nor does it document a full REST/OpenAPI surface (probe shows no discoverable OpenAPI spec, pipecat-probe-2) for programmatic parity with any cloud console. Missing for 10: explicit UI/API parity statement, documented OpenAPI/REST API for Pipecat Cloud console features, and confirmation no dashboard-only settings exist.

   • [probe] “PROBE runtime (recorded 2026-09-05): the official Pipecat CLI (pypi pipecat-ai[cli]) ran keylessly via uvx — `pipecat --help` lists init (pr…”
   • [claimed-docs] “Install the Pipecat CLI and scaffold the quickstart project (also writes AGENTS.md + CLAUDE.md)”
   • [claimed-docs] “pipecat init quickstart # Change to the project directory cd pipecat-quickstart”
   • [claimed-docs] “Pipecat Cloud offers different agent profiles, to best support your use case and compute needs.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.pipecat.ai/openapi.json, https://docs.pipecat.ai/swagger.json, https://docs.pipecat.ai/…”
   • [claimed-docs] “Pipecat ships a built-in development runner (pipecat.runner.run) that handles the server-side glue most bots need during development: creati…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round to Pipecat
   Retell AI—none0/10

   Docs describe API/SDK/CLI access, data retention (auto-deletion) and analytics dashboards, but nowhere document a data-export or portability feature that would let a user pull all call/chat data, transcripts, and recordings in an open format to leave the platform; retention docs only cover automatic deletion, not export.

   • [claimed-docs] “Configure per-agent data retention to automatically delete call and chat data — transcripts, recordings, and logs — after a set period for c…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent. After the retention period expires, call and chat data associated with tha…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent.”
   • [claimed-docs] “Retell provides official SDKs for Node.js and Python to simplify integration with our platform.”
   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”

   proof ↗⚑ flag

   Pipecat~partialprobed5/10

   Pipecat is open-source (BSD-2) and self-hostable, meaning users inherently retain full control of their data (audio, transcripts, logs) rather than it being locked in a vendor's system — recordings via AudioBufferProcessor and transcripts via turn events are captured directly in the developer's own infrastructure. However, there's no explicit documentation of an 'export' feature, standardized open data formats, or migration tooling for leaving a hosted deployment (e.g., Pipecat Cloud). missing for 10: explicit data-export/format documentation, Pipecat Cloud data portability/exit guarantees, independent confirmation of format openness.

   • [claimed-docs] “Pipecat's \`AudioBufferProcessor\` makes it easy to capture high-quality audio recordings of both the user and bot during interactions.”
   • [claimed-docs] “Pipecat's turn events make it easy to collect both user and assistant messages as they occur.”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi pipecat-ai 1.8.1 (BSD-2 OSS) installs and `import pipecat` succeeds with no key, printing its star…”
   • [claimed-docs] “Open source Python framework for building voice and multimodal AI pipelines. Orchestrate 150+ AI services with ultra-low latency.”

   proof ↗⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round to Pipecat
   Retell AI—none0/10

   Retell AI is a closed, proprietary SaaS platform; the evidence pack shows SDKs, CLI, MCP server and docs but no mention of source code being published under any open license, nor any GitHub repo for the core platform. Missing for 10: any open-source license grant, public source repository, or licensing terms for the core agent/voice engine.

   ⚑ flag

   Pipecat✓fullprobed9/10

   Pipecat is explicitly documented as an 'Open source Python framework' with a public GitHub repo, and a runtime probe confirms the pip package is BSD-2 licensed and installable/importable without any key, confirming genuine open-source source access. Missing for 10: no explicit citation of a LICENSE file or license text itself, only the probe's characterization.

   • [claimed-docs] “Open source Python framework for building voice and multimodal AI pipelines. Orchestrate 150+ AI services with ultra-low latency.”
   • [github] “Build a single voice agent or a full multi-agent system where specialists hand off, fan out in parallel, and coordinate over a shared bus, l…”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi pipecat-ai 1.8.1 (BSD-2 OSS) installs and `import pipecat` succeeds with no key, printing its star…”

   proof ↗⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round to Pipecat
   Retell AI—none0/10

   Retell AI is presented entirely as a hosted SaaS (cloud dashboard, hosted APIs, hosted MCP server, usage-based pricing) with no mention anywhere in docs of a self-hosted or on-prem deployment option; all evidence points to a fully managed cloud product.

   • [claimed-docs] “Retell starts at $0, pay only for what you use.”
   • [claimed-docs] “Build your first Retell AI phone agent in 15 minutes: create an account, pick a template, test in the dashboard, deploy to a phone number, a…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”
   • [claimed-docs] “Integrate Retell voice agents with your own telephony provider using elastic SIP trunking or imported numbers from Twilio, Telnyx, and Vonag…”

   proof ↗⚑ flag

   Pipecat✓fullprobed8/10

   Pipecat is an open-source Python framework (BSD-2 licensed), pip-installable and runnable with no key/cloud dependency, with a documented dev runner and explicit note that the same bot file is portable to self-hosting setups; local inference (Ollama, Whisper) further confirms self-hosting without vendor lock-in. missing for 10: no dedicated self-hosting/production deployment guide (e.g., Docker/Kubernetes reference architecture) or independent case study of a large-scale self-hosted deployment.

   • [probe] “PROBE runtime (recorded 2026-09-05): pypi pipecat-ai 1.8.1 (BSD-2 OSS) installs and `import pipecat` succeeds with no key, printing its star…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Pipecat CLI (pypi pipecat-ai[cli]) ran keylessly via uvx — `pipecat --help` lists init (pr…”
   • [claimed-docs] “This is the property that makes the same bot file portable across the development runner, Pipecat Cloud, and most production self-hosting se…”
   • [community] “Local inference is already supported via Pipecat, you can use ollama or any custom OpenAI endpoint. Local STT is also supported via whisper,…”
   • [community] “Yes, Pipecat already supports that natively, so this can be done easily with ollama... Also, check out any provider they support, and it can…”

   proof ↗⚑ flag

Pricing

1. founderSee published per-minute or usage pricing and estimate cost per call before committing

   weight 2 · round to Retell AI
   Retell AI✓fullcommunity8/10

   Retell's pricing page explicitly states pay-as-you-go pricing starting at $0 with no annual contracts, offers an interactive calculator to estimate approximate monthly cost by adjusting key parameters, and new accounts get $10 free trial credit to test before paying — directly enabling a founder to estimate call costs before committing. Community discussion also confirms a concrete per-minute rate (17¢/min) is being charged in practice, corroborating usage-based billing exists. missing for 10: the per-minute rate itself is not shown in first-party docs (only inferred from a community comment), and there's no independent verification of the calculator's accuracy.

   • [claimed-docs] “Retell starts at $0, pay only for what you use.”
   • [claimed-docs] “Adjust key parameters to see an approximate monthly cost for your Retell AI voice agents.”
   • [claimed-docs] “Most voice AI platforms require annual contracts before you write a single line of code. Retell starts at $0, pay only for what you use.”
   • [claimed-docs] “New accounts start with \$10 in free trial credits, no payment method required.”
   • [community] “Founder (yanyan_evie) noted voice provider costs differ: 'The different providers have different prices. openai tts & deepgram are cheaper, …”
   • [community] “A commenter noted per-minute pricing (17¢/min) could be exploited: a malicious caller could keep the bot on the line for hours across many p…”

   proof ↗⚑ flag

   Pipecat—none0/10

   The evidence pack references a Pipecat Cloud pricing page (concurrency, agent profiles) but never shows actual per-minute/usage rates, a pricing table, or a cost calculator that would let a founder estimate cost per call; the core open-source framework itself is free/self-hosted with no billing model at all.

   • [claimed-docs] “Pipecat Cloud supports unlimited concurrency.”
   • [claimed-docs] “Pipecat Cloud offers different agent profiles, to best support your use case and compute needs.”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round to Pipecat
   Retell AI—none0/10

   No evidence in the pack mentions data residency, region selection, or geographic storage options; the closest related feature is per-agent data retention/deletion (retell-docs-15/27/44), which addresses retention duration, not location of storage. This axis is applicable to a cloud AI voice platform handling call/chat data, so absence of evidence yields 'none'. Missing for 10: any documentation of selectable regions, data residency guarantees, or geo-specific hosting.

   • [claimed-docs] “Configure per-agent data retention to automatically delete call and chat data — transcripts, recordings, and logs — after a set period for c…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent. After the retention period expires, call and chat data associated with tha…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent.”

   proof ↗⚑ flag

   Pipecat~partialprobed3/10

   Pipecat is an open-source, self-hostable framework (pip-installable, runs locally or on your own infrastructure), which implicitly lets an operator choose where data/compute resides by choosing their own deployment location; however there is no explicit documentation of region/residency selection controls, especially for the hosted Pipecat Cloud offering. Missing for 10: explicit region-selection UI/API for Pipecat Cloud, data-residency compliance documentation (GDPR/SOC2 region guarantees), and confirmation that self-hosting is the only path to residency control.

   • [probe] “PROBE runtime (recorded 2026-09-05): pypi pipecat-ai 1.8.1 (BSD-2 OSS) installs and `import pipecat` succeeds with no key, printing its star…”
   • [claimed-docs] “Pipecat ships a built-in development runner (pipecat.runner.run) that handles the server-side glue most bots need during development: creati…”
   • [claimed-docs] “This is the property that makes the same bot file portable across the development runner, Pipecat Cloud, and most production self-hosting se…”
   • [claimed-docs] “Pipecat Cloud offers different agent profiles, to best support your use case and compute needs.”

   proof ↗⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   Retell AI—none0/10

   The evidence only covers configurable data retention (auto-deleting transcripts/recordings/logs after a set period) for compliance, not an explicit opt-out or guarantee against using customer data for AI model training. No docs mention training-data usage policy, opt-out toggles, or contractual no-train clauses.

   • [claimed-docs] “Configure per-agent data retention to automatically delete call and chat data — transcripts, recordings, and logs — after a set period for c…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent. After the retention period expires, call and chat data associated with tha…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent.”

   proof ↗⚑ flag

   Pipecat—none0/10

   The evidence pack contains no mention of data-training opt-outs, data-usage policies, or privacy commitments regarding customer data and AI model training for either the open-source Pipecat framework or the hosted Pipecat Cloud service. Since Pipecat Cloud is a real hosted product where such a policy could plausibly exist, this axis applies, but no evidence supports it. Missing for 10: any privacy policy, ToS clause, or documentation stating customer data is not used to train models, and no independent corroboration.

   ⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round to Retell AI
   Retell AI!disputedcontradicted5/10

   Retell's docs describe per-agent configurable data retention with automatic, permanent deletion of call/chat transcripts, recordings, and logs after a set period (retell-docs-15, retell-docs-27, retell-docs-44), which is solid first-party evidence for the retention-control axis. However, an independent community report describes the company refusing to delete a user's stored data (credit card) on request, forcing the user to escalate to bank disputes/consumer-affairs complaints — a concrete real-world case where deletion did not work as a customer expected (retell-comm-14). Missing for 10: independent verification that call/chat data deletion itself (not just billing data) works as documented, and no public response/resolution to the deletion complaint.

   • [claimed-docs] “Configure per-agent data retention to automatically delete call and chat data — transcripts, recordings, and logs — after a set period for c…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent. After the retention period expires, call and chat data associated with tha…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent.”
   • [community] “Thread titled 'Retellai won't delete my credit card' - a user complained the company would not delete their stored credit card; commenters s…”

   proof ↗⚑ flag

   Pipecat—none0/10

   Pipecat is a self-hostable open-source framework and also offers a hosted Pipecat Cloud option that processes voice/audio data, so data-retention/deletion controls are a fair question for this product category, but the evidence pack contains no documentation of retention policies, deletion APIs, or data lifecycle controls — only unrelated features like recording/transcription and metrics.

   ⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Retell AI—none0/10

   The evidence pack covers data retention for call/chat data (transcripts, recordings, logs) but contains no mention of telemetry or usage-tracking opt-out controls for the product itself (e.g., CLI/SDK/dashboard analytics collection). Data retention (retell-docs-15/27/44) addresses deletion of customer call data, not opting out of Retell's own telemetry collection.

   ⚑ flag

   Pipecat—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

Call control

1. developerEscalate a live call to a human with warm or blind transfer, passing context along

   weight 2 · round to Retell AI
   Retell AI~partialclaimed4/10

   Retell's function-calling docs list 'transfer calls' as a built-in agent action alongside ending calls, booking appointments, etc., which supports the general concept of escalating a call to a human. However, the evidence never distinguishes warm vs. blind transfer modes nor describes passing conversational context/metadata to the receiving human agent. missing for 10: explicit warm-transfer vs blind-transfer configuration, evidence of context/data hand-off to the human agent, and any hands-on confirmation of this feature working in practice.

   • [claimed-docs] “Function calling lets Retell single or multi-prompt agents take real actions — transfer calls, end calls, book appointments, send SMS, and c…”
   • [claimed-docs] “Function calling transforms your AI agent from a conversational interface into an action-oriented assistant.”

   proof ↗⚑ flag

   Pipecat—none0/10

   The evidence pack covers Pipecat's telephony deployment (webhook/carrier integration for Twilio, Telnyx, etc.), conversation flow transitions via Pipecat Flows, and subagent hand-off for tasks, but nothing addresses call transfer to a human agent (warm or blind) or passing call context/metadata during such a transfer. This is a reasonable telephony axis for a voice AI framework, but no citation demonstrates or documents this capability.

   ⚑ flag

2. developerMy agent can send DTMF keypresses, navigate IVR menus, and detect or leave voicemail

   weight 1 · round drawn
   Retell AI—none0/10

   The evidence pack covers function calling, custom telephony/SIP trunking, transcription, and MCP tool integration, but nowhere mentions DTMF keypress sending, IVR menu navigation, or voicemail detection/leaving capabilities. This is a fair capability question for a telephony voice-agent platform, but no evidence confirms it is supported.

   ⚑ flag

   Pipecat—none0/10

   Pipecat clearly supports telephony transports (Twilio, Telnyx, Plivo, Exotel, SIP) for building call-handling bots, so this axis applies, but the evidence pack contains no mention of DTMF keypress sending, IVR menu navigation, or voicemail detection/leaving capabilities anywhere in the docs or community sources.

   • [claimed-docs] “Telephony bots have a different shape than WebRTC bots. The session doesn't start because a client sent your dispatcher an HTTP request — it…”
   • [claimed-docs] “the carrier (Twilio, Telnyx, Plivo, Exotel, or your SIP provider) is calling your webhook to tell you there's an inbound call”
   • [claimed-docs] “Any Transport: WebRTC, SIP, PSTN”
   • [claimed-docs] “python bot.py -t twilio -x your-name.ngrok.io”

   proof ↗⚑ flag

Campaigns

1. founderRun batch outbound call campaigns with scheduling and throughput controls

   weight 2 · round drawn
   Retell AI—none0/10

   Docs describe outbound vs inbound concurrency reservation (retell-docs-11, retell-docs-41) and custom telephony/SIP integration, but there is no evidence of a batch outbound campaign feature — no mention of scheduling calls, uploading contact lists, or campaign-level throughput controls beyond general concurrency limits. Axis is plausible for a voice-agent platform but unsupported by the evidence pack.

   • [claimed-docs] “Reserved inbound concurrency protects inbound calls from being crowded out by outbound traffic.”
   • [claimed-docs] “When `reserved_inbound_concurrency` is configured, outbound calls can use at most your concurrency limit minus the reserved amount.”
   • [claimed-docs] “Integrate Retell voice agents with your own telephony provider using elastic SIP trunking or imported numbers from Twilio, Telnyx, and Vonag…”

   proof ↗⚑ flag

   Pipecat—none0/10

   Pipecat's telephony docs cover per-call webhook-driven session handling (inbound/outbound single calls via Twilio/Telnyx/etc.) and unlimited concurrency on Pipecat Cloud, but there is no evidence of batch campaign management, call scheduling, pacing/throughput controls, or dialer-list orchestration for outbound campaigns — this is a fair ask for a telephony-capable voice AI framework but no such capability is documented.

   • [claimed-docs] “Telephony bots have a different shape than WebRTC bots. The session doesn't start because a client sent your dispatcher an HTTP request — it…”
   • [claimed-docs] “the carrier (Twilio, Telnyx, Plivo, Exotel, or your SIP provider) is calling your webhook to tell you there's an inbound call”
   • [claimed-docs] “python bot.py -t twilio -x your-name.ngrok.io”
   • [claimed-docs] “Pipecat Cloud supports unlimited concurrency.”
   • [claimed-docs] “Any Transport: WebRTC, SIP, PSTN”

   proof ↗⚑ flag

Numbers

1. developerProvision phone numbers and run both inbound and outbound calls through the platform's API

   weight 3 · round to Retell AI
   Retell AI✓fullprobed8/10

   Retell's docs show phone-number provisioning via native assignment (docs-32) or importing/SIP-trunking your own numbers from Twilio/Telnyx/Vonage (docs-10,40,49), and both inbound and outbound calling are explicitly supported and distinguished (reserved inbound concurrency vs. outbound traffic in docs-11/41). Full API/SDK coverage for voice endpoints (docs-2,33,46) and a CLI to manage phone numbers (docs-3,21) round out programmatic control, with a working keyless CLI/MCP probe corroborating API-level access (retell-probe-rt-1/2). Missing for 10: explicit documentation of a native 'buy a new number' provisioning endpoint (as opposed to importing existing numbers) and a concrete outbound-call code example.

   • [claimed-docs] “Build your first Retell AI phone agent in 15 minutes: create an account, pick a template, test in the dashboard, deploy to a phone number, a…”
   • [claimed-docs] “Integrate Retell voice agents with your own telephony provider using elastic SIP trunking or imported numbers from Twilio, Telnyx, and Vonag…”
   • [claimed-docs] “Reserved inbound concurrency protects inbound calls from being crowded out by outbound traffic.”
   • [claimed-docs] “This guide shows how to integrate Retell agents with your telephony provider and use your own numbers.”
   • [claimed-docs] “When `reserved_inbound_concurrency` is configured, outbound calls can use at most your concurrency limit minus the reserved amount.”
   • [claimed-docs] “You set up a SIP trunk, configure your number to point to it, and import that number to Retell.”
   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Retell provides official SDKs for Node.js and Python to simplify integration with our platform.”
   • [claimed-docs] “Install the Retell CLI to manage agents, phone numbers, knowledge bases, and other Retell resources from your terminal with simple commands.”
   • [claimed-docs] “The Retell CLI lets you manage Retell resources from your terminal.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @retell-ai/retell-cli --version` printed `retell 0.13.0 (OpenAPI 3.0.0, catalog v4)` keylessly …”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”

   proof ↗⚑ flag

   Pipecat~partialclaimed4/10

   Docs show Pipecat integrates with telephony carriers (Twilio, Telnyx, Plivo, Exotel, SIP) to receive inbound call webhooks and run bots over PSTN/SIP transports, and CLI examples show running a bot against Twilio (`python bot.py -t twilio -x ...`), covering the 'run calls' half of the story. However, there is no evidence Pipecat itself exposes an API to provision/purchase phone numbers — that is left to the carrier's own console/API, not documented as a Pipecat capability. Missing for 10: native phone-number provisioning API/CLI, outbound call initiation API details, and any first-party number-management workflow.

   • [claimed-docs] “Telephony bots have a different shape than WebRTC bots. The session doesn't start because a client sent your dispatcher an HTTP request — it…”
   • [claimed-docs] “the carrier (Twilio, Telnyx, Plivo, Exotel, or your SIP provider) is calling your webhook to tell you there's an inbound call”
   • [claimed-docs] “Any Transport: WebRTC, SIP, PSTN”
   • [claimed-docs] “python bot.py -t twilio -x your-name.ngrok.io”

   proof ↗⚑ flag

Sip

1. platform-engineerConnect my own carrier or PBX via SIP trunking (or import Twilio/Telnyx numbers) instead of being locked to bundled telephony

   weight 2 · round to Retell AI
   Retell AI✓fullclaimed8/10

   Official docs explicitly describe elastic SIP trunking and importing numbers from Twilio, Telnyx, and Vonage, walking through setting up a SIP trunk and pointing/importing numbers into Retell — directly matching the story. This is first-party documentation without independent hands-on corroboration of the SIP flow itself. Missing for 10: independent/community verification of a real SIP trunk setup working end-to-end, and detail on carrier-specific edge cases (codecs, failover, latency).

   • [claimed-docs] “Integrate Retell voice agents with your own telephony provider using elastic SIP trunking or imported numbers from Twilio, Telnyx, and Vonag…”
   • [claimed-docs] “This guide shows how to integrate Retell agents with your telephony provider and use your own numbers.”
   • [claimed-docs] “You set up a SIP trunk, configure your number to point to it, and import that number to Retell.”

   proof ↗⚑ flag

   Pipecat✓fullclaimed7/10

   Docs explicitly describe telephony bots being invoked by carrier webhooks including 'Twilio, Telnyx, Plivo, Exotel, or your SIP provider,' and highlight 'Any Transport: WebRTC, SIP, PSTN' plus a working CLI example (`python bot.py -t twilio -x ...ngrok.io`), showing genuine carrier/SIP flexibility rather than lock-in to one bundled provider. Missing for 10: concrete walkthrough of raw SIP trunk/PBX configuration (vs. just carrier webhooks), and independent/hands-on confirmation of a non-Twilio/Telnyx SIP trunk actually working in production.

   • [claimed-docs] “Telephony bots have a different shape than WebRTC bots. The session doesn't start because a client sent your dispatcher an HTTP request — it…”
   • [claimed-docs] “the carrier (Twilio, Telnyx, Plivo, Exotel, or your SIP provider) is calling your webhook to tell you there's an inbound call”
   • [claimed-docs] “Any Transport: WebRTC, SIP, PSTN”
   • [claimed-docs] “python bot.py -t twilio -x your-name.ngrok.io”

   proof ↗⚑ flag

Analytics

1. ai-native userThe platform's AI reviews my calls for me — scoring quality, flagging failures, and analyzing resolution automatically

   weight 2 · round to Retell AI
   Retell AI~partialclaimed6/10

   Retell's AI QA feature automatically scores sampled calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage, and the Analytics dashboard tracks success rate and other metrics, directly matching the story's core ask of automated call scoring and quality flagging. However, evidence shows AI QA only evaluates a 'sampled set' of calls (not all calls) against user-configured rules, and there's no documented automatic 'resolution analysis' narrative beyond metrics/dashboards, nor independent hands-on validation of AI QA accuracy. missing for 10: evidence of full (non-sampled) coverage, independent/third-party validation of AI QA scoring accuracy, and detail on automated resolution/failure-cause analysis beyond configured metrics.

   • [claimed-docs] “AI QA scores Retell calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage to surface quality trends and issues…”
   • [claimed-docs] “AI QA automatically evaluates a sampled set of your calls against rules and metrics you configure.”
   • [claimed-docs] “Build custom Retell analytics dashboards to track call and chat metrics like success rate, latency, cost, and concurrency, with charts, filt…”
   • [claimed-docs] “The Analytics dashboard charts your call and chat data so you can see how your agents are performing over time.”
   • [claimed-docs] “Built-in call success and sentiment scoring, plus custom fields synced to your CRM.”

   proof ↗⚑ flag

   Pipecat—none0/10

   Pipecat ships developer-facing dev-time Evals (describe expected conversation, run against real agent) plus metrics/transcripts/audio-recording primitives, but there is no evidence of an AI system that automatically reviews production calls, scores quality, flags failures, or analyzes resolution after the fact — Evals is a pre-deployment testing tool, not post-call analytics.

   • [claimed-docs] “Pipecat Evals is the framework's built-in system for testing agent behavior. You describe a conversation and the behavior you expect, and Pi…”
   • [claimed-docs] “You describe a conversation and the behavior you expect, and Pipecat runs it against your real agent (the same pipeline, the same services, …”
   • [claimed-docs] “Pipecat offers built-in metrics for both, which can be enabled with straightforward configuration options.”
   • [claimed-docs] “Pipecat's turn events make it easy to collect both user and assistant messages as they occur.”

   proof ↗⚑ flag

2. founderSee call analytics — success rates, durations, outcomes, sentiment — in dashboards without building my own

   weight 2 · round to Retell AI
   Retell AI✓fullclaimed8/10

   Retell docs describe a built-in Analytics dashboard with charts, filters, and breakdowns for success rate, latency, cost, and concurrency, plus AI QA scoring for sentiment, hallucination, and tool usage—directly matching the founder's need for out-of-box call analytics. Missing for 10: independent/hands-on confirmation of dashboard usability and no evidence of exportable reports or deeper outcome breakdowns beyond what's documented.

   • [claimed-docs] “Build custom Retell analytics dashboards to track call and chat metrics like success rate, latency, cost, and concurrency, with charts, filt…”
   • [claimed-docs] “The Analytics dashboard charts your call and chat data so you can see how your agents are performing over time.”
   • [claimed-docs] “AI QA scores Retell calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage to surface quality trends and issues…”
   • [claimed-docs] “AI QA automatically evaluates a sampled set of your calls against rules and metrics you configure.”
   • [claimed-docs] “Built-in call success and sentiment scoring, plus custom fields synced to your CRM.”

   proof ↗⚑ flag

   Pipecat—none0/10

   Pipecat exposes low-level building blocks (metrics, transcripts, audio recording, evals) that a developer could use to construct analytics, and a community debugging tool (Whisker) for pipeline tracing — but there is no evidence of a built-in dashboard surfacing success rates, call durations, outcomes, or sentiment without the founder building it themselves.

   • [claimed-docs] “Pipecat offers built-in metrics for both, which can be enabled with straightforward configuration options.”
   • [claimed-docs] “Pipecat's turn events make it easy to collect both user and assistant messages as they occur.”
   • [claimed-docs] “Pipecat's \`AudioBufferProcessor\` makes it easy to capture high-quality audio recordings of both the user and bot during interactions.”
   • [community] “With Whisker you can: View a live graph of your pipeline, watch frame processors flash in real time, select a processor to inspect its frame…”
   • [claimed-docs] “You describe a conversation and the behavior you expect, and Pipecat runs it against your real agent (the same pipeline, the same services, …”

   proof ↗⚑ flag

Monitoring

1. platform-engineerMonitor live calls in production and get alerts when agents misbehave or error rates spike

   weight 1 · round to Retell AI
   Retell AI~partialclaimed6/10

   Retell provides real building blocks for production monitoring — a customizable analytics dashboard tracking success rate, latency, cost and concurrency (retell-docs-13/42), AI QA that scores calls for hallucination, sentiment and tool-usage issues (retell-docs-14/43), and webhooks that push real-time event notifications (retell-docs-12/24) which a platform engineer could wire into an alerting pipeline. However there is no documented native alerting/threshold system (e.g., automatic notification when error rates spike or an agent misbehaves) or a live in-call monitoring view — engineers must build that themselves on top of webhooks/dashboard APIs. Missing for 10: built-in threshold-based alerts or anomaly detection, a real-time 'in-progress calls' monitoring view, and any independent evidence that alerting/monitoring works reliably in production.

   • [claimed-docs] “Webhooks allow your application to receive real-time notifications about events that occur in your Retell AI account.”
   • [claimed-docs] “Build custom Retell analytics dashboards to track call and chat metrics like success rate, latency, cost, and concurrency, with charts, filt…”
   • [claimed-docs] “AI QA scores Retell calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage to surface quality trends and issues…”
   • [claimed-docs] “webhooks push data to your application as events happen, making your integrations more efficient and responsive.”
   • [claimed-docs] “The Analytics dashboard charts your call and chat data so you can see how your agents are performing over time.”
   • [claimed-docs] “AI QA automatically evaluates a sampled set of your calls against rules and metrics you configure.”

   proof ↗⚑ flag

   Pipecat~partialcommunity4/10

   Pipecat provides building blocks for observability — built-in metrics (docs-5), turn-event transcripts (docs-4), audio recording (docs-3/21/29), and the community-highlighted Whisker tool that shows a live pipeline graph and frame-level debugger (pipecat-comm-3/4/5) — but these are development/debugging aids, not a production monitoring or alerting system. There is no evidence of dashboards for live production calls, error-rate-spike detection, or alert/notification integrations (e.g., PagerDuty, Slack alerts) for misbehaving agents. Missing for 10: production-grade dashboards for live call monitoring, automated error-rate/anomaly alerting, and integration with on-call/alerting systems.

   • [claimed-docs] “Pipecat offers built-in metrics for both, which can be enabled with straightforward configuration options.”
   • [claimed-docs] “Pipecat's turn events make it easy to collect both user and assistant messages as they occur.”
   • [community] “As someone who spends a lot of time looking at timestamped log lines to debug Pipecat pipelines, I'm a big fan of this work... For 1 and 3 […”
   • [community] “With Whisker you can: View a live graph of your pipeline, watch frame processors flash in real time, select a processor to inspect its frame…”
   • [community] “I had been thinking of working on something like this recently as a way to debug Pipecat pipelines. But the work Aleix has done goes far bey…”

   proof ↗⚑ flag

Testing

1. developerTest agents with simulated conversations or evals before putting them on real phone calls

   weight 2 · round to Pipecat
   Retell AI~partialclaimed5/10

   Retell's dashboard has a free 'Test' web-call button that lets developers try an agent before assigning it to a real phone number, and AI QA can score calls on hallucination, accuracy, sentiment, etc., which supports some testing/analytics workflow. However there's no documented feature for automated simulated-conversation test suites or eval scripts run pre-deployment — AI QA appears to operate on sampled real calls rather than synthetic scripted evals. Missing for 10: dedicated simulation/eval framework for scripted test conversations, batch eval tooling, and any independent verification that pre-call testing catches issues before production use.

   • [claimed-docs] “Click the "Test" button to start a web call with your agent... This step is free and doesn't need a phone number or payment method.”
   • [claimed-docs] “Click the "Test" button to start a web call with your agent”
   • [claimed-docs] “AI QA scores Retell calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage to surface quality trends and issues…”
   • [claimed-docs] “AI QA automatically evaluates a sampled set of your calls against rules and metrics you configure.”
   • [claimed-docs] “Build custom Retell analytics dashboards to track call and chat metrics like success rate, latency, cost, and concurrency, with charts, filt…”

   proof ↗⚑ flag

   Pipecat✓fullprobed8/10

   Pipecat Evals is a documented, built-in framework feature letting developers describe conversations/expected behavior and run them against the real agent pipeline before deployment, and the runtime probe confirms a working `pipecat eval` CLI subcommand exists alongside `init`/`cloud`. This directly addresses simulated-conversation testing before real phone calls, though evidence doesn't show telephony-specific eval scenarios or third-party validation of eval accuracy. Missing for 10: independent/community corroboration of Evals in practice, and explicit examples of testing telephony-specific call flows via evals.

   • [claimed-docs] “Pipecat Evals is the framework's built-in system for testing agent behavior. You describe a conversation and the behavior you expect, and Pi…”
   • [claimed-docs] “You describe a conversation and the behavior you expect, and Pipecat runs it against your real agent (the same pipeline, the same services, …”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Pipecat CLI (pypi pipecat-ai[cli]) ran keylessly via uvx — `pipecat --help` lists init (pr…”

   proof ↗⚑ flag

Post call

1. developerExtract structured data from every call — outcomes, entities, dispositions — delivered via API or webhook after the call

   weight 2 · round to Retell AI
   Retell AI✓fullclaimed7/10

   Retell explicitly supports post-call structured extraction: built-in call success/sentiment scoring plus custom fields synced to CRM, delivered via real-time webhooks and viewable/aggregated in the analytics/AI QA dashboards. This directly covers outcomes (success rate), dispositions (sentiment), and entities (custom fields) delivered via API/webhook as the story requires. Missing for 10: no independent/hands-on confirmation of the specific post-call-analysis JSON schema or webhook payload structure, and no detail on how custom entity fields are defined/configured.

   • [claimed-docs] “Built-in call success and sentiment scoring, plus custom fields synced to your CRM.”
   • [claimed-docs] “Webhooks allow your application to receive real-time notifications about events that occur in your Retell AI account.”
   • [claimed-docs] “webhooks push data to your application as events happen, making your integrations more efficient and responsive.”
   • [claimed-docs] “Build custom Retell analytics dashboards to track call and chat metrics like success rate, latency, cost, and concurrency, with charts, filt…”
   • [claimed-docs] “The Analytics dashboard charts your call and chat data so you can see how your agents are performing over time.”
   • [claimed-docs] “AI QA scores Retell calls on hallucination, knowledge base accuracy, latency, sentiment, and tool usage to surface quality trends and issues…”
   • [claimed-docs] “AI QA automatically evaluates a sampled set of your calls against rules and metrics you configure.”

   proof ↗⚑ flag

   Pipecat~partialclaimed4/10

   Pipecat provides building blocks that a developer could use to build this — Flows functions can 'process data by interfacing with external systems and APIs' (function calling) and turn events/transcripts capture user and assistant messages, plus telephony webhooks for call events — but there is no documented turnkey feature for automatic post-call structured extraction (outcomes/entities/dispositions) delivered via a dedicated API or webhook. Missing for 10: a built-in post-call summary/disposition-extraction pipeline, a documented structured-output schema for call outcomes, and an out-of-box webhook delivery mechanism specifically for post-call analytics (only inbound-call webhooks and generic function-calling/transcript primitives are documented).

   • [claimed-docs] “Functions in Pipecat Flows serve two key purposes: 1. Process data ... 2. Progress the conversation by transitioning between nodes”
   • [claimed-docs] “Functions in Pipecat Flows serve two key purposes: Process data by interfacing with external systems and APIs...Progress the conversation by…”
   • [claimed-docs] “Functions in Pipecat Flows serve two key purposes: 1. Process data by interfacing with external systems and APIs... 2. Progress the conversa…”
   • [claimed-docs] “Flows auto-derives the function's metadata — name, description, parameter properties (with their descriptions), and which parameters are req…”
   • [claimed-docs] “Pipecat's turn events make it easy to collect both user and assistant messages as they occur.”
   • [claimed-docs] “Telephony bots have a different shape than WebRTC bots. The session doesn't start because a client sent your dispatcher an HTTP request — it…”
   • [claimed-docs] “the carrier (Twilio, Telnyx, Plivo, Exotel, or your SIP provider) is calling your webhook to tell you there's an inbound call”

   proof ↗⚑ flag

Tools

1. ai-native userMy voice agent can plug in MCP servers as tool sources so one integration grants it whole toolsets mid-call

   weight 2 · round to Retell AI
   Retell AI✓fullclaimed8/10

   Retell documents connecting single/multi-prompt voice agents to remote MCP servers so the agent can call the server's tools mid-call (retell-docs-6/23/36), directly matching the story, and this is distinct from Retell's own client-side MCP server for building agents. missing for 10: independent third-party corroboration of mid-call MCP tool invocation in production (only first-party docs and a probe of the client-facing MCP server, not the agent-as-MCP-client tool-call path, are available).

   • [claimed-docs] “Connect a Retell single- or multi-prompt agent to a remote MCP server so it can call the server's tools during a live voice or chat conversa…”
   • [claimed-docs] “Connect your single- or multi-prompt agent to a remote Model Context Protocol (MCP) server, and the agent can call that server's tools durin…”
   • [claimed-docs] “Connect your single- or multi-prompt agent to a remote [Model Context Protocol (MCP)] server, and the agent can call that server's tools dur…”
   • [claimed-docs] “Function calling lets Retell single or multi-prompt agents take real actions — transfer calls, end calls, book appointments, send SMS, and c…”
   • [claimed-docs] “Function calling transforms your AI agent from a conversational interface into an action-oriented assistant.”

   proof ↗⚑ flag

   Pipecat—none0/10

   The evidence only shows Pipecat's context-hub *acting as* an MCP server for coding-agent tooling (pipecat-docs-11/38), which is the opposite direction from what the story asks — a voice agent consuming external MCP servers as tool sources mid-call. Function/tool calling is documented via Pipecat Flows (pipecat-docs-6/22/30/39), but no evidence shows the voice agent connecting to or importing toolsets from external MCP servers during a call.

   • [claimed-docs] “pipecat context-hub install registers the hub as an MCP server with your coding agent and builds the local index”
   • [claimed-docs] “pipecat context-hub install` registers the hub as an MCP server with your coding agent and builds the local index”
   • [claimed-docs] “Functions in Pipecat Flows serve two key purposes: 1. Process data ... 2. Progress the conversation by transitioning between nodes”
   • [claimed-docs] “Functions in Pipecat Flows serve two key purposes: Process data by interfacing with external systems and APIs...Progress the conversation by…”

   proof ↗⚑ flag

2. developerMy agent can call external APIs and custom functions mid-conversation and speak the result without awkward dead air

   weight 3 · round to Pipecat
   Retell AI~partialprobed6/10

   Retell's docs clearly document function calling that lets agents call external APIs mid-conversation (retell-docs-5, retell-docs-35) and even connect to remote MCP servers to invoke tools live during a call (retell-docs-6, retell-docs-23, retell-docs-36), confirmed by a keyless runtime probe showing the hosted MCP server exposing invoke_api_endpoint tools (retell-probe-rt-2). However, no evidence specifically addresses the 'no awkward dead air' requirement — there's no documented filler-speech, streaming-response, or latency-masking mechanism specifically tied to function-call execution; transcription-mode docs (retell-docs-9/39/48) discuss general STT latency tradeoffs, not function-call dead-air handling. missing for 10: explicit filler/backchannel speech during function execution, hands-on demonstration of seamless mid-call tool latency, independent confirmation of no dead-air behavior.

   • [claimed-docs] “Function calling lets Retell single or multi-prompt agents take real actions — transfer calls, end calls, book appointments, send SMS, and c…”
   • [claimed-docs] “Function calling transforms your AI agent from a conversational interface into an action-oriented assistant.”
   • [claimed-docs] “Connect a Retell single- or multi-prompt agent to a remote MCP server so it can call the server's tools during a live voice or chat conversa…”
   • [claimed-docs] “Connect your single- or multi-prompt agent to a remote Model Context Protocol (MCP) server, and the agent can call that server's tools durin…”
   • [claimed-docs] “Connect your single- or multi-prompt agent to a remote [Model Context Protocol (MCP)] server, and the agent can call that server's tools dur…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the hosted MCP server at https://mcp.retellai.com completed a full KEYLESS JSON-RPC initialize handshak…”

   proof ↗⚑ flag

   Pipecat✓fullclaimed8/10

   Pipecat Flows functions explicitly support 'processing data by interfacing with external systems and APIs' mid-conversation and auto-derive function schemas from code, while interruption handling and turn detection docs describe how the pipeline manages timing/dead-air around such calls. Missing for 10: no concrete hands-on example/demo showing latency-masking (e.g., filler speech) during a live API call, and no independent benchmark of perceived dead-air during function execution.

   • [claimed-docs] “Functions in Pipecat Flows serve two key purposes: 1. Process data ... 2. Progress the conversation by transitioning between nodes”
   • [claimed-docs] “Functions in Pipecat Flows serve two key purposes: Process data by interfacing with external systems and APIs...Progress the conversation by…”
   • [claimed-docs] “Functions in Pipecat Flows serve two key purposes: 1. Process data by interfacing with external systems and APIs... 2. Progress the conversa…”
   • [claimed-docs] “Flows auto-derives the function's metadata — name, description, parameter properties (with their descriptions), and which parameters are req…”
   • [claimed-docs] “When the user starts speaking while the bot is talking, the bot stops immediately, in-flight work is cancelled, and the pipeline is ready fo…”
   • [claimed-docs] “Interruptions (also called barge-in) let the user talk over the bot. When the user starts speaking while the bot is talking, the bot stops i…”
   • [claimed-docs] “Smart Turn Detection uses an advanced machine learning model to determine when a user has finished speaking and your bot should respond.”

   proof ↗⚑ flag

Recording

1. platform-engineerRetrieve full call recordings and transcripts programmatically for every call

   weight 2 · round to Retell AI
   Retell AI~partialclaimed6/10

   Docs confirm that Retell stores per-call transcripts and recordings (referenced in data-retention docs) and that full voice/chat API endpoint coverage exists via official SDKs, implying programmatic retrieval, but no evidence pack item explicitly documents a 'get call' or 'list calls' API endpoint returning recording URLs/transcript text, nor examples of pulling them via SDK/CLI. Missing for 10: explicit API reference for call/recording/transcript retrieval endpoints, CLI/SDK code samples showing recording download or transcript fetch, and any independent confirmation of this working end-to-end.

   • [claimed-docs] “Configure per-agent data retention to automatically delete call and chat data — transcripts, recordings, and logs — after a set period for c…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent. After the retention period expires, call and chat data associated with tha…”
   • [claimed-docs] “Retell allows you to configure a data retention period per agent.”
   • [claimed-docs] “Official Retell SDKs for Node.js and Python. Typed clients with API key auth, structured errors, and full voice and chat endpoint coverage.”
   • [claimed-docs] “Retell provides official SDKs for Node.js and Python to simplify integration with our platform.”
   • [claimed-docs] “Webhooks allow your application to receive real-time notifications about events that occur in your Retell AI account.”
   • [claimed-docs] “webhooks push data to your application as events happen, making your integrations more efficient and responsive.”

   proof ↗⚑ flag

   Pipecat~partialclaimed5/10

   Pipecat provides building blocks — AudioBufferProcessor for capturing audio recordings and turn-event based transcript collection — that a platform engineer could wire into their own storage/retrieval system, but there's no documented built-in centralized recordings/transcripts store or query API for retrieving them programmatically across all calls. Missing for 10: a first-party recordings/transcripts storage service or retrieval API, evidence of per-call metadata/indexing, and any hands-on confirmation of retrieving historical call data at scale.

   • [claimed-docs] “Pipecat's \`AudioBufferProcessor\` makes it easy to capture high-quality audio recordings of both the user and bot during interactions.”
   • [claimed-docs] “Pipecat's AudioBufferProcessor makes it easy to capture high-quality audio recordings of both the user and bot during interactions.”
   • [claimed-docs] “Pipecat's `AudioBufferProcessor` makes it easy to capture high-quality audio recordings of both the user and bot during interactions.”
   • [claimed-docs] “Pipecat's turn events make it easy to collect both user and assistant messages as they occur.”

   proof ↗⚑ flag

Transcription

1. developerGet accurate real-time transcription with control over the STT provider, language models, or key terms

   weight 2 · round to Pipecat
   Retell AI~partialcommunity4/10

   Retell docs confirm real-time transcription with a documented latency/accuracy trade-off (interim vs context-aware endpointing), giving developers some control over accuracy tuning, and community notes hint at underlying provider choices (e.g., Deepgram) for voice pipelines. However, there is no documented ability to select or swap STT providers, choose an ASR language model, or configure custom vocabulary/key-term boosting for transcription accuracy. Missing for 10: explicit STT provider selection API, language model choice for transcription, custom vocabulary/key-term boosting support, and independent accuracy benchmarks.

   • [claimed-docs] “Pick the right Retell transcription mode to balance accuracy against latency — interim results for speed or context-aware results for higher…”
   • [claimed-docs] “Real time transcription is often a trade off between latency and accuracy.”
   • [claimed-docs] “optimize for speed: uses the latest interim results with a low endpointing setting... optimize for accuracy: uses the results with a higher …”
   • [community] “Founder (yanyan_evie) noted voice provider costs differ: 'The different providers have different prices. openai tts & deepgram are cheaper, …”

   proof ↗⚑ flag

   Pipecat~partialcommunity6/10

   Docs and community evidence confirm Pipecat lets developers swap STT providers and language models with one line of code (200+ services, local Whisper/Ollama), and it exposes STT-specific latency metrics (TTFS, P99) for tuning accuracy/speed tradeoffs. However, there is no evidence of support for passing custom vocabulary or 'key terms'/boost-word lists to STT providers, and no independent benchmark validating transcription accuracy claims. Missing for 10: explicit key-terms/vocabulary-boost configuration support, and independent/hands-on accuracy validation.

   • [claimed-docs] “Swap speech, language, and vision services from any of 200+ integrated providers and services — usually one line of code.”
   • [claimed-docs] “Pipecat ships with measured P99 latency values for each supported service so that turn detection can account for this delay automatically.”
   • [claimed-docs] “Time To Final Segment (TTFS) measures how long it takes from the moment a user stops speaking until the STT service delivers the final trans…”
   • [community] “Local inference is already supported via Pipecat, you can use ollama or any custom OpenAI endpoint. Local STT is also supported via whisper,…”
   • [community] “Yes, Pipecat already supports that natively, so this can be done easily with ollama... Also, check out any provider they support, and it can…”

   proof ↗⚑ flag

Voices

1. founderClone a custom brand voice and use it for my agents, with a documented consent process

   weight 2 · round to Retell AI
   Retell AI~partialclaimed5/10

   Retell documents voice cloning from uploaded audio files (up to 25 files) and lets you attach the cloned voice to an agent via the voice selector, covering the core 'clone and use a custom voice' capability. However, none of the evidence describes any documented consent-verification step (e.g., consent recording, rights attestation) as part of the cloning flow. Missing for 10: an explicit consent-collection/verification mechanism in the clone-voice API or dashboard docs, and any compliance guidance tying voice cloning to consent requirements.

   • [claimed-docs] “Add custom voices to your Retell agent — search ElevenLabs community voices, import a voice clone, or train a clone for a unique brand-speci…”
   • [claimed-docs] “Clone a voice from audio files”
   • [claimed-docs] “Audio files to use for voice cloning. Up to 25 files allowed.”
   • [claimed-docs] “You can also add a voice clone by clicking "Add custom voice" in the voice selector.”

   proof ↗⚑ flag

   Pipecat—none0/10

   Pipecat's evidence covers swapping among 200+ TTS/STT providers and general voice pipeline features, but nothing addresses custom voice cloning or any documented consent process for cloning a brand voice — this is a fair question since Pipecat is a voice-AI framework, but no evidence supports it being delivered.

   • [claimed-docs] “Swap speech, language, and vision services from any of 200+ integrated providers and services — usually one line of code.”
   • [claimed-docs] “Open source Python framework for building voice and multimodal AI pipelines. Orchestrate 150+ AI services with ultra-low latency.”

   proof ↗⚑ flag

2. developerChoose from a broad voice library or plug in multiple TTS providers to get the voice I want

   weight 2 · round to Pipecat
   Retell AI✓fullcommunity7/10

   Docs confirm a searchable voice library (ElevenLabs community voices) plus voice cloning options (retell-docs-7, retell-docs-37, retell-docs-38), and a founder community comment independently confirms multiple underlying TTS providers (OpenAI TTS, Deepgram, ElevenLabs) with different pricing tiers (retell-comm-10), showing developers can indeed pick across providers/voices. Missing for 10: a first-party docs page enumerating all supported TTS providers and API-level provider-switching parameters beyond the voice-selector UI, and independent hands-on comparison of voice quality across providers.

   • [claimed-docs] “Add custom voices to your Retell agent — search ElevenLabs community voices, import a voice clone, or train a clone for a unique brand-speci…”
   • [claimed-docs] “You can also add a voice clone by clicking "Add custom voice" in the voice selector.”
   • [claimed-docs] “In the voice selector, you can click "Add custom voice" to search and add publicly available community voices.”
   • [community] “Founder (yanyan_evie) noted voice provider costs differ: 'The different providers have different prices. openai tts & deepgram are cheaper, …”

   proof ↗⚑ flag

   Pipecat✓fullcommunity8/10

   Docs and community evidence confirm Pipecat supports swapping speech services (TTS/STT) across 200+ integrated providers with typically one line of code, plus community confirmation of easy onboarding of any provider and local TTS/STT options (ollama, whisper). Missing for 10: an explicit enumerated 'voice library' list or curated voice catalog UI, and independent benchmark of ease/quality across providers beyond vendor/community anecdotes.

   • [claimed-docs] “Swap speech, language, and vision services from any of 200+ integrated providers and services — usually one line of code.”
   • [claimed-docs] “Open source Python framework for building voice and multimodal AI pipelines. Orchestrate 150+ AI services with ultra-low latency.”
   • [community] “Local inference is already supported via Pipecat, you can use ollama or any custom OpenAI endpoint. Local STT is also supported via whisper,…”
   • [community] “Yes, Pipecat already supports that natively, so this can be done easily with ollama... Also, check out any provider they support, and it can…”
   • [claimed-docs] “Pipecat ships with measured P99 latency values for each supported service so that turn detection can account for this delay automatically.”

   proof ↗⚑ flag