ElevenLabs Agents vs LiveKit Agents

Agent ops

1. ai-native userMy coding agent can provision a complete voice agent end to end — create the agent, attach a number, and place a call — through the API, CLI, or MCP without touching the dashboard

   weight 3 · round to ElevenLabs Agents
   ElevenLabs Agents~partialprobed6/10

   Docs and runtime probes confirm agents can be created and managed via API, a real CLI (@elevenlabs/cli, verified live on npm), and a hosted MCP server (verified live with OAuth), all without the dashboard (docs-1,3,4,27,28,38; probe-rt-1,2). Phone-number attachment is also supported via SIP trunking to route calls to agents (docs-12,36), but there is no explicit documentation or evidence of actually placing/initiating an outbound call end-to-end through the API/CLI/MCP — only routing inbound infrastructure integration is described. Missing for 10: explicit API/CLI/MCP call to place an outbound call, and a concrete end-to-end walkthrough combining create+attach+call in one non-dashboard flow.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “Use the ElevenLabs agents skill to build and manage voice agents from your AI coding assistant: npx skills add elevenlabs/skills --skill age…”
   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
   • [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”
   • [claimed-docs] “Route calls to AI agents without changing your existing phone infrastructure”
   • [claimed-docs] “connect the hosted MCP server to Claude or any other MCP client to create and manage agents through natural language”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”

   proof ↗⚑ flag

   LiveKit Agents~partialprobed5/10

   LiveKit provides a real CLI (`lk`), a documented telephony/SIP stack for provisioning inbound/outbound trunks and placing calls, and framework docs explicitly note it's 'built for coding agents like Claude Code, Cursor, and Codex,' suggesting API/CLI-driven workflows are possible without the dashboard. However, the MCP support documented is for the agent's own tool-calling (consuming MCP servers as tools), not an MCP server exposing agent-provisioning/number-attachment/call-placing actions to a coding agent, and no single evidenced workflow shows an agent programmatically creating an agent, attaching a number, and placing a call end-to-end via CLI/API in one flow — the Agent Builder path shown is dashboard/browser-based (no-code), which contradicts the 'without touching dashboard' framing. Missing for 10: an MCP server (or CLI/API recipe) that lets a coding agent itself create an agent, provision/attach a phone number, and place a call in one documented end-to-end sequence.

   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”
   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [claimed-docs] “Outbound trunks are used to place outgoing calls.”
   • [probe] “official CLI documented at https://github.com/livekit/livekit-cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): `lk --version` printed `lk version 2.18.6` after a plain `brew install livekit-cli` — the official CLI …”
   • [claimed-docs] “LiveKit is built for coding agents like Claude Code, Cursor, and Codex.”
   • [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
   • [claimed-docs] “LiveKit Agents has first-class support for Model Context Protocol (MCP) servers.”

   proof ↗⚑ flag

2. ai-native userThe platform's own AI helps me author agents — generating or improving prompts, flows, and test cases from a description

   weight 1 · round to ElevenLabs Agents
   ElevenLabs Agents~partialprobed5/10

   ElevenLabs documents that an external AI assistant (e.g., Claude) can create, configure, and manage agents through natural language via the hosted MCP server, and agent-testing docs describe turning real conversations into test cases for regression testing — both are AI-assisted authoring flows. However, this relies on a third-party AI client (Claude) rather than a built-in, platform-native 'describe your agent and we generate it' copilot for prompts/flows within the dashboard itself. Missing for 10: evidence of a first-party generative wizard that authors prompts/flows purely from a natural-language description inside the ElevenLabs UI, and evidence of AI-generated (not just conversation-derived) test-case authoring.

   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
   • [claimed-docs] “connect the hosted MCP server to Claude or any other MCP client to create and manage agents through natural language”
   • [claimed-docs] “Transform real conversations into test cases when you find an interaction where the agent underperformed.”
   • [claimed-docs] “Agent testing lets you verify conversational responses, tool usage, and full multi-turn outcomes before you deploy.”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   Evidence shows a no-code 'Agent Builder' for browser prototyping and a testing framework, but nothing indicates the platform itself uses AI to generate or improve prompts, flows, or test cases from a natural-language description of the desired agent.

   • [claimed-docs] “LiveKit Agent Builder to prototype and deploy agents directly in your browser without writing code”
   • [claimed-docs] “LiveKit Agent Builder: Prototype and deploy voice agents directly in your browser, without writing any code.”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes... Agent simulations run end-to-end conversations between your agent and…”

   proof ↗⚑ flag

Build

1. developerBuild a working phone voice agent — prompt, voice, and phone number — and take my first live call within an hour

   weight 3 · round to LiveKit Agents
   ElevenLabs Agents~partialclaimed7/10

   Docs show a clear quickstart path — create an agent via dashboard/API, pick from 5k+ voices, and connect phone numbers via SIP trunking to route calls to the agent without changing existing phone infrastructure — which together cover the prompt+voice+phone number pieces of the story. However, there's no explicit end-to-end 'first call in under an hour' walkthrough or independent timing evidence, and phone-number provisioning specifics (vs. just SIP integration) are thin. Missing for 10: an explicit timed/complete quickstart-to-live-call walkthrough, first-party or independent evidence of the actual time-to-first-call, and clearer native phone-number purchase/setup docs beyond SIP trunking.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “Select from 5k+ voices across 31 languages with customization options”
   • [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”
   • [claimed-docs] “Route calls to AI agents without changing your existing phone infrastructure”
   • [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”
   • [claimed-docs] “This assistant can be embedded into your website or app to provide real-time support to your customers”

   proof ↗⚑ flag

   LiveKit Agents✓fullprobed7/10

   Docs show a <10‑minute voice-assistant quickstart (prompt+voice) plus a dedicated telephony/SIP stack for inbound/outbound calls (trunks, phone number), and runtime probes confirm the framework and self-hosted server actually install and boot without extra keys. Together these cover prompt, voice, and phone number needed for a first live call within an hour. Missing for 10: a single unified, hands-on-verified tutorial that walks through phone-number provisioning and first live call end-to-end (currently voice-quickstart and telephony docs are separate), and independent (non-vendor) confirmation of the 'within an hour' timeline.

   • [claimed-docs] “Build and deploy a simple voice assistant in less than 10 minutes.”
   • [claimed-docs] “Build and deploy a simple voice assistant with Python or Node.js in less than 10 minutes.”
   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [claimed-docs] “Enable your voice agent to make or take phone calls.”
   • [claimed-docs] “Outbound trunks are used to place outgoing calls.”
   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `lk --version` printed `lk version 2.18.6` after a plain `brew install livekit-cli` — the official CLI …”
   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”

   proof ↗⚑ flag

2. developerRun conversations in multiple languages, including detecting and switching language mid-call

   weight 2 · round to ElevenLabs Agents
   ElevenLabs Agents~partialclaimed4/10

   Docs confirm broad multi-language support (70+ languages, language-specific voice settings) and voice-switching for multi-character use, but there is no explicit documentation of automatic language detection or switching languages mid-call. Missing for 10: explicit language-detection/mid-call switching feature docs, and any hands-on demonstration of a live language switch during a single conversation.

   • [claimed-docs] “Configure, deploy and monitor natural, human-sounding agents in 70+ languages with leading accuracy and ultra-low latency across voice or ch…”
   • [claimed-docs] “You can customize various aspects of your AI agent's voice to create a more natural and engaging conversation experience. This includes cont…”
   • [claimed-docs] “Enable your agent to switch between different voices for multi-character conversations, storytelling, and language tutoring.”
   • [claimed-docs] “Select from 5k+ voices across 31 languages with customization options”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   No evidence in the pack addresses multilingual conversation support, language detection, or dynamic language switching mid-call; the docs cover turn detection, interruption handling, tool use, MCP, and telephony but never mention STT/TTS language selection or switching logic. This is a fair axis for a voice-agent framework (many STT/TTS providers support multi-language), so absence of evidence means 'none' rather than 'na'.

   ⚑ flag

3. founderDesign multi-step conversation flows in a visual builder with branching, states, and handoffs without writing code

   weight 2 · round to ElevenLabs Agents
   ElevenLabs Agents~partialclaimed6/10

   Docs explicitly claim a 'visual workflow builder' for multi-step workflows, plus dashboard-based agent creation, conversation-flow settings (turn-taking, interruptions, timeouts), and tools/handoff-like actions (webhooks, client tools) — supporting the no-code, branching, multi-step story. However, evidence never details branching logic, explicit 'states' or agent-to-agent 'handoffs' within the builder, nor independent hands-on confirmation of the visual builder's depth. Missing for 10: concrete detail/screenshots of branching and state logic in the builder, explicit handoff-between-agents feature description, and independent/third-party corroboration of the builder's no-code sufficiency.

   • [claimed-docs] “Build multi-step workflows with visual workflow builder”
   • [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”
   • [claimed-docs] “Configure turn-taking, interruptions, and timeout settings”
   • [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”
   • [claimed-docs] “Enable agents to call clients & APIs to perform actions”

   proof ↗⚑ flag

   LiveKit Agents~partialclaimed3/10

   LiveKit mentions a browser-based 'Agent Builder' for no-code prototyping of voice agents, but the evidence never describes visual branching, explicit states, or handoff design — the core framework is fundamentally code-first (Python/Node.js) with tools, MCP, and turn-detection logic. missing for 10: documentation of branching/state UI, handoff modeling, or any screenshots/examples of the Agent Builder's flow-design capabilities beyond a vague no-code prototyping claim.

   • [claimed-docs] “Prototype and deploy voice agents directly in your browser, without writing any code.”
   • [claimed-docs] “LiveKit Agent Builder to prototype and deploy agents directly in your browser without writing code”
   • [claimed-docs] “LiveKit Agent Builder: Prototype and deploy voice agents directly in your browser, without writing any code.”

   proof ↗⚑ flag

Personalization

1. developerInject dynamic variables and per-caller context at call time so each conversation is personalized

   weight 2 · round to ElevenLabs Agents
   ElevenLabs Agents✓fullclaimed8/10

   Docs explicitly state 'Use dynamic variables and overrides for per-conversation customization' as a core build feature, directly matching the story of injecting per-caller context at call time. Missing for 10: no independent/hands-on example showing a real dynamic-variable injection call in code, and no detail on variable scoping/limits beyond the single doc mention.

   • [claimed-docs] “Use dynamic variables and overrides for per-conversation customization”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   The evidence pack covers tool use, MCP, telephony, testing, and turn detection, but nothing addresses injecting dynamic variables or per-caller context (e.g., participant/room metadata, job context) into an agent's prompt or session at call start. Missing for 10: any documentation of job/participant metadata APIs, per-call context injection, or dynamic prompt personalization.

   ⚑ flag

2. developerGround the agent on my documents with a built-in knowledge base or RAG so it answers from my content

   weight 2 · round to ElevenLabs Agents
   ElevenLabs Agents✓fullclaimed8/10

   Docs explicitly state agents support uploading documents and enabling RAG for grounded responses, directly matching the story's built-in knowledge base/RAG capability. Missing for 10: independent/hands-on corroboration of RAG quality and details on knowledge base management (chunking, size limits, sources).

   • [claimed-docs] “Upload documents and enable RAG for grounded responses”

   proof ↗⚑ flag

   LiveKit Agents~partialclaimed4/10

   LiveKit Agents supports calling external APIs/tools that a developer can use to implement RAG ("Call external APIs or lookup data for RAG"), and has full LLM tool-use and MCP integration for wiring in retrieval systems, but there is no built-in knowledge base, document ingestion, or vector-store/RAG pipeline shipped by the framework itself — developers must bring their own RAG implementation via the tools API. Missing for 10: built-in document indexing/vector store, out-of-the-box knowledge-base feature, first-party RAG pipeline or example showing document grounding end-to-end.

   • [claimed-docs] “Call external APIs or lookup data for RAG.”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use. This feature allows you to create a custom library of tools to extend your agent's context”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use.”
   • [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
   • [claimed-docs] “Wrap an MCP server in an \`MCPToolset\` and pass it to the agent's \`tools\` parameter”

   proof ↗⚑ flag

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to ElevenLabs Agents
   ElevenLabs Agents✓fullprobed9/10

   ElevenLabs publishes a working llms.txt at the root (HTTP 200) and structures its docs as machine-readable .md pages with an explicit pointer to a full docs/llms.txt index, exactly matching the ai-native 'point an agent at llms.txt' pattern; this was independently verified via live probes rather than just claimed docs. Missing for 10: no evidence of a hands-on agent actually consuming the file end-to-end to complete a task (only that the endpoint/format is live and documented).

   • [probe] “PROBE llms.txt: HTTP 200 at https://elevenlabs.io/llms.txt # ElevenLabs ElevenLabs is an AI research and product company transforming how w…”
   • [probe] “PROBE docs-md: HTTP 200 at https://elevenlabs.io/docs/eleven-agents/overview.md > This is a page from the ElevenLabs documentation. For a co…”

   proof ↗⚑ flag

   LiveKit Agents✓fullprobed8/10

   LiveKit publishes a working llms.txt (verified live at docs.livekit.io/llms.txt, HTTP 200) and agent-friendly markdown doc endpoints (e.g. /agents/.md), plus explicit docs noting the platform is 'built for coding agents like Claude Code, Cursor, and Codex,' confirming intentional support for AI-native doc consumption. missing for 10: no independent third-party confirmation that external agents successfully consume these endpoints in practice, and no dedicated documentation page explaining the llms.txt/agent-doc strategy itself.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.livekit.io/llms.txt # LiveKit docs > LiveKit is an open-source platform for building voice, video,…”
   • [probe] “PROBE docs-md: HTTP 200 at https://docs.livekit.io/agents/.md LiveKit docs › Build Agents › Get Started › Introduction --- # Introduction …”
   • [claimed-docs] “LiveKit is built for coding agents like Claude Code, Cursor, and Codex.”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to LiveKit Agents
   ElevenLabs Agents~partialprobed6/10

   ElevenLabs Agents can be managed via a full REST API and an official CLI (verified runtime install/run), enabling scripted/headless creation and configuration of agents outside the dashboard, which supports CI-style automation. However, evidence is docs/probe-based with no CI pipeline examples, no first-class 'run in CI' guidance, and Agent testing is described as pre-deploy validation without confirmation it can be triggered non-interactively/headlessly. Missing for 10: explicit CI/CD integration examples, headless testing invocation docs, and independent hands-on confirmation of automated pipelines.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”
   • [claimed-docs] “Agent testing lets you verify conversational responses, tool usage, and full multi-turn outcomes before you deploy.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”

   proof ↗⚑ flag

   LiveKit Agents✓fullprobed8/10

   LiveKit Agents is a Python/Node.js SDK plus self-hostable server that runs as a headless programmatic participant with no UI dependency; it's pip/npm-installable, has a built-in test framework (pytest/Vitest, agent simulations) suitable for CI, and the server/CLI were verified runtime to work keylessly. missing for 10: no explicit documented CI pipeline example (e.g., GitHub Actions config) or headless-mode confirmation beyond inference from server/testing docs.

   • [claimed-docs] “A programmatic participant is any code that joins a LiveKit room as a participant — this includes AI agents, media processors, or custom log…”
   • [claimed-docs] “The Agents framework isn't limited to AI agents. You can use it to deploy any code that needs to process realtime media and data streams as …”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
   • [claimed-docs] “Agent simulations run end-to-end conversations between your agent and an LLM-driven user, then evaluate the results across the full interact…”
   • [github] “Builtin test framework: Write tests and use judges to ensure your agent is performing as expected.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round drawn
   ElevenLabs Agents✓fullclaimed8/10

   Docs explicitly describe connecting conversational agents to external MCP servers so agents can access data sources and tools via MCP, directly matching the story, and this is corroborated by dedicated documentation pages (mcp.md) rather than just passing mention. Missing for 10: independent/hands-on verification of an actual third-party MCP server being plugged in and tool-calls succeeding in production (only docs and a runtime probe on the separate hosted-server feature are present).

   • [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers. This enables your agents to: Access and process informa…”
   • [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers.”
   • [claimed-docs] “Access and process information from various data sources via the MCP server”

   proof ↗⚑ flag

   LiveKit Agents✓fullclaimed8/10

   Docs and GitHub explicitly state 'first-class support for Model Context Protocol (MCP) servers' with a documented pattern to wrap an MCP server in an MCPToolset and pass it to the agent's tools parameter, described as one line of code integration. This is first-party documentation without independent hands-on verification of MCP-specific usage. Missing for 10: independent/community confirmation of MCP tool integration working end-to-end, and more detail on multi-server or auth configurations.

   • [claimed-docs] “Wrap an MCP server in an \`MCPToolset\` and pass it to the agent's \`tools\` parameter”
   • [claimed-docs] “Wrap an MCP server in an `MCPToolset` and pass it to the agent's `tools` parameter”
   • [claimed-docs] “LiveKit Agents has first-class support for Model Context Protocol (MCP) servers.”
   • [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
   • [github] “Native support for MCP. Integrate tools provided by MCP servers with one line of code.”

   proof ↗⚑ flag

4. ai-native userConnect an agent via an official MCP server

   weight 3 · round to ElevenLabs Agents
   ElevenLabs Agents✓fullprobed9/10

   ElevenLabs explicitly documents and operates a hosted MCP server (api.elevenlabs.io/v1/mcp) that lets external MCP clients like Claude connect to create, configure, and manage agents through natural language, with a runtime probe confirming the live endpoint correctly responds with the MCP OAuth challenge. This is a first-party 'official MCP server' offering, not just client-side MCP consumption, matching the story precisely. Missing for 10: independent/community hands-on confirmation beyond ElevenLabs' own docs and the single runtime probe.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
   • [claimed-docs] “connect the hosted MCP server to Claude or any other MCP client to create and manage agents through natural language”
   • [claimed-docs] “create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally”
   • [probe] “official MCP server documented at https://elevenlabs.io/docs/eleven-agents/operate/hosted-mcp”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

5. ai-native userUse an official CLI

   weight 2 · round drawn
   ElevenLabs Agents✓fullprobed9/10

   ElevenLabs documents an official Agents CLI for managing agents, and runtime probing confirms `npx @elevenlabs/cli` installs and runs keylessly from npm, corroborating the docs claim with hands-on evidence. Missing for 10: deeper documentation of full CLI command surface/subcommands beyond basic install confirmation.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “Use the ElevenLabs agents skill to build and manage voice agents from your AI coding assistant: npx skills add elevenlabs/skills --skill age…”
   • [claimed-docs] “Use the ElevenLabs agents skill to build and manage voice agents from your AI coding assistant”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”

   proof ↗⚑ flag

   LiveKit Agents✓fullprobed9/10

   LiveKit ships an official CLI (livekit-cli), documented on GitHub and verified hands-on to install and run (`lk --version` works via brew install), confirming it's a real, functional official CLI supporting the agent workflow. missing for 10: no deep documentation of full CLI command surface for agent-specific workflows within the evidence pack.

   • [probe] “official CLI documented at https://github.com/livekit/livekit-cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): `lk --version` printed `lk version 2.18.6` after a plain `brew install livekit-cli` — the official CLI …”

   proof ↗⚑ flag

6. ai-native userDrive the product through a documented public API

   weight 3 · round to ElevenLabs Agents
   ElevenLabs Agents✓fullprobed9/10

   ElevenLabs Agents ships a documented public API alongside a dashboard, CLI, and hosted MCP server for programmatic agent creation/management, with quickstart docs explicitly noting 'creating an agent via the API' and full customization (tools, voice, analysis, testing) accessible programmatically. Runtime probes confirm the CLI installs keylessly and the hosted MCP endpoint is live and functional. Missing for 10: no independent third-party API reference/SDK usage examples or community corroboration beyond first-party docs.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”
   • [claimed-docs] “Enable agents to call clients & APIs to perform actions”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”

   proof ↗⚑ flag

   LiveKit Agents✓fullprobed8/10

   LiveKit Agents ships a well-documented, pip/npm-installable Python and Node.js SDK with extensive public API surface (tools, MCP, turn detection, telephony, testing), plus a llms.txt and hands-on verified installs/runtime confirming the API is real and usable by AI-native developers. Missing for 10: independent third-party API reference/versioning audit and deeper evidence of API stability guarantees beyond docs and probes.

   • [claimed-docs] “The Agents framework lets you add any Python or Node.js program to LiveKit rooms as full realtime participants.”
   • [claimed-docs] “A programmatic participant is any code that joins a LiveKit room as a participant — this includes AI agents, media processors, or custom log…”
   • [claimed-docs] “The Agents framework isn't limited to AI agents. You can use it to deploy any code that needs to process realtime media and data streams as …”
   • [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use. This feature allows you to create a custom library of tools to extend your agent's context”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.livekit.io/llms.txt # LiveKit docs > LiveKit is an open-source platform for building voice, video,…”
   • [probe] “PROBE docs-md: HTTP 200 at https://docs.livekit.io/agents/.md LiveKit docs › Build Agents › Get Started › Introduction --- # Introduction …”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”

   proof ↗⚑ flag

7. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round drawn
   ElevenLabs Agents—none0/10

   Evidence covers agent-side authentication (protecting inbound access to a deployed agent) and OAuth for the hosted MCP server, but there is no documentation of issuing scoped or least-privilege API keys/tokens that limit what an agent itself can do via the ElevenLabs API. Missing for 10: any docs on API key permission scopes, workspace role-based tokens, or restricted-credential issuance for agents.

   • [claimed-docs] “Secure agent access | Authentication | Implement custom authentication for protected agent access”
   • [claimed-docs] “Implement custom authentication for protected agent access”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   The evidence pack covers voice/agent features, tool-calling, MCP, testing, and telephony, but contains no mention of scoped or least-privilege API key/token issuance for agents (e.g., LiveKit's grant-based access tokens or credential scoping). Since LiveKit is a platform with API keys and would plausibly support such scoping, this is an applicable but undocumented axis in the given evidence.

   ⚑ flag

8. ai-native userBuild against official SDKs

   weight 2 · round to LiveKit Agents
   ElevenLabs Agents~partialprobed4/10

   Docs confirm programmatic access via the ElevenLabs API and an official CLI (`@elevenlabs/cli`, verified runtime) for building agents, but no evidence pack item explicitly documents a first-party language SDK (Python/Node/etc.) for Agents beyond the generic API reference. Missing for 10: explicit official SDK docs/repos, SDK code examples, and independent confirmation of SDK feature parity with the API.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”

   proof ↗⚑ flag

   LiveKit Agents✓fullprobed9/10

   LiveKit provides official Python and Node.js SDKs (livekit-agents), verified hands-on to pip-install and import with no key, plus documented plugin ecosystem, tool use, MCP support, testing frameworks, and telephony integration—clearly an official SDK ecosystem for AI-native building. Missing for 10: independent third-party benchmarking of SDK API stability/versioning beyond community latency complaints unrelated to SDK build story.

   • [claimed-docs] “The Agents framework lets you add any Python or Node.js program to LiveKit rooms as full realtime participants.”
   • [claimed-docs] “LiveKit Agents includes a large ecosystem of open source plugins for a variety of AI providers.”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use. This feature allows you to create a custom library of tools to extend your agent's context”
   • [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”

   proof ↗⚑ flag

9. ai-native userSubscribe to events via webhooks

   weight 2 · round drawn
   ElevenLabs Agents—none0/10

   The evidence describes 'webhook tools' that let an agent make outbound calls to external endpoints during a conversation (docs-14, docs-29, docs-31), which is the opposite of subscribing to platform-emitted events via webhooks. There is no documented mechanism for a developer to register a webhook URL to receive event notifications (e.g., conversation-ended, call-completed) from ElevenLabs Agents.

   • [claimed-docs] “Tools give your assistant the ability to make external function calls to third party apps so you can get real-time information.”
   • [claimed-docs] “Tools allow ElevenLabs agents to perform actions beyond generating text responses.”
   • [claimed-docs] “allow an assistant to trigger authenticated actions based on the conversation, like scheduling meetings or initiating order returns”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   No evidence pack items mention webhooks or an event-subscription mechanism for LiveKit Agents; the docs cover MCP tool integration, telephony, testing, and turn detection, but nothing about webhook-based event notifications.

   ⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round to ElevenLabs Agents
   ElevenLabs Agents~partialclaimed5/10

   Agent-analysis features (sentiment analysis, success evaluation, structured data extraction, semantic conversation search) generate AI-derived insights from the user's own conversation data, which partially satisfies the story. However, these are evaluation/reporting metrics rather than proactive AI-generated suggestions or recommendations for next actions, and there is no independent/hands-on corroboration of this analytics experience. Missing for 10: evidence of proactive suggestion generation (e.g., recommended actions/optimizations), a dashboard summarization view, and independent verification beyond first-party docs.

   • [claimed-docs] “Success evaluation: Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Data collection: Extract specific data points from conversations such as contact information, issue details, or any structured information”
   • [claimed-docs] “Sentiment analysis: Understand user sentiment across completed conversations”
   • [claimed-docs] “Searching conversations: Find messages by keyword or meaning across your conversation history.”
   • [claimed-docs] “Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Extract structured information from conversations such as contact details and business data”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to LiveKit Agents
   ElevenLabs Agents~partialclaimed4/10

   ElevenLabs Agents supports tool-calling, webhook actions, and multi-step workflows that let a voice agent perform actions (scheduling, API calls, MCP access) during a conversation, and agents can be created/managed via API for programmatic triggering, which give some automation flavor. But all documented action-taking is tied to a live conversational turn (voice/chat) rather than an autonomous background process that runs without a triggering conversation or human interaction, so the 'runs autonomously in the background' framing is only weakly supported. Missing for 10: evidence of scheduled/cron-triggered autonomous runs, evidence of multi-step tasks executing without a live conversation session, and independent confirmation of background automation outside the demoed conversational/tool-call flow.

   • [claimed-docs] “Build multi-step workflows with visual workflow builder”
   • [claimed-docs] “Enable agents to call clients & APIs to perform actions”
   • [claimed-docs] “Tools give your assistant the ability to make external function calls to third party apps so you can get real-time information.”
   • [claimed-docs] “allow an assistant to trigger authenticated actions based on the conversation, like scheduling meetings or initiating order returns”
   • [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers. This enables your agents to: Access and process informa…”
   • [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”

   proof ↗⚑ flag

   LiveKit Agents✓fullprobed7/10

   LiveKit Agents workers run as persistent 'programmatic participants' deployed to LiveKit Cloud or self-hosted infrastructure, processing realtime streams and telephony calls autonomously in the background without human intervention (docs-8, docs-20, docs-30, docs-26, gh-2), and this is verified hands-on as a real pip-installable OSS framework with a working self-hosted backbone (probe-rt-2, probe-rt-3). Missing for 10: explicit scheduling/cron-trigger mechanisms, independent third-party reports of long-running unattended stability (the one community mention is about observability tooling gaps, not autonomy failure).

   • [claimed-docs] “A programmatic participant is any code that joins a LiveKit room as a participant — this includes AI agents, media processors, or custom log…”
   • [claimed-docs] “The Agents framework isn't limited to AI agents. You can use it to deploy any code that needs to process realtime media and data streams as …”
   • [claimed-docs] “Run your agent on LiveKit Cloud's global infrastructure.”
   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”

   proof ↗⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round to ElevenLabs Agents
   ElevenLabs Agents~partialprobed5/10

   ElevenLabs documents a first-party hosted MCP server letting an assistant like Claude create, configure, and manage agents in the workspace via natural language, with nothing to install locally — a form of task delegation to an AI assistant for managing the product. However this is not a native in-product chat/assistant UI; it requires an external MCP client (e.g., Claude) to act as the assistant, and delegation is scoped to agent-management tasks rather than general product tasks. Missing for 10: a built-in assistant surface inside the ElevenLabs dashboard itself, evidence of broader task delegation beyond agent CRUD, and independent hands-on confirmation of the natural-language workflow.

   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language”
   • [claimed-docs] “connect the hosted MCP server to Claude or any other MCP client to create and manage agents through natural language”
   • [claimed-docs] “create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round to ElevenLabs Agents
   ElevenLabs Agents✓fullprobed8/10

   ElevenLabs Agents ships a hosted MCP server explicitly designed so assistants like Claude can 'create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally,' and a runtime probe confirms the endpoint is live and speaks the MCP OAuth flow. This is a direct, first-party natural-language control surface, not just a client-side integration. Missing for 10: independent/community hands-on confirmation of a full natural-language agent-management session (only endpoint liveness was probed, not an actual successful NL task).

   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language”
   • [claimed-docs] “connect the hosted MCP server to Claude or any other MCP client to create and manage agents through natural language”
   • [claimed-docs] “create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”
   • [probe] “official MCP server documented at https://elevenlabs.io/docs/eleven-agents/operate/hosted-mcp”

   proof ↗⚑ flag

   LiveKit Agents~partialprobed4/10

   LiveKit ships an 'Agent Builder' that lets users prototype/deploy voice agents in-browser without code, and docs explicitly note the framework is 'built for coding agents like Claude Code, Cursor, and Codex' plus agent-readable llms.txt docs — suggesting some AI-native/no-code operability. However there is no evidence of a native natural-language command interface for operating LiveKit itself (e.g., NL-driven CLI, chatops control plane); the MCP support described is for agents calling external tools, not for controlling LiveKit via NL. Missing for 10: a documented NL-driven interface/CLI for configuring or operating the platform itself, and independent confirmation of the Agent Builder's no-code UX actually being NL-based rather than GUI-based.

   • [claimed-docs] “LiveKit is built for coding agents like Claude Code, Cursor, and Codex.”
   • [claimed-docs] “Prototype and deploy voice agents directly in your browser, without writing any code.”
   • [claimed-docs] “LiveKit Agent Builder to prototype and deploy agents directly in your browser without writing code”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.livekit.io/llms.txt # LiveKit docs > LiveKit is an open-source platform for building voice, video,…”
   • [probe] “PROBE docs-md: HTTP 200 at https://docs.livekit.io/agents/.md LiveKit docs › Build Agents › Get Started › Introduction --- # Introduction …”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   ElevenLabs Agents—none0/10

   The evidence pack contains extensive markdown documentation for ElevenLabs Agents (quickstart, customization, tools, etc.) but nothing describes an interactive API reference page with runnable/'try it' examples — no mention of a Swagger/OpenAPI explorer, live code sandbox, or embedded runnable snippets. Since ElevenLabs Agents exposes an API, this axis is applicable, but no evidence supports the specific capability.

   ⚑ flag

   LiveKit Agents—none0/10

   Evidence shows extensive quickstart guides, markdown-based docs (llms.txt), and a no-code 'Agent Builder' prototyping tool, but nothing describes an interactive API reference (e.g., embedded code sandbox, live runnable examples, or Swagger-like explorer) for exploring the SDK/API itself.

   ⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   ElevenLabs Agents—none0/10

   The evidence pack documents the API, CLI, dashboard, and hosted MCP server for ElevenLabs Agents, but nowhere mentions a downloadable OpenAPI/Swagger spec or machine-readable schema for the API. This axis clearly applies to an API-driven product, but no evidence shows it being delivered.

   ⚑ flag

   LiveKit Agents—none0/10

   The evidence pack documents LiveKit's SDKs, CLI, docs (llms.txt), and framework capabilities but contains no mention of a downloadable OpenAPI/Swagger spec or other machine-readable API description for LiveKit's server or Agents APIs. Since LiveKit exposes a real API surface (server API, cloud API), this axis applies but is unevidenced.

   • [probe] “official CLI documented at https://github.com/livekit/livekit-cli”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.livekit.io/llms.txt # LiveKit docs > LiveKit is an open-source platform for building voice, video,…”
   • [probe] “PROBE docs-md: HTTP 200 at https://docs.livekit.io/agents/.md LiveKit docs › Build Agents › Get Started › Introduction --- # Introduction …”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round to LiveKit Agents
   ElevenLabs Agents~partialclaimed4/10

   Docs describe an 'agent testing' feature to verify conversational responses, tool usage, and multi-turn outcomes 'before you deploy', implying a pre-production test workflow, and tool execution can run in a 'sandboxed environment' on ElevenLabs' infrastructure. However there is no explicit documentation of a separate sandbox/staging environment or test data isolation from production conversation/analytics data. Missing for 10: explicit sandbox/staging environment separate from production data, documentation of test-vs-prod data isolation, independent confirmation that test runs don't affect production analytics or logs.

   • [claimed-docs] “Agent testing lets you verify conversational responses, tool usage, and full multi-turn outcomes before you deploy.”
   • [claimed-docs] “Agent testing lets you verify conversational responses, tool usage, and full multi-turn outcomes before you deploy”
   • [claimed-docs] “Custom JavaScript executed in a sandboxed environment on ElevenLabs' infrastructure.”
   • [claimed-docs] “Custom JavaScript executed in a sandboxed environment on ElevenLabs' infrastructure”

   proof ↗⚑ flag

   LiveKit Agents~partialprobed6/10

   LiveKit provides a self-hostable dev server (`livekit-server --dev`) that boots locally with placeholder keys, separate from any production deployment, plus a built-in testing framework with behavioral tests and LLM-driven agent simulations that evaluate agent behavior without needing real production data. However, there is no explicitly branded 'sandbox environment' or staging/production data-isolation feature documented — the sandbox-like capability is inferred from dev-mode self-hosting and test simulations rather than a dedicated sandbox product feature. Missing for 10: an explicit sandbox/staging environment offering with documented separation from production data, and independent confirmation that test simulations never touch production data stores.

   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
   • [claimed-docs] “Agent simulations run end-to-end conversations between your agent and an LLM-driven user, then evaluate the results across the full interact…”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes... Agent simulations run end-to-end conversations between your agent and…”
   • [github] “Builtin test framework: Write tests and use judges to ensure your agent is performing as expected.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   ElevenLabs Agents—none0/10

   No evidence in the pack mentions API versioning scheme, version numbers, or any documented deprecation policy for ElevenLabs Agents' API/CLI/MCP interfaces. missing for 10: versioned API endpoints or version headers, changelog/release notes, explicit deprecation policy or sunset timelines, migration guides for breaking changes.

   ⚑ flag

   LiveKit Agents—none0/10

   No evidence in the pack mentions API versioning scheme, version compatibility guarantees, or a documented deprecation policy for LiveKit Agents' SDKs or APIs.

   ⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round drawn
   ElevenLabs Agents—none0/10

   The evidence documents API, CLI, and MCP-based management of agents, but nothing describes bulk/batch operations (e.g., batch-create, batch-update, or batch-delete across many agents or conversations at once). This axis is fair to ask given the product exposes an API and CLI, but no evidence confirms bulk-operation support.

   ⚑ flag

   LiveKit Agents—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to ElevenLabs Agents
   ElevenLabs Agents~partialclaimed6/10

   ElevenLabs Agents supports tool-calling and webhook actions that are triggered automatically based on conversation events (e.g., 'trigger authenticated actions based on the conversation, like scheduling meetings or initiating order returns'), plus multi-step visual workflows and MCP-based external actions. This covers event-triggered automation within a conversation but the evidence doesn't show general rule-based triggers on arbitrary system events (e.g., scheduled/external triggers outside a live conversation) — missing for 10: documentation of standalone event/rule engine independent of a live conversation, examples of conditional branching logic beyond tool calls, and independent verification of automation reliability.

   • [claimed-docs] “Enable agents to call clients & APIs to perform actions”
   • [claimed-docs] “Tools give your assistant the ability to make external function calls to third party apps so you can get real-time information.”
   • [claimed-docs] “allow an assistant to trigger authenticated actions based on the conversation, like scheduling meetings or initiating order returns”
   • [claimed-docs] “Build multi-step workflows with visual workflow builder”
   • [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers. This enables your agents to: Access and process informa…”

   proof ↗⚑ flag

   LiveKit Agents~partialclaimed5/10

   LiveKit Agents supports event-driven automatic behaviors — tool-calling on LLM decisions, turn-detector triggering response timing, and adaptive interruption handling triggering barge-in logic — all of which are 'events auto-trigger actions' patterns built into the agent runtime. However, the evidence shows these as built-in framework behaviors and developer-coded event handlers rather than a user-facing declarative rule engine (e.g., no evidence of a 'when X happens do Y' config surface, webhook/rule subscription API, or no-code rule builder). Missing for 10: explicit rule-definition/webhook-trigger API exposed to non-developer users, documentation of a generalized event-subscription system beyond tool calls and turn logic, and independent confirmation of custom rule automation in production use.

   • [claimed-docs] “Call external APIs or lookup data for RAG.”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use. This feature allows you to create a custom library of tools to extend your agent's context”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use.”
   • [claimed-docs] “LiveKit's `TurnDetector` is an audio model that encodes user audio directly, capturing both _what_ is said and _how_ it's said.”
   • [claimed-docs] “A turn detector model can predict that they have more to say and wait for them to finish before responding.”
   • [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response... to identify intentional interruptions (…”
   • [claimed-docs] “the model analyzes the acoustic signals to identify intentional interruptions (barge-ins) from conversational backchanneling”
   • [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response.”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round drawn
   ElevenLabs Agents—none0/10

   The evidence pack covers multi-step workflow building, tool calling, MCP integration, and analysis, but no documentation mentions scheduling recurring jobs, cron-like triggers, or automated recurring workflow execution. This is a fair capability to expect from an automation-focused agent platform, but no evidence supports it.

   ⚑ flag

   LiveKit Agents—none0/10

   The evidence pack covers LiveKit Agents' realtime voice/video agent capabilities, tool use, MCP integration, telephony, and testing, but contains no mention of any scheduling, cron-like recurring job, or workflow-automation trigger mechanism. This is a fair capability to ask about for an automation-focused agent framework, but no evidence supports it.

   ⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round to LiveKit Agents
   ElevenLabs Agents—none0/10

   The docs describe building, testing, and analyzing agents (agent-testing, agent-analysis) but there is no mention of version history, change review workflows, or rollback/revert capability for agent configurations. Missing for 10: version history UI, rollback/restore of prior agent configs, change-review/approval workflow, audit trail of edits.

   ⚑ flag

   LiveKit Agents~partialclaimed4/10

   The pricing page explicitly claims 'Instant rollback to a previous agent deployment,' and the testing/simulation framework (behavioral tests, LLM-judge simulations) supports a review step before deployment, but there's no documented versioning system, changelog, or diff/history UI for automations. Missing for 10: explicit version history/diffing of agent configs, a documented review/approval workflow beyond test suites, and independent confirmation that rollback works in practice.

   • [claimed-docs] “Instant rollback to a previous agent deployment”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
   • [claimed-docs] “Agent simulations run end-to-end conversations between your agent and an LLM-driven user, then evaluate the results across the full interact…”
   • [github] “Builtin test framework: Write tests and use judges to ensure your agent is performing as expected.”

   proof ↗⚑ flag

Compliance

1. founderMeet call-recording consent and disclosure obligations with per-call recording controls and configurable data retention

   weight 2 · round to ElevenLabs Agents
   ElevenLabs Agents~partialclaimed4/10

   Docs confirm configurable data-retention settings for conversation transcripts and audio recordings, which addresses the retention half of the story, plus HIPAA/BAA support suggesting compliance posture. However, there is no evidence of per-call recording enable/disable controls or built-in consent/disclosure messaging mechanisms for calls. Missing for 10: per-call recording toggle, consent/disclosure prompt configuration, and any independent verification of retention enforcement.

   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings.”
   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings”
   • [claimed-docs] “ElevenLabs allows such customers to develop AI-powered voice agents for the handling Protected Health Information (PHI)”
   • [claimed-docs] “ElevenLabs Agents is one of ElevenLabs' HIPAA-eligible services, and we offer Business Associate Agreements (BAAs) to eligible customers”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   No evidence in the pack mentions call recording, consent disclosure, or data retention controls anywhere in LiveKit Agents docs, GitHub, or community sources; the pack covers telephony, tool use, MCP, testing, and turn detection but nothing about recording/retention compliance features. missing for 10: recording consent/disclosure mechanisms, per-call recording toggles, configurable data retention policies, and any documentation or hands-on evidence of these compliance controls.

   ⚑ flag

2. platform-engineerRun regulated workloads with HIPAA/BAA support, SOC 2, and data-residency options

   weight 2 · round to ElevenLabs Agents
   ElevenLabs Agents~partialclaimed5/10

   ElevenLabs Agents documents HIPAA eligibility and BAA availability for eligible customers, directly supporting regulated PHI workloads. However, there is no evidence in the pack of SOC 2 certification or data-residency options, both explicitly named in the story. Missing for 10: SOC 2 attestation/report evidence, data-residency region selection or documentation, independent audit corroboration of HIPAA/BAA claims.

   • [claimed-docs] “ElevenLabs allows such customers to develop AI-powered voice agents for the handling Protected Health Information (PHI)”
   • [claimed-docs] “ElevenLabs allows such customers to develop AI-powered voice agents for the handling Protected Health Information (PHI).”
   • [claimed-docs] “ElevenLabs Agents is one of ElevenLabs' HIPAA-eligible services, and we offer Business Associate Agreements (BAAs) to eligible customers”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   The evidence pack contains no mention of HIPAA/BAA agreements, SOC 2 certification, or data-residency controls anywhere in LiveKit's docs, GitHub repo, or probes; self-hosting and TURN/security items only cover TLS/SSL and self-hosted deployment, not compliance attestations. As a platform serving enterprise/regulated workloads, this axis clearly applies, so lack of evidence yields 'none'.

   • [claimed-docs] “The good news is LiveKit includes an embedded TURN server. It's a secure TURN implementation that has integrated authentication with the res…”
   • [claimed-docs] “In order to have a secure LiveKit deployment, you will need a domain as well as a SSL certificate for that domain.”

   proof ↗⚑ flag

Scale

1. platform-engineerSee documented concurrency limits and scale to many simultaneous calls without manual capacity begging

   weight 2 · round to LiveKit Agents
   ElevenLabs Agents—none0/10

   No evidence pack item documents concurrency limits, per-account call caps, autoscaling behavior, or a process for scaling to many simultaneous calls without contacting sales/support; docs cover features like tools, voice customization, SIP trunking, HIPAA, and MCP but nothing on capacity/scale numbers or self-service scaling guarantees.

   ⚑ flag

   LiveKit Agents~partialcommunity3/10

   Docs mention running agents on LiveKit Cloud's global infrastructure and self-hosting with an embedded TURN server, implying scale-out capability, but no evidence pack item cites concrete documented concurrency limits, per-instance call caps, or autoscaling guarantees; a community report even flags lack of visibility into per-call cost/latency at scale. missing for 10: published concurrency/capacity numbers, autoscaling documentation, load-testing benchmarks, and confirmation that scaling requires no manual quota requests.

   • [claimed-docs] “Run your agent on LiveKit Cloud's global infrastructure.”
   • [claimed-docs] “The good news is LiveKit includes an embedded TURN server. It's a secure TURN implementation that has integrated authentication with the res…”
   • [community] “We've been running multiple voice AI agents on LiveKit and kept running into visibility issues — no way to measure TTFT, latency across STT …”

   proof ↗⚑ flag

Self host

1. platform-engineerSelf-host the voice agent runtime from open-source code on my own infrastructure

   weight 3 · round to LiveKit Agents
   ElevenLabs Agents—none0/10

   ElevenLabs Agents is entirely a managed/hosted service — the CLI and MCP server are clients/interfaces to ElevenLabs' cloud infrastructure, not open-source runtime code that can be deployed on a platform-engineer's own servers. Nothing in the evidence pack mentions open-sourcing the runtime, self-hosting instructions, or on-prem deployment options; all references (dashboard, API, CLI, hosted MCP, SIP trunking) point to ElevenLabs-operated infrastructure.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
   • [claimed-docs] “create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”

   proof ↗⚑ flag

   LiveKit Agents✓fullprobed9/10

   LiveKit Agents is open source, pip/npm installable and runs the agent runtime independent of any vendor account, and the underlying WebRTC/SIP server (livekit-server) is Apache-2.0 and self-hostable, confirmed via hands-on probes (booting `livekit-server --dev` locally with no keys, and installing livekit-agents/CLI keylessly) plus official self-hosting deployment docs covering TURN, TLS/domain setup. Missing for 10: no first-party production-scale self-hosting case study or independent report of large-scale self-hosted deployment beyond dev-mode probe.

   • [probe] “PROBE runtime (recorded 2026-09-05): `lk --version` printed `lk version 2.18.6` after a plain `brew install livekit-cli` — the official CLI …”
   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”
   • [claimed-docs] “The good news is LiveKit includes an embedded TURN server. It's a secure TURN implementation that has integrated authentication with the res…”
   • [claimed-docs] “In order to have a secure LiveKit deployment, you will need a domain as well as a SSL certificate for that domain.”
   • [claimed-docs] “A programmatic participant is any code that joins a LiveKit room as a participant — this includes AI agents, media processors, or custom log…”

   proof ↗⚑ flag

Latency

1. platform-engineerSee documented end-to-end voice latency numbers or tuning guidance backing the platform's speed claims

   weight 3 · round to ElevenLabs Agents
   ElevenLabs Agents~partialclaimed4/10

   ElevenLabs markets 'ultra-low latency' (docs-37) and does document conversation-flow tuning knobs like turn-taking, interruption handling and timeouts (docs-33, docs-42, docs-19/49/54), which counts as tuning guidance for latency/turn-taking behavior. However there are no documented end-to-end latency benchmark numbers (e.g., ms figures, percentile breakdowns) anywhere in the evidence pack. Missing for 10: concrete latency benchmark numbers/metrics, methodology for measuring end-to-end latency, and any independent corroboration of the 'ultra-low latency' marketing claim.

   • [claimed-docs] “Configure, deploy and monitor natural, human-sounding agents in 70+ languages with leading accuracy and ultra-low latency across voice or ch…”
   • [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”
   • [claimed-docs] “Configure turn-taking, interruptions, and timeout settings”
   • [claimed-docs] “The Max conversation duration setting limits the total time a conversation can remain active. ... The default is 600 seconds (10 minutes).”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   The evidence pack contains no documented end-to-end latency benchmarks or explicit tuning guidance for reducing turn-taking latency — only feature descriptions (turn detector, adaptive interruption handling) without numbers or configuration guidance. A community report explicitly states users could not measure TTFT or STT→LLM→TTS latency, reinforcing that this visibility/documentation is absent.

   • [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response... to identify intentional interruptions (…”
   • [claimed-docs] “LiveKit's `TurnDetector` is an audio model that encodes user audio directly, capturing both _what_ is said and _how_ it's said.”
   • [claimed-docs] “A turn detector model can predict that they have more to say and wait for them to finish before responding.”
   • [claimed-docs] “the model analyzes the acoustic signals to identify intentional interruptions (barge-ins) from conversational backchanneling”
   • [community] “We've been running multiple voice AI agents on LiveKit and kept running into visibility issues — no way to measure TTFT, latency across STT …”

   proof ↗⚑ flag

Turn taking

1. developerRely on the agent to handle interruptions (barge-in) gracefully — stopping speech, updating context, and recovering the turn

   weight 3 · round to LiveKit Agents
   ElevenLabs Agents~partialclaimed6/10

   Docs explicitly document configuring turn-taking, interruptions, and timeout settings, and conversation-flow settings that 'determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavior,' confirming barge-in handling is a first-class configurable feature. However, evidence lacks detail on how context is updated after an interruption or how turn recovery specifically behaves, and there's no hands-on/independent validation of graceful recovery in practice. Missing for 10: technical detail on context truncation/update after interruption, independent or hands-on testing confirming graceful recovery, and latency benchmarks for barge-in specifically.

   • [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”
   • [claimed-docs] “Configure turn-taking, interruptions, and timeout settings”
   • [claimed-docs] “The Max conversation duration setting limits the total time a conversation can remain active. ... The default is 600 seconds (10 minutes).”

   proof ↗⚑ flag

   LiveKit Agents✓fullprobed8/10

   LiveKit Agents has well-documented, purpose-built features for barge-in: adaptive interruption handling distinguishes intentional interruptions from backchanneling, and the TurnDetector model analyzes acoustic and semantic signals to manage turn-taking recovery. These are first-party docs with specific technical detail on the mechanism (stopping speech, updating context, recovering turn), and the runtime probes confirm the framework is real and installable. Missing for 10: independent/hands-on evidence specifically testing barge-in behavior in production (only community evidence is about latency observability, not interruption handling itself) and no third-party benchmark of interruption accuracy.

   • [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response... to identify intentional interruptions (…”
   • [claimed-docs] “the model analyzes the acoustic signals to identify intentional interruptions (barge-ins) from conversational backchanneling”
   • [claimed-docs] “LiveKit's `TurnDetector` is an audio model that encodes user audio directly, capturing both _what_ is said and _how_ it's said.”
   • [claimed-docs] “A turn detector model can predict that they have more to say and wait for them to finish before responding.”
   • [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response.”
   • [claimed-docs] “LiveKit's TurnDetector is an audio model that encodes user audio directly, capturing both what is said and how it's said.”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”

   proof ↗⚑ flag

2. developerEnable noise suppression or audio filtering so the agent stays coherent on noisy real-world calls

   weight 1 · round drawn
   ElevenLabs Agents—none0/10

   The evidence pack covers voice customization, turn-taking/interruption settings, tools, testing, and telephony (SIP) integration, but nowhere mentions noise suppression, background noise handling, or audio filtering for noisy real-world calls. This is a fair capability to expect for a voice-agent platform handling phone calls, but no documentation or feature reference confirms it exists.

   • [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”
   • [claimed-docs] “Configure turn-taking, interruptions, and timeout settings”
   • [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   The evidence pack covers turn detection, interruption handling, telephony, MCP tools, and testing, but contains no mention of noise suppression or audio filtering plugins/features anywhere. Missing for 10: any docs or plugin reference to noise cancellation (e.g. Krisp/BVC), background noise filtering, or audio pre-processing configuration.

   ⚑ flag

3. developerUse model-based end-of-turn detection beyond simple VAD silence timeouts so the agent doesn't talk over slow speakers

   weight 2 · round to LiveKit Agents
   ElevenLabs Agents—none0/10

   Docs mention configurable turn-taking, interruption handling, and timeout settings (docs-33, docs-42), but nothing in the evidence describes a model-based end-of-turn detector that goes beyond a VAD/silence-timeout mechanism — no mention of semantic or ML-based turn-completion detection to avoid interrupting slow speakers. Missing for 10: any documentation of a dedicated end-of-turn prediction model, benchmarks or comparisons versus simple VAD timeouts, or hands-on evidence that the agent waits appropriately for slow speakers.

   • [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”
   • [claimed-docs] “Configure turn-taking, interruptions, and timeout settings”
   • [claimed-docs] “The Max conversation duration setting limits the total time a conversation can remain active. ... The default is 600 seconds (10 minutes).”

   proof ↗⚑ flag

   LiveKit Agents✓fullclaimed9/10

   LiveKit's TurnDetector is a dedicated audio+semantic model (not just VAD silence) that predicts whether a user has finished speaking or has more to say, waiting accordingly, and adaptive interruption handling further distinguishes real barge-ins from backchanneling — directly addressing not talking over slow speakers. This is documented in detail across multiple first-party doc pages plus GitHub feature lists. missing for 10: independent/hands-on benchmark or community validation of the turn-detector's real-world latency/accuracy impact.

   • [claimed-docs] “LiveKit's `TurnDetector` is an audio model that encodes user audio directly, capturing both _what_ is said and _how_ it's said.”
   • [claimed-docs] “A turn detector model can predict that they have more to say and wait for them to finish before responding.”
   • [claimed-docs] “LiveKit's TurnDetector is an audio model that encodes user audio directly, capturing both what is said and how it's said.”
   • [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response... to identify intentional interruptions (…”
   • [claimed-docs] “the model analyzes the acoustic signals to identify intentional interruptions (barge-ins) from conversational backchanneling”
   • [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response.”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to ElevenLabs Agents
   ElevenLabs Agents~partialprobed7/10

   Docs show agents can be created, configured, and managed via the API, CLI, dashboard, or hosted MCP server, and quickstart explicitly says agents can be created 'via the API or the web dashboard,' implying broad parity across surfaces (elevenlabs-agents-docs-1, docs-39, docs-44). However, there's no explicit feature-by-feature confirmation that every dashboard capability (e.g., visual workflow builder, analytics dashboards, testing UI) has a full API equivalent, nor independent verification of parity. Missing for 10: an explicit parity statement or changelog confirming all dashboard features (workflow builder, analytics, testing suite) are fully scriptable via API, and independent/hands-on confirmation of that parity.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”
   • [claimed-docs] “create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally”
   • [claimed-docs] “Build multi-step workflows with visual workflow builder”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”

   proof ↗⚑ flag

   LiveKit Agents~partialprobed6/10

   LiveKit Agents is API/SDK-first (Python/Node.js) with a full programmatic surface for building agents, tools, telephony, and testing, and there is also a no-code 'Agent Builder' UI for prototyping. However, the evidence doesn't confirm that everything achievable in that browser-based Agent Builder UI (or LiveKit Cloud dashboard features like deployment management, rollback) is equally exposed via API/CLI, so parity between UI and API is not fully demonstrated. missing for 10: explicit evidence that Agent Builder's no-code UI actions (and Cloud dashboard deployment/rollback controls) are all reachable via API/CLI, and confirmation of full CLI/API parity with dashboard features.

   • [claimed-docs] “Prototype and deploy voice agents directly in your browser, without writing any code.”
   • [claimed-docs] “LiveKit Agent Builder to prototype and deploy agents directly in your browser without writing code”
   • [claimed-docs] “LiveKit Agent Builder: Prototype and deploy voice agents directly in your browser, without writing any code.”
   • [claimed-docs] “Instant rollback to a previous agent deployment”
   • [probe] “official CLI documented at https://github.com/livekit/livekit-cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): `lk --version` printed `lk version 2.18.6` after a plain `brew install livekit-cli` — the official CLI …”
   • [claimed-docs] “A programmatic participant is any code that joins a LiveKit room as a participant — this includes AI agents, media processors, or custom log…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round drawn
   ElevenLabs Agents—none0/10

   No evidence describes a data export feature, open-format data dumps, or account deletion/portability tooling; only conversation retention settings are documented, not export mechanisms. Missing for 10: any documented export API/CLI command, supported open export formats (e.g., JSON/CSV), and confirmation of full data portability/account closure process.

   ⚑ flag

   LiveKit Agents—none0/10

   The evidence pack documents LiveKit Agents' open-source, self-hostable nature (Apache-2.0 server, pip/npm installable framework) but contains no explicit mention of data export tooling, open data formats for conversation/session logs, or a documented exit/migration path for user data. Self-hosting mitigates lock-in in principle, but that is not the same as a documented 'export all data' capability.

   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”
   • [claimed-docs] “The good news is LiveKit includes an embedded TURN server. It's a secure TURN implementation that has integrated authentication with the res…”

   proof ↗⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round to LiveKit Agents
   ElevenLabs Agents—none0/10

   ElevenLabs Agents is a closed, proprietary SaaS product; the CLI is open on npm but no evidence indicates the core Agents platform/backend source is available under an open license. No license file, GitHub repo, or open-source statement is present in the evidence pack.

   ⚑ flag

   LiveKit Agents✓fullprobed8/10

   LiveKit Agents is hosted at github.com/livekit/agents and documented as an open-source framework, with hands-on verification that the packages install and run without any account or key (pip/npm) and that the underlying server is Apache-2.0 OSS. The llms.txt docs also explicitly describe LiveKit as 'an open-source platform.' Missing for 10: an explicit license file/badge citation specifically for the livekit/agents repo (only inferred via probe evidence and sibling repo license).

   • [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.livekit.io/llms.txt # LiveKit docs > LiveKit is an open-source platform for building voice, video,…”
   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”
   • [probe] “official CLI documented at https://github.com/livekit/livekit-cli”

   proof ↗⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round to LiveKit Agents
   ElevenLabs Agents—none0/10

   ElevenLabs Agents is entirely a hosted SaaS product—managed via dashboard, API, CLI, and hosted MCP server, all pointing to ElevenLabs' own infrastructure; there is no evidence of an open-source release, downloadable server binary, or self-hosting option for the core voice-agent runtime. missing for 10: any open-source repo or self-hosted deployment package, docs describing running the core service on one's own infrastructure, independent confirmation of self-hosting.

   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
   • [claimed-docs] “create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally”

   proof ↗⚑ flag

   LiveKit Agents✓fullprobed9/10

   LiveKit provides a documented, Apache-2.0 open-source self-hosting deployment guide including embedded TURN server and SSL setup, and this was hands-on verified: `livekit-server --dev` boots with no account or key, and the open-source livekit-agents framework installs and imports keylessly via pip/npm. This directly demonstrates the core product (server + agents framework) can be self-hosted. Missing for 10: independent third-party production self-hosting case study beyond the probe verification.

   • [claimed-docs] “The good news is LiveKit includes an embedded TURN server. It's a secure TURN implementation that has integrated authentication with the res…”
   • [claimed-docs] “In order to have a secure LiveKit deployment, you will need a domain as well as a SSL certificate for that domain.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”
   • [probe] “PROBE runtime (recorded 2026-09-05): `lk --version` printed `lk version 2.18.6` after a plain `brew install livekit-cli` — the official CLI …”

   proof ↗⚑ flag

Pricing

1. founderSee published per-minute or usage pricing and estimate cost per call before committing

   weight 2 · round drawn
   ElevenLabs Agents—none0/10

   No evidence item in the pack references pricing, per-minute rates, usage-based billing, or cost calculators for ElevenLabs Agents; all citations are product/docs feature descriptions. Since founders evaluating a voice-agent SaaS could reasonably expect published pricing, the axis applies but is entirely unaddressed in this evidence pack.

   ⚑ flag

   LiveKit Agents—none0/10

   The pack references a pricing URL (livekit.io/pricing) only for an unrelated 'rollback' feature and never quotes actual per-minute or usage rates; a community comment even highlights that operators struggled with 'cost per call' visibility, but no evidence shows published pricing figures a founder could use to estimate costs before committing.

   • [claimed-docs] “Instant rollback to a previous agent deployment”
   • [community] “We've been running multiple voice AI agents on LiveKit and kept running into visibility issues — no way to measure TTFT, latency across STT …”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round to LiveKit Agents
   ElevenLabs Agents—none0/10

   Evidence covers data retention duration settings, HIPAA/BAA eligibility, and infrastructure details, but nothing addresses selectable data region/residency options for storage location. missing for 10: any mention of region selection, data residency controls, or geographic storage options.

   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings.”
   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings”
   • [claimed-docs] “ElevenLabs Agents is one of ElevenLabs' HIPAA-eligible services, and we offer Business Associate Agreements (BAAs) to eligible customers”

   proof ↗⚑ flag

   LiveKit Agents~partialprobed5/10

   LiveKit Agents/server is open-source and self-hostable (embedded TURN server, deployment docs, hands-on verified self-host boot), which lets an operator choose their own hosting region/data location, but there is no explicit documented region-selection or data-residency feature for LiveKit Cloud itself. missing for 10: explicit LiveKit Cloud region-selection/data-residency controls, compliance certifications (e.g. GDPR/SOC2 region guarantees), and any customer-facing UI for choosing storage region.

   • [claimed-docs] “The good news is LiveKit includes an embedded TURN server. It's a secure TURN implementation that has integrated authentication with the res…”
   • [claimed-docs] “In order to have a secure LiveKit deployment, you will need a domain as well as a SSL certificate for that domain.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”

   proof ↗⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   ElevenLabs Agents—none0/10

   The evidence pack covers data retention settings (how long transcripts/audio are stored) and HIPAA/BAA compliance, but nowhere mentions an explicit opt-out or policy regarding use of customer data/conversations for AI model training. missing for 10: explicit training-data opt-out policy or setting, terms-of-service language on model training use, any statement distinguishing enterprise vs free-tier data usage for training.

   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings.”
   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings”
   • [claimed-docs] “ElevenLabs Agents is one of ElevenLabs' HIPAA-eligible services, and we offer Business Associate Agreements (BAAs) to eligible customers”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   No evidence in the pack addresses data usage for AI model training, opt-out policies, or any privacy commitments regarding training data; the evidence pack is entirely about agent framework features (tools, telephony, testing, MCP support). This is a plausible axis for a platform handling user voice/media data, but absent any documentation on training-data usage or opt-out, it cannot be credited as delivered.

   ⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round to ElevenLabs Agents
   ElevenLabs Agents~partialclaimed6/10

   Docs explicitly document configurable retention settings for conversation transcripts and audio recordings, giving users control over how long data is stored. However, evidence does not detail a deletion mechanism (e.g., data subject deletion requests, API/CLI commands to purge stored data) beyond retention duration settings. Missing for 10: explicit deletion/export controls, granular per-conversation deletion API, and independent verification of retention enforcement.

   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings.”
   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   The evidence pack contains no mention of data retention policies, recording/session deletion controls, or configurable retention windows for LiveKit Agents. While self-hosting is documented (implying infrastructure control), no explicit retention/deletion feature or API is evidenced.

   ⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   ElevenLabs Agents—none0/10

   The evidence pack covers conversation transcript/audio retention settings (docs-26, docs-52) but nothing about opting out of product telemetry or usage tracking (e.g., CLI analytics, dashboard usage metrics) for the AI-native user. Retention configuration is about customer conversation data, not the vendor's own telemetry collection, so this axis has no supporting evidence.

   ⚑ flag

   LiveKit Agents—none0/10

   No evidence pack item discusses telemetry, usage tracking, analytics collection, or an opt-out mechanism for LiveKit Agents; the framework being open-source and self-hostable is not sufficient evidence of an explicit telemetry opt-out control.

   ⚑ flag

Call control

1. developerEscalate a live call to a human with warm or blind transfer, passing context along

   weight 2 · round drawn
   ElevenLabs Agents—none0/10

   The evidence pack covers telephony features like SIP trunking, tools, webhooks, and MCP integrations, but nowhere documents call transfer (warm or blind) to a human agent or passing conversation context during handoff. Missing for 10: any mention of transfer_to_number/transfer_to_agent tool types, warm/blind transfer configuration, or context-passing during escalation.

   ⚑ flag

   LiveKit Agents—none0/10

   LiveKit Agents documents inbound/outbound telephony (SIP trunks, phone calls) but the evidence pack contains no mention of call transfer (warm or blind) or passing conversational context to a human agent during a live call — this specific escalation capability is never described.

   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [claimed-docs] “Enable your voice agent to make or take phone calls.”
   • [claimed-docs] “Outbound trunks are used to place outgoing calls.”
   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”

   proof ↗⚑ flag

2. developerMy agent can send DTMF keypresses, navigate IVR menus, and detect or leave voicemail

   weight 1 · round drawn
   ElevenLabs Agents—none0/10

   The evidence pack confirms ElevenLabs Agents supports telephony integration via SIP trunking and phone numbers, so the telephony axis clearly applies, but nothing in the docs mentions DTMF tone sending, IVR menu navigation, or voicemail detection/leaving capabilities.

   • [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”
   • [claimed-docs] “Route calls to AI agents without changing your existing phone infrastructure”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   Evidence confirms LiveKit's telephony/SIP stack supports inbound/outbound calls (docs-26, docs-31, docs-32, gh-2), but nothing in the pack mentions DTMF keypress sending, IVR menu navigation, or voicemail detection/leaving capabilities specifically. Missing for 10: any documentation or example of DTMF tone generation, automated IVR menu traversal, or voicemail detection/answering-machine-detection logic.

   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [claimed-docs] “Enable your voice agent to make or take phone calls.”
   • [claimed-docs] “Outbound trunks are used to place outgoing calls.”
   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”

   proof ↗⚑ flag

Campaigns

1. founderRun batch outbound call campaigns with scheduling and throughput controls

   weight 2 · round drawn
   ElevenLabs Agents—none0/10

   The evidence pack documents phone-number/SIP trunking integration for routing calls to agents, but contains no mention of batch outbound calling, campaign scheduling, or throughput/rate controls for outbound call volume. Since telephony campaign management is a fair capability to expect from a voice-agent telephony product, absence of evidence here is 'none' rather than 'na'.

   • [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”
   • [claimed-docs] “Route calls to AI agents without changing your existing phone infrastructure”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   Evidence shows LiveKit Agents can place individual outbound calls via SIP outbound trunks (docs-26, docs-32, docs-31), but there is no evidence of any batch campaign management, call scheduling, or throughput/rate-limiting controls for running outbound calling campaigns at scale — these features are simply not documented anywhere in the pack.

   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [claimed-docs] “Outbound trunks are used to place outgoing calls.”
   • [claimed-docs] “Enable your voice agent to make or take phone calls.”
   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”

   proof ↗⚑ flag

Numbers

1. developerProvision phone numbers and run both inbound and outbound calls through the platform's API

   weight 3 · round drawn
   ElevenLabs Agents~partialclaimed6/10

   Docs confirm phone/telephony integration via SIP trunking, letting agents route inbound/outbound calls over existing phone infrastructure, and agents can be created/configured via the API. However, evidence does not explicitly show a documented API endpoint for provisioning new phone numbers or explicitly triggering outbound calls programmatically vs. via SIP trunk setup. Missing for 10: explicit phone-number provisioning API endpoint docs, explicit outbound-call-initiation API example, and independent/hands-on confirmation of inbound+outbound call flows.

   • [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”
   • [claimed-docs] “Route calls to AI agents without changing your existing phone infrastructure”
   • [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”

   proof ↗⚑ flag

   LiveKit Agents~partialclaimed6/10

   LiveKit's SIP/telephony stack clearly supports inbound and outbound calling ('LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls', 'Outbound trunks are used to place outgoing calls', 'Enable your voice agent to make or take phone calls') and integrates with the agents runtime via SIP trunks. However, the evidence never shows LiveKit itself provisioning phone numbers through its own API — telephony typically requires configuring an external SIP trunk provider, and no docs here describe a native number-provisioning endpoint. Missing for 10: explicit documentation/evidence of a LiveKit API call that provisions/purchases phone numbers directly (rather than just configuring trunks against externally-acquired numbers), and independent/hands-on confirmation of the inbound+outbound call flow.

   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [claimed-docs] “Enable your voice agent to make or take phone calls.”
   • [claimed-docs] “Outbound trunks are used to place outgoing calls.”
   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”

   proof ↗⚑ flag

Sip

1. platform-engineerConnect my own carrier or PBX via SIP trunking (or import Twilio/Telnyx numbers) instead of being locked to bundled telephony

   weight 2 · round to LiveKit Agents
   ElevenLabs Agents~partialclaimed7/10

   Docs explicitly describe a SIP trunking integration that lets customers connect their existing phone systems/carrier infrastructure to ElevenLabs agents without changing carriers, directly addressing the core ask of bringing your own PBX/carrier. However, the pack has no explicit mention of importing existing Twilio/Telnyx numbers or detailed carrier onboarding steps, and no independent/hands-on confirmation of a live SIP trunk setup working end-to-end. Missing for 10: explicit Twilio/Telnyx number import instructions, and third-party/hands-on validation of the SIP trunking flow.

   • [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”
   • [claimed-docs] “Route calls to AI agents without changing your existing phone infrastructure”

   proof ↗⚑ flag

   LiveKit Agents✓fullclaimed7/10

   LiveKit's SIP-based telephony stack lets agents make/receive calls via inbound and outbound SIP trunks, which is the standard mechanism for bringing your own carrier or PBX (docs-26, docs-32, gh-2, docs-31). This is architecture-agnostic SIP, not a bundled/proprietary telephony lock-in, satisfying the platform-engineer's need to connect external trunks. Missing for 10: explicit documented walkthroughs or examples of importing Twilio/Telnyx numbers specifically, and no independent/hands-on confirmation of a real carrier trunk connection succeeding.

   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [claimed-docs] “Outbound trunks are used to place outgoing calls.”
   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”
   • [claimed-docs] “Enable your voice agent to make or take phone calls.”

   proof ↗⚑ flag

Analytics

1. ai-native userThe platform's AI reviews my calls for me — scoring quality, flagging failures, and analyzing resolution automatically

   weight 2 · round to ElevenLabs Agents
   ElevenLabs Agents✓fullclaimed7/10

   Docs explicitly describe automated post-call analysis: 'Success evaluation' scores conversation quality/goal achievement, 'Sentiment analysis' flags user sentiment, and 'Data collection' extracts structured resolution details — directly matching the story's scoring/flagging/resolution-analysis ask. Missing for 10: no independent/hands-on validation of analysis accuracy, and no explicit 'failure flagging' terminology beyond success-criteria evaluation.

   • [claimed-docs] “Success evaluation: Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Data collection: Extract specific data points from conversations such as contact information, issue details, or any structured information”
   • [claimed-docs] “Sentiment analysis: Understand user sentiment across completed conversations”
   • [claimed-docs] “Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Extract structured information from conversations such as contact details and business data.”

   proof ↗⚑ flag

   LiveKit Agents!disputedcontradicted3/10

   LiveKit ships a dev-time testing framework with 'judges' and agent simulations to evaluate scripted interactions (docs-9/10/25, gh-4), but this is pre-deployment test tooling, not automatic scoring/flagging/resolution-analysis of live production calls. A hands-on community report explicitly says operators running real agents on LiveKit have no built-in way to measure latency or cost per call, let alone automated quality/resolution review, contradicting the idea that the platform reviews calls for you. Missing for 10: production call-level QA scoring, automatic failure flagging on real traffic, resolution/outcome analysis dashboards.

   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
   • [claimed-docs] “Agent simulations run end-to-end conversations between your agent and an LLM-driven user, then evaluate the results across the full interact…”
   • [github] “Builtin test framework: Write tests and use judges to ensure your agent is performing as expected.”
   • [community] “We've been running multiple voice AI agents on LiveKit and kept running into visibility issues — no way to measure TTFT, latency across STT …”

   proof ↗⚑ flag

2. founderSee call analytics — success rates, durations, outcomes, sentiment — in dashboards without building my own

   weight 2 · round to ElevenLabs Agents
   ElevenLabs Agents~partialclaimed6/10

   Docs confirm built-in agent-analysis features—success evaluation, sentiment analysis, data extraction, and conversation search—managed via the ElevenAgents dashboard, which covers most of the story's asked-for metrics without custom-building analytics. However, there's no explicit evidence of a dashboard showing aggregated success-rate percentages or call-duration analytics/rollups across calls, only per-conversation criteria and settings. Missing for 10: dashboard screenshots/evidence of aggregate success-rate metrics, call-duration reporting/rollups, and independent corroboration that these appear in a unified analytics view.

   • [claimed-docs] “Success evaluation: Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Data collection: Extract specific data points from conversations such as contact information, issue details, or any structured information”
   • [claimed-docs] “Sentiment analysis: Understand user sentiment across completed conversations”
   • [claimed-docs] “Searching conversations: Find messages by keyword or meaning across your conversation history.”
   • [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
   • [claimed-docs] “Extract structured information from conversations such as contact details and business data.”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   The evidence pack shows testing/simulation tools (pytest/Vitest, LLM-driven simulations) but no built-in production analytics dashboard for success rates, durations, outcomes, or sentiment. A community report explicitly confirms this gap — users running LiveKit voice agents said they had 'no way to measure TTFT, latency across STT→LLM→TTS, or even cost per call' and found debugging painful, indicating no such dashboard exists out of the box.

   • [community] “We've been running multiple voice AI agents on LiveKit and kept running into visibility issues — no way to measure TTFT, latency across STT …”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
   • [claimed-docs] “Agent simulations run end-to-end conversations between your agent and an LLM-driven user, then evaluate the results across the full interact…”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes... Agent simulations run end-to-end conversations between your agent and…”

   proof ↗⚑ flag

Monitoring

1. platform-engineerMonitor live calls in production and get alerts when agents misbehave or error rates spike

   weight 1 · round to ElevenLabs Agents
   ElevenLabs Agents~partialclaimed4/10

   Docs show post-call analytics (sentiment analysis, success evaluation, data extraction, conversation search) and a marketing claim to 'monitor' agents, but there is no evidence of live/real-time call monitoring dashboards, error-rate metrics, or alerting/notification mechanisms for misbehaving agents. Missing for 10: real-time monitoring dashboard, error-rate/latency metrics, alerting/notification system, on-call integration (e.g., PagerDuty/Slack alerts).

   • [claimed-docs] “Success evaluation: Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Sentiment analysis: Understand user sentiment across completed conversations”
   • [claimed-docs] “Searching conversations: Find messages by keyword or meaning across your conversation history.”
   • [claimed-docs] “Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Configure, deploy and monitor natural, human-sounding agents in 70+ languages with leading accuracy and ultra-low latency across voice or ch…”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   No evidence pack items describe production monitoring dashboards, error-rate alerting, or call-quality observability tooling for LiveKit Agents; the only related evidence is a community report explicitly describing the lack of visibility into TTFT, per-call latency, and cost — a gap, not a delivered capability.

   • [community] “We've been running multiple voice AI agents on LiveKit and kept running into visibility issues — no way to measure TTFT, latency across STT …”

   proof ↗⚑ flag

Testing

1. developerTest agents with simulated conversations or evals before putting them on real phone calls

   weight 2 · round drawn
   ElevenLabs Agents✓fullclaimed8/10

   ElevenLabs documents an explicit Agent Testing feature that verifies conversational responses, tool usage, and full multi-turn outcomes before deployment, including converting real conversations into simulated test cases, plus success-evaluation criteria for assessing conversation quality/goal achievement — directly matching the pre-deployment simulated testing story. Missing for 10: no independent/hands-on report validating test accuracy or CI-integration details, and no explicit mention of automated eval scoring pipelines beyond docs description.

   • [claimed-docs] “Agent testing lets you verify conversational responses, tool usage, and full multi-turn outcomes before you deploy.”
   • [claimed-docs] “Transform real conversations into test cases when you find an interaction where the agent underperformed.”
   • [claimed-docs] “Agent testing lets you verify conversational responses, tool usage, and full multi-turn outcomes before you deploy”
   • [claimed-docs] “Success evaluation: Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”

   proof ↗⚑ flag

   LiveKit Agents✓fullclaimed8/10

   Docs explicitly describe both behavioral tests (pytest/Vitest) and agent simulations that run end-to-end conversations with an LLM-driven user, evaluating results before deployment, and GitHub notes a 'builtin test framework' with judges — directly matching the testing-before-real-calls story, complemented by separate telephony/SIP support for real calls. Missing for 10: independent/hands-on validation of the eval/simulation framework and explicit documentation tying test simulations to pre-phone-call validation workflows.

   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
   • [claimed-docs] “Agent simulations run end-to-end conversations between your agent and an LLM-driven user, then evaluate the results across the full interact…”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes... Agent simulations run end-to-end conversations between your agent and…”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes... using pytest (Python) or Vitest (Node.js)”
   • [github] “Builtin test framework: Write tests and use judges to ensure your agent is performing as expected.”
   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”
   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”

   proof ↗⚑ flag

Post call

1. developerExtract structured data from every call — outcomes, entities, dispositions — delivered via API or webhook after the call

   weight 2 · round to ElevenLabs Agents
   ElevenLabs Agents~partialclaimed6/10

   Docs show structured post-call outputs (data collection with contact/business data extraction, success-evaluation criteria, sentiment analysis) available via 'agent-analysis' features, and the platform is manageable via API. However, the pack never explicitly documents a webhook payload/callback delivering this structured analysis after a call, only webhook *tools* for in-call actions — missing for 10: explicit post-call webhook delivery mechanism/schema for outcomes and dispositions, and confirmation these analysis fields are retrievable via a documented API endpoint.

   • [claimed-docs] “Success evaluation: Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
   • [claimed-docs] “Data collection: Extract specific data points from conversations such as contact information, issue details, or any structured information”
   • [claimed-docs] “Sentiment analysis: Understand user sentiment across completed conversations”
   • [claimed-docs] “Extract structured information from conversations such as contact details and business data”
   • [claimed-docs] “Extract structured information from conversations such as contact details and business data.”
   • [claimed-docs] “Tools give your assistant the ability to make external function calls to third party apps so you can get real-time information.”
   • [claimed-docs] “allow an assistant to trigger authenticated actions based on the conversation, like scheduling meetings or initiating order returns”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   LiveKit Agents provides tool-calling, MCP integration, and telephony hooks, but nothing in the evidence describes a built-in mechanism for automatically extracting structured call outcomes/entities/dispositions and delivering them via API or webhook after a call ends — that would have to be custom-built by the developer using the tool-calling primitives. Since no such capability is documented, this applicable axis is unmet.

   • [claimed-docs] “Call external APIs or lookup data for RAG.”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use. This feature allows you to create a custom library of tools to extend your agent's context”
   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [claimed-docs] “Outbound trunks are used to place outgoing calls.”

   proof ↗⚑ flag

Tools

1. ai-native userMy voice agent can plug in MCP servers as tool sources so one integration grants it whole toolsets mid-call

   weight 2 · round to LiveKit Agents
   ElevenLabs Agents✓fullclaimed8/10

   Docs explicitly describe connecting conversational agents to external MCP servers as a tool source, enabling access to data/processing capabilities via that server mid-call, directly matching the story. Missing for 10: independent hands-on confirmation of mid-call multi-tool retrieval from a connected MCP server (only vendor docs, no third-party validation) and detail on how many tools/toolsets a single MCP connection exposes.

   • [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers. This enables your agents to: Access and process informa…”
   • [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers.”
   • [claimed-docs] “Access and process information from various data sources via the MCP server”

   proof ↗⚑ flag

   LiveKit Agents✓fullclaimed9/10

   LiveKit Agents has first-class, documented MCP support ("first-class support for Model Context Protocol (MCP) servers", "Integrate tools provided by MCP servers with one line of code") via `MCPToolset`, which wraps an MCP server and passes it directly to the agent's tools parameter — enabling mid-call toolset access for voice agents. This is corroborated across both docs and GitHub README. Missing for 10: independent hands-on validation of actual mid-call MCP toolset usage beyond vendor docs.

   • [claimed-docs] “Wrap an MCP server in an \`MCPToolset\` and pass it to the agent's \`tools\` parameter”
   • [claimed-docs] “Wrap an MCP server in an `MCPToolset` and pass it to the agent's `tools` parameter”
   • [claimed-docs] “LiveKit Agents has first-class support for Model Context Protocol (MCP) servers.”
   • [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
   • [github] “Native support for MCP. Integrate tools provided by MCP servers with one line of code.”

   proof ↗⚑ flag

2. developerMy agent can call external APIs and custom functions mid-conversation and speak the result without awkward dead air

   weight 3 · round to LiveKit Agents
   ElevenLabs Agents✓fullclaimed7/10

   Docs clearly describe multiple tool-calling mechanisms — webhook tools for external API calls, client-side tools, sandboxed custom JS, and MCP server integration — explicitly for making external function calls mid-conversation to fetch real-time info and trigger authenticated actions. Conversation-flow settings (turn-taking, interruption handling, ultra-low latency) address smooth conversational pacing, though not explicitly framed as avoiding dead air during tool execution. Missing for 10: explicit documentation of filler/hold-speech behavior while awaiting API results, and independent/hands-on verification that tool calls don't introduce audible pauses.

   • [claimed-docs] “Enable agents to call clients & APIs to perform actions”
   • [claimed-docs] “Tools executed directly on the client-side application (e.g., web browser, mobile app).”
   • [claimed-docs] “Tools give your assistant the ability to make external function calls to third party apps so you can get real-time information.”
   • [claimed-docs] “Custom JavaScript executed in a sandboxed environment on ElevenLabs' infrastructure.”
   • [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers. This enables your agents to: Access and process informa…”
   • [claimed-docs] “Tools allow ElevenLabs agents to perform actions beyond generating text responses.”
   • [claimed-docs] “allow an assistant to trigger authenticated actions based on the conversation, like scheduling meetings or initiating order returns”
   • [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”
   • [claimed-docs] “Configure, deploy and monitor natural, human-sounding agents in 70+ languages with leading accuracy and ultra-low latency across voice or ch…”

   proof ↗⚑ flag

   LiveKit Agents✓fullprobed9/10

   LiveKit Agents has full documented support for LLM tool/function calling (custom Python/Node functions calling external APIs mid-conversation) plus MCP server tool integration, combined with adaptive interruption handling and turn-detection to avoid awkward dead air while tools execute. Runtime probes confirm the framework is real and installable, corroborating the docs claims. missing for 10: independent/hands-on evidence specifically showing tool-call latency handled gracefully in a live conversation (only vendor docs cover this exact combination).

   • [claimed-docs] “Call external APIs or lookup data for RAG.”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use. This feature allows you to create a custom library of tools to extend your agent's context”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use.”
   • [claimed-docs] “LiveKit Agents has first-class support for Model Context Protocol (MCP) servers.”
   • [claimed-docs] “Wrap an MCP server in an \`MCPToolset\` and pass it to the agent's \`tools\` parameter”
   • [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response... to identify intentional interruptions (…”
   • [claimed-docs] “the model analyzes the acoustic signals to identify intentional interruptions (barge-ins) from conversational backchanneling”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”

   proof ↗⚑ flag

Recording

1. platform-engineerRetrieve full call recordings and transcripts programmatically for every call

   weight 2 · round to ElevenLabs Agents
   ElevenLabs Agents~partialclaimed5/10

   Docs confirm that conversation transcripts and audio recordings are stored with configurable retention, and that conversations can be searched, analyzed, and have structured data extracted (docs-26/52, docs-21/23/56), implying underlying storage of call recordings/transcripts. However, there is no explicit documentation of a specific API endpoint or method for programmatically retrieving full call recordings and transcripts on a per-call basis. Missing for 10: an explicit 'get conversation' or 'get recording/transcript' API reference, example request/response showing programmatic export of audio + transcript per call.

   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings.”
   • [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings”
   • [claimed-docs] “Data collection: Extract specific data points from conversations such as contact information, issue details, or any structured information”
   • [claimed-docs] “Searching conversations: Find messages by keyword or meaning across your conversation history.”
   • [claimed-docs] “Extract structured information from conversations such as contact details and business data.”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   The evidence pack covers agent building, tool use, MCP, telephony, and testing but contains no mention of recording APIs, transcript storage, or programmatic retrieval of call recordings/transcripts for platform engineers.

   ⚑ flag

Transcription

1. developerGet accurate real-time transcription with control over the STT provider, language models, or key terms

   weight 2 · round to LiveKit Agents
   ElevenLabs Agents~partialclaimed4/10

   Docs confirm agents let developers choose or bring custom LLMs (docs-41) and mention real-time voice interaction (docs-37), but there is no evidence of selectable STT providers, transcription accuracy tuning, or 'key terms'/vocabulary boosting controls anywhere in the pack. Missing for 10: STT provider selection, transcription-specific accuracy settings, key-term/glossary support, and any independent verification of transcription accuracy claims.

   • [claimed-docs] “Choose from supported LLMs or bring your own custom model”
   • [claimed-docs] “Configure, deploy and monitor natural, human-sounding agents in 70+ languages with leading accuracy and ultra-low latency across voice or ch…”
   • [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”

   proof ↗⚑ flag

   LiveKit Agents~partialclaimed6/10

   LiveKit Agents supports swapping STT/LLM providers via its open-source plugin ecosystem and lets you override API key/base URL for any provider (docs-12, docs-24, docs-35), giving real control over transcription and language-model choice. However there is no evidence of support for custom vocabulary/key-term or hotword boosting in STT, and no independent benchmark of transcription accuracy. Missing for 10: key-term/hotword/phrase-hint configuration support, accuracy benchmarks or independent corroboration of transcription quality.

   • [claimed-docs] “LiveKit Agents includes a large ecosystem of open source plugins for a variety of AI providers.”
   • [claimed-docs] “You can use LiveKit Inference to access many of these models directly through LiveKit Cloud, or you can use the open source plugins to conne…”
   • [claimed-docs] “For any provider not included, you can override the API key and base URL at initialization for the LLM, STT, and TTS interfaces in the plugi…”

   proof ↗⚑ flag

Voices

1. founderClone a custom brand voice and use it for my agents, with a documented consent process

   weight 2 · round drawn
   ElevenLabs Agents—none0/10

   The evidence pack describes selecting from 5k+ pre-built voices and voice customization (speed, multi-voice switching) but contains no mention of a voice-cloning feature for creating a custom brand voice, nor any documented consent process for cloning (e.g., voice verification, spoken consent statement) required by ElevenLabs' broader platform. Missing for 10: any docs on voice cloning/Instant or Professional Voice Cloning for Agents, and any consent-verification workflow.

   • [claimed-docs] “Select from 5k+ voices across 31 languages with customization options”
   • [claimed-docs] “Enable your agent to switch between different voices for multi-character conversations, storytelling, and language tutoring.”
   • [claimed-docs] “You can customize various aspects of your AI agent's voice to create a more natural and engaging conversation experience. This includes cont…”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

2. developerChoose from a broad voice library or plug in multiple TTS providers to get the voice I want

   weight 2 · round to LiveKit Agents
   ElevenLabs Agents~partialclaimed5/10

   Docs confirm a broad in-house voice library (5k+ voices across 31 languages, docs-6) with customization like speed, pronunciation, and multi-voice switching (docs-17/32/48), satisfying the first half of the story. However, there is no evidence of plugging in third-party/alternative TTS providers — the docs only mention bring-your-own LLM (docs-41), not bring-your-own TTS engine. Missing for 10: any documented support for external/third-party TTS provider integration, evidence of provider-agnostic voice pipeline, independent confirmation of using non-ElevenLabs voices.

   • [claimed-docs] “Select from 5k+ voices across 31 languages with customization options”
   • [claimed-docs] “Enable your agent to switch between different voices for multi-character conversations, storytelling, and language tutoring.”
   • [claimed-docs] “You can customize various aspects of your AI agent's voice to create a more natural and engaging conversation experience. This includes cont…”
   • [claimed-docs] “Enable your agent to switch between different voices for multi-character conversations, storytelling, and language tutoring”
   • [claimed-docs] “Choose from supported LLMs or bring your own custom model”

   proof ↗⚑ flag

   LiveKit Agents~partialclaimed6/10

   Docs confirm a large open-source plugin ecosystem for TTS/STT/LLM providers, plus LiveKit Inference for provider access, and an override mechanism for API key/base URL for providers not natively included, supporting multi-provider TTS flexibility. However, there's no explicit mention of a 'voice library' (e.g., curated voice catalog/selection UI) or enumeration of specific TTS providers/voices, so the 'broad voice library' half of the story is unevidenced. Missing for 10: explicit voice catalog/list of supported TTS providers and voices, evidence of ease of switching between TTS voices, independent confirmation of provider breadth.

   • [claimed-docs] “LiveKit Agents includes a large ecosystem of open source plugins for a variety of AI providers.”
   • [claimed-docs] “You can use LiveKit Inference to access many of these models directly through LiveKit Cloud, or you can use the open source plugins to conne…”
   • [claimed-docs] “For any provider not included, you can override the API key and base URL at initialization for the LLM, STT, and TTS interfaces in the plugi…”

   proof ↗⚑ flag