LiveKit Agents vs Deepgram Voice Agent
LiveKit Agents
LiveKit
LiveKit Agents wins · 22–14 (25 drawn)
Agent building — building agents — abstractions, tool wiring, control flowAgent building
Building agents — abstractions, tool wiring, control flow
Agent ops
ai-native userMy coding agent can provision a complete voice agent end to end — create the agent, attach a number, and place a call — through the API, CLI, or MCP without touching the dashboard
weight 3 · round drawnLiveKit provides a real CLI (`lk`), a documented telephony/SIP stack for provisioning inbound/outbound trunks and placing calls, and framework docs explicitly note it's 'built for coding agents like Claude Code, Cursor, and Codex,' suggesting API/CLI-driven workflows are possible without the dashboard. However, the MCP support documented is for the agent's own tool-calling (consuming MCP servers as tools), not an MCP server exposing agent-provisioning/number-attachment/call-placing actions to a coding agent, and no single evidenced workflow shows an agent programmatically creating an agent, attaching a number, and placing a call end-to-end via CLI/API in one flow — the Agent Builder path shown is dashboard/browser-based (no-code), which contradicts the 'without touching dashboard' framing. Missing for 10: an MCP server (or CLI/API recipe) that lets a coding agent itself create an agent, provision/attach a phone number, and place a call in one documented end-to-end sequence.
- [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”
- [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
- [claimed-docs] “Outbound trunks are used to place outgoing calls.”
- [probe] “official CLI documented at https://github.com/livekit/livekit-cli”
- [probe] “PROBE runtime (recorded 2026-09-05): `lk --version` printed `lk version 2.18.6` after a plain `brew install livekit-cli` — the official CLI …”
- [claimed-docs] “LiveKit is built for coding agents like Claude Code, Cursor, and Codex.”
- [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
- [claimed-docs] “LiveKit Agents has first-class support for Model Context Protocol (MCP) servers.”
Deepgram documents agent creation via reusable agent configs (UUID-based), telephony connectivity for inbound/outbound calls, and a CLI with a built-in MCP server giving AI coding tools API access — the building blocks for programmatic provisioning exist. However, the evidence shows phone-call handling is done via a Twilio bridge (server code required) rather than a native Deepgram 'attach number/place call' API, and no doc shows the CLI or MCP server actually exposing agent-create + number-attach + call-place as a single end-to-end flow. Missing for 10: explicit CLI/MCP commands for provisioning a phone number and placing a call, and confirmation that this full workflow avoids external glue code/dashboard steps.
- [claimed-docs] “Reusable Agent Configurations allow you to define and persist the agent block of your Settings message using the Deepgram API. Once created,…”
- [claimed-docs] “Telephony: connect voice agents to phone networks for inbound and outbound calls.”
- [claimed-docs] “The dg CLI includes a built-in MCP (Model Context Protocol) server that gives AI coding tools direct access to Deepgram APIs.”
- [claimed-docs] “Twilio streams the call audio to your server, and your server bridges that audio to the Deepgram Voice Agent API, a single WebSocket that ru…”
- [probe] “official MCP server documented at https://developers.deepgram.com/developer-tools/agentic-tools”
- [probe] “official CLI documented at https://developers.deepgram.com/developer-tools/cli/getting-started”
ai-native userThe platform's own AI helps me author agents — generating or improving prompts, flows, and test cases from a description
weight 1 · round drawnLiveKit Agentsnone0/10Evidence shows a no-code 'Agent Builder' for browser prototyping and a testing framework, but nothing indicates the platform itself uses AI to generate or improve prompts, flows, or test cases from a natural-language description of the desired agent.
- [claimed-docs] “LiveKit Agent Builder to prototype and deploy agents directly in your browser without writing code”
- [claimed-docs] “LiveKit Agent Builder: Prototype and deploy voice agents directly in your browser, without writing any code.”
- [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
- [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes... Agent simulations run end-to-end conversations between your agent and…”
Deepgram Voice Agentnone0/10Evidence covers building agents manually (prompting, function calling, reusable configs, MCP server for coding tools) but there is no mention of the platform's own AI generating or improving prompts, flows, or test cases from a description; the MCP server exposes Deepgram APIs to external coding assistants rather than being an AI author within the platform itself.
- [claimed-docs] “Prompting: write system prompts that shape live-call behavior.”
- [claimed-docs] “Reusable Agent Configurations allow you to define and persist the agent block of your Settings message using the Deepgram API. Once created,…”
- [claimed-docs] “The dg CLI includes a built-in MCP (Model Context Protocol) server that gives AI coding tools direct access to Deepgram APIs.”
- [probe] “official MCP server documented at https://developers.deepgram.com/developer-tools/agentic-tools”
Build
developerBuild a working phone voice agent — prompt, voice, and phone number — and take my first live call within an hour
weight 3 · round to LiveKit AgentsDocs show a <10‑minute voice-assistant quickstart (prompt+voice) plus a dedicated telephony/SIP stack for inbound/outbound calls (trunks, phone number), and runtime probes confirm the framework and self-hosted server actually install and boot without extra keys. Together these cover prompt, voice, and phone number needed for a first live call within an hour. Missing for 10: a single unified, hands-on-verified tutorial that walks through phone-number provisioning and first live call end-to-end (currently voice-quickstart and telephony docs are separate), and independent (non-vendor) confirmation of the 'within an hour' timeline.
- [claimed-docs] “Build and deploy a simple voice assistant in less than 10 minutes.”
- [claimed-docs] “Build and deploy a simple voice assistant with Python or Node.js in less than 10 minutes.”
- [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
- [claimed-docs] “Enable your voice agent to make or take phone calls.”
- [claimed-docs] “Outbound trunks are used to place outgoing calls.”
- [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”
- [probe] “PROBE runtime (recorded 2026-09-05): `lk --version` printed `lk version 2.18.6` after a plain `brew install livekit-cli` — the official CLI …”
- [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
- [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”
Deepgram provides all core building blocks — prompting, voice/LLM selection, telephony via Twilio bridging, and a single WebSocket API — that together could plausibly get a developer to a first live call quickly. However, the telephony path requires standing up your own server to bridge Twilio audio to the Voice Agent WebSocket, which is nontrivial integration work rather than a turnkey 'phone number in one hour' flow, and there's no first-party quickstart or time-to-first-call benchmark cited. missing for 10: an end-to-end quickstart/tutorial demonstrating full setup within an hour, evidence of a managed/no-code telephony number provisioning path, and independent hands-on confirmation of setup speed.
- [claimed-docs] “Deepgram's Voice Agent API collapses that stack into a single, unified API. You open one WebSocket connection, send audio in, and receive au…”
- [claimed-docs] “Telephony: connect voice agents to phone networks for inbound and outbound calls.”
- [claimed-docs] “Prompting: write system prompts that shape live-call behavior.”
- [claimed-docs] “Supported LLM providers | Parameter | open_ai | anthropic | aws_bedrock | google | groq | nvidia”
- [claimed-docs] “Twilio streams the call audio to your server, and your server bridges that audio to the Deepgram Voice Agent API, a single WebSocket that ru…”
- [claimed-docs] “That includes barge-in, the ability to talk over the agent and have it stop instantly.”
developerRun conversations in multiple languages, including detecting and switching language mid-call
weight 2 · round to Deepgram Voice AgentLiveKit Agentsnone0/10No evidence in the pack addresses multilingual conversation support, language detection, or dynamic language switching mid-call; the docs cover turn detection, interruption handling, tool use, MCP, and telephony but never mention STT/TTS language selection or switching logic. This is a fair axis for a voice-agent framework (many STT/TTS providers support multi-language), so absence of evidence means 'none' rather than 'na'.
Deepgram's docs confirm multilingual voice agent support, noting that STT and TTS model choices must be configured for the target language (deepgram-docs-10), but the evidence pack does not explicitly document automatic language detection or dynamic switching mid-call — only static multilingual configuration is described. Missing for 10: explicit documentation of automatic language detection, mid-call language switching mechanics, and independent/hands-on confirmation of this behavior in production.
- [claimed-docs] “A multilingual voice agent has two model decisions: which STT model transcribes the user, and which TTS model speaks the agent.”
founderDesign multi-step conversation flows in a visual builder with branching, states, and handoffs without writing code
weight 2 · round to LiveKit AgentsLiveKit mentions a browser-based 'Agent Builder' for no-code prototyping of voice agents, but the evidence never describes visual branching, explicit states, or handoff design — the core framework is fundamentally code-first (Python/Node.js) with tools, MCP, and turn-detection logic. missing for 10: documentation of branching/state UI, handoff modeling, or any screenshots/examples of the Agent Builder's flow-design capabilities beyond a vague no-code prototyping claim.
- [claimed-docs] “Prototype and deploy voice agents directly in your browser, without writing any code.”
- [claimed-docs] “LiveKit Agent Builder to prototype and deploy agents directly in your browser without writing code”
- [claimed-docs] “LiveKit Agent Builder: Prototype and deploy voice agents directly in your browser, without writing any code.”
Deepgram Voice Agentnone0/10All evidence describes a code/API-first architecture (WebSocket connections, JSON Settings messages, system prompts, function calling, CLI/SDK) rather than a visual no-code builder; there is no mention of a drag-and-drop flow designer, branching UI, or state-machine editor. missing for 10: any visual builder UI, no-code branching/state design, or drag-and-drop handoff configuration.
- [claimed-docs] “Deepgram's Voice Agent API collapses that stack into a single, unified API. You open one WebSocket connection, send audio in, and receive au…”
- [claimed-docs] “Multi-Agent Architecture: orchestrate multiple specialized agents that hand off based on context, intent, or domain.”
- [claimed-docs] “Reusable Agent Configurations allow you to define and persist the agent block of your Settings message using the Deepgram API. Once created,…”
- [claimed-docs] “Prompting: write system prompts that shape live-call behavior.”
- [claimed-docs] “Supported LLM providers | Parameter | open_ai | anthropic | aws_bedrock | google | groq | nvidia”
Personalization
developerInject dynamic variables and per-caller context at call time so each conversation is personalized
weight 2 · round to Deepgram Voice AgentLiveKit Agentsnone0/10The evidence pack covers tool use, MCP, telephony, testing, and turn detection, but nothing addresses injecting dynamic variables or per-caller context (e.g., participant/room metadata, job context) into an agent's prompt or session at call start. Missing for 10: any documentation of job/participant metadata APIs, per-call context injection, or dynamic prompt personalization.
Deepgram supports prompting, reusable agent configs (passed at call time via UUID), function calling, and mid-call message injection, which together allow injecting per-call context/variables into agent behavior; A/B testing of configs also implies runtime parameterization. However there is no explicit documented mechanism for templated dynamic variables (e.g., {{caller_name}}) or a dedicated per-caller context API akin to other platforms. missing for 10: explicit dynamic variable templating syntax, dedicated per-caller metadata injection API, hands-on example of personalizing a call with caller-specific data.
- [claimed-docs] “Reusable Agent Configurations allow you to define and persist the agent block of your Settings message using the Deepgram API. Once created,…”
- [claimed-docs] “Prompting: write system prompts that shape live-call behavior.”
- [claimed-docs] “Inject agent message | Mid-call | InjectAgentMessage | Makes the agent speak a specific line”
- [claimed-docs] “A/B testing voices or prompts — Run two configurations in parallel and measure conversion, CSAT, or containment rate to pick a winner—no cod…”
- [claimed-docs] “In the context of Deepgram Voice Agents, function calling enables your agent to perform tasks during a live conversation.”
developerGround the agent on my documents with a built-in knowledge base or RAG so it answers from my content
weight 2 · round to LiveKit AgentsLiveKit Agents supports calling external APIs/tools that a developer can use to implement RAG ("Call external APIs or lookup data for RAG"), and has full LLM tool-use and MCP integration for wiring in retrieval systems, but there is no built-in knowledge base, document ingestion, or vector-store/RAG pipeline shipped by the framework itself — developers must bring their own RAG implementation via the tools API. Missing for 10: built-in document indexing/vector store, out-of-the-box knowledge-base feature, first-party RAG pipeline or example showing document grounding end-to-end.
- [claimed-docs] “Call external APIs or lookup data for RAG.”
- [claimed-docs] “LiveKit Agents has full support for LLM tool use. This feature allows you to create a custom library of tools to extend your agent's context”
- [claimed-docs] “LiveKit Agents has full support for LLM tool use.”
- [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
- [claimed-docs] “Wrap an MCP server in an \`MCPToolset\` and pass it to the agent's \`tools\` parameter”
Deepgram Voice Agentnone0/10The evidence pack covers architecture, function calling, multi-agent handoff, telephony, prompting, and LLM providers, but nowhere documents a built-in knowledge base or RAG feature for grounding the agent on user documents. Function calling ([deepgram-docs-2]) could theoretically be wired to an external retrieval system, but that is not the same as a built-in KB/RAG capability, and no such feature is described.
- [claimed-docs] “In the context of Deepgram Voice Agents, function calling enables your agent to perform tasks during a live conversation.”
- [claimed-docs] “Multi-Agent Architecture: orchestrate multiple specialized agents that hand off based on context, intent, or domain.”
- [claimed-docs] “Prompting: write system prompts that shape live-call behavior.”
Agenticness — how well agents can access and operate the productAgenticness
How well agents can access and operate the product
Agent access
ai-native userPoint an agent at llms.txt or agent-oriented docs
weight 2 · round to Deepgram Voice AgentLiveKit publishes a working llms.txt (verified live at docs.livekit.io/llms.txt, HTTP 200) and agent-friendly markdown doc endpoints (e.g. /agents/.md), plus explicit docs noting the platform is 'built for coding agents like Claude Code, Cursor, and Codex,' confirming intentional support for AI-native doc consumption. missing for 10: no independent third-party confirmation that external agents successfully consume these endpoints in practice, and no dedicated documentation page explaining the llms.txt/agent-doc strategy itself.
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.livekit.io/llms.txt # LiveKit docs > LiveKit is an open-source platform for building voice, video,…”
- [probe] “PROBE docs-md: HTTP 200 at https://docs.livekit.io/agents/.md LiveKit docs › Build Agents › Get Started › Introduction --- # Introduction …”
- [claimed-docs] “LiveKit is built for coding agents like Claude Code, Cursor, and Codex.”
Direct probe confirms llms.txt exists at developers.deepgram.com/llms.txt (HTTP 200) with explicit AI-agent instructions, and docs pages support .md suffix for clean markdown retrieval, both verified by live probes. missing for 10: no independent third-party confirmation of an agent actually consuming this in practice.
- [probe] “PROBE llms.txt: HTTP 200 at https://developers.deepgram.com/llms.txt # Deepgram's Docs ## Instructions for AI Agents - For clean Markdown …”
- [probe] “PROBE docs-md: HTTP 200 at https://developers.deepgram.com/docs/voice-agent.md > For clean Markdown of any page, append .md to the page URL.…”
ai-native userRun the product headlessly / in CI for automation
weight 2 · round to LiveKit AgentsLiveKit Agents is a Python/Node.js SDK plus self-hostable server that runs as a headless programmatic participant with no UI dependency; it's pip/npm-installable, has a built-in test framework (pytest/Vitest, agent simulations) suitable for CI, and the server/CLI were verified runtime to work keylessly. missing for 10: no explicit documented CI pipeline example (e.g., GitHub Actions config) or headless-mode confirmation beyond inference from server/testing docs.
- [claimed-docs] “A programmatic participant is any code that joins a LiveKit room as a participant — this includes AI agents, media processors, or custom log…”
- [claimed-docs] “The Agents framework isn't limited to AI agents. You can use it to deploy any code that needs to process realtime media and data streams as …”
- [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
- [claimed-docs] “Agent simulations run end-to-end conversations between your agent and an LLM-driven user, then evaluate the results across the full interact…”
- [github] “Builtin test framework: Write tests and use judges to ensure your agent is performing as expected.”
- [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
- [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”
The Voice Agent is a WebSocket API (no GUI dependency) and Deepgram ships a terminal-based `dg` CLI for scripting Deepgram operations (transcribe, synthesize, manage account) which is inherently automatable/CI-friendly, and temporary tokens support secure automated auth. However, there is no explicit documentation of running the Voice Agent itself headlessly in CI pipelines, no CI/CD examples, and the CLI's primary use-cases described are transcription/synthesis rather than orchestrating full voice-agent sessions programmatically. Missing for 10: dedicated CI/headless automation guide for Voice Agent sessions, example CI pipeline configs, and confirmation the CLI can drive the Agent API end-to-end rather than just STT/TTS.
- [claimed-docs] “Deepgram's Voice Agent API collapses that stack into a single, unified API. You open one WebSocket connection, send audio in, and receive au…”
- [claimed-docs] “The dg CLI lets you transcribe files, stream live audio, synthesize speech, analyze text, and manage your Deepgram account from the terminal…”
- [claimed-docs] “Unlike traditional API keys, temporary tokens are ideal for real-time applications requiring secure, temporary access to Deepgram's services…”
- [probe] “official CLI documented at https://developers.deepgram.com/developer-tools/cli/getting-started”
ai-native userPlug MCP servers into this product so it can use their tools
weight 3 · round to LiveKit AgentsDocs and GitHub explicitly state 'first-class support for Model Context Protocol (MCP) servers' with a documented pattern to wrap an MCP server in an MCPToolset and pass it to the agent's tools parameter, described as one line of code integration. This is first-party documentation without independent hands-on verification of MCP-specific usage. Missing for 10: independent/community confirmation of MCP tool integration working end-to-end, and more detail on multi-server or auth configurations.
- [claimed-docs] “Wrap an MCP server in an \`MCPToolset\` and pass it to the agent's \`tools\` parameter”
- [claimed-docs] “Wrap an MCP server in an `MCPToolset` and pass it to the agent's `tools` parameter”
- [claimed-docs] “LiveKit Agents has first-class support for Model Context Protocol (MCP) servers.”
- [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
- [github] “Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
Deepgram Voice Agentnone0/10Deepgram Voice Agent supports function calling for custom tool use, but there is no evidence the Voice Agent can act as an MCP client to consume external MCP servers' tools. The only MCP-related evidence is a built-in MCP *server* in the `dg` CLI that lets coding tools access Deepgram's APIs — the opposite direction of the story.
- [claimed-docs] “In the context of Deepgram Voice Agents, function calling enables your agent to perform tasks during a live conversation.”
- [claimed-docs] “The dg CLI includes a built-in MCP (Model Context Protocol) server that gives AI coding tools direct access to Deepgram APIs.”
- [probe] “official MCP server documented at https://developers.deepgram.com/developer-tools/agentic-tools”
ai-native userConnect an agent via an official MCP server
weight 3 · round to Deepgram Voice AgentLiveKit Agentsnone0/10The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)
Deepgram ships an official MCP server (bundled in the dg CLI) that lets AI coding tools/agents connect directly to Deepgram's APIs, confirmed both in docs and via a dedicated agentic-tools page. This directly satisfies the story of connecting an agent to Deepgram via an official MCP server. Missing for 10: independent/hands-on verification that the MCP server works reliably in practice, and more detail on which Voice Agent capabilities specifically are exposed through it.
- [claimed-docs] “The dg CLI includes a built-in MCP (Model Context Protocol) server that gives AI coding tools direct access to Deepgram APIs.”
- [probe] “official MCP server documented at https://developers.deepgram.com/developer-tools/agentic-tools”
- [claimed-docs] “The dg CLI lets you transcribe files, stream live audio, synthesize speech, analyze text, and manage your Deepgram account from the terminal…”
ai-native userUse an official CLI
weight 2 · round to LiveKit AgentsLiveKit ships an official CLI (livekit-cli), documented on GitHub and verified hands-on to install and run (`lk --version` works via brew install), confirming it's a real, functional official CLI supporting the agent workflow. missing for 10: no deep documentation of full CLI command surface for agent-specific workflows within the evidence pack.
Deepgram ships an official 'dg' CLI for transcribing files, streaming audio, synthesizing speech, and managing accounts from the terminal, and it even embeds an MCP server for AI coding tools, directly supporting agentic/AI-native workflows. Missing for 10: independent/hands-on corroboration beyond first-party docs and more detail on CLI coverage of Voice Agent-specific features.
- [claimed-docs] “The dg CLI lets you transcribe files, stream live audio, synthesize speech, analyze text, and manage your Deepgram account from the terminal…”
- [claimed-docs] “The dg CLI includes a built-in MCP (Model Context Protocol) server that gives AI coding tools direct access to Deepgram APIs.”
- [probe] “official CLI documented at https://developers.deepgram.com/developer-tools/cli/getting-started”
- [probe] “official MCP server documented at https://developers.deepgram.com/developer-tools/agentic-tools”
ai-native userDrive the product through a documented public API
weight 3 · round to Deepgram Voice AgentLiveKit Agents ships a well-documented, pip/npm-installable Python and Node.js SDK with extensive public API surface (tools, MCP, turn detection, telephony, testing), plus a llms.txt and hands-on verified installs/runtime confirming the API is real and usable by AI-native developers. Missing for 10: independent third-party API reference/versioning audit and deeper evidence of API stability guarantees beyond docs and probes.
- [claimed-docs] “The Agents framework lets you add any Python or Node.js program to LiveKit rooms as full realtime participants.”
- [claimed-docs] “A programmatic participant is any code that joins a LiveKit room as a participant — this includes AI agents, media processors, or custom log…”
- [claimed-docs] “The Agents framework isn't limited to AI agents. You can use it to deploy any code that needs to process realtime media and data streams as …”
- [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
- [claimed-docs] “LiveKit Agents has full support for LLM tool use. This feature allows you to create a custom library of tools to extend your agent's context”
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.livekit.io/llms.txt # LiveKit docs > LiveKit is an open-source platform for building voice, video,…”
- [probe] “PROBE docs-md: HTTP 200 at https://docs.livekit.io/agents/.md LiveKit docs › Build Agents › Get Started › Introduction --- # Introduction …”
- [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”
Deepgram publishes a documented public API (WebSocket-based Voice Agent API plus OpenAPI spec confirmed live), extensive docs covering endpoints, function calling, LLM providers, telephony, and auth, and even a CLI/MCP server for programmatic/agentic access, making it clearly drivable by an AI-native user. Missing for 10: independent third-party corroboration of API usage beyond vendor docs.
- [claimed-docs] “Deepgram's Voice Agent API collapses that stack into a single, unified API. You open one WebSocket connection, send audio in, and receive au…”
- [probe] “PROBE openapi: HTTP 200 at https://developers.deepgram.com/openapi.json — contains "openapi" key”
- [claimed-docs] “The dg CLI lets you transcribe files, stream live audio, synthesize speech, analyze text, and manage your Deepgram account from the terminal…”
- [claimed-docs] “The dg CLI includes a built-in MCP (Model Context Protocol) server that gives AI coding tools direct access to Deepgram APIs.”
- [probe] “official CLI documented at https://developers.deepgram.com/developer-tools/cli/getting-started”
- [claimed-docs] “Unlike traditional API keys, temporary tokens are ideal for real-time applications requiring secure, temporary access to Deepgram's services…”
ai-native userIssue scoped/least-privilege API credentials for an agent
weight 2 · round to Deepgram Voice AgentLiveKit Agentsnone0/10The evidence pack covers voice/agent features, tool-calling, MCP, testing, and telephony, but contains no mention of scoped or least-privilege API key/token issuance for agents (e.g., LiveKit's grant-based access tokens or credential scoping). Since LiveKit is a platform with API keys and would plausibly support such scoping, this is an applicable but undocumented axis in the given evidence.
Deepgram documents temporary, short-lived tokens as an alternative to traditional API keys for secure, limited-duration access (deepgram-docs-14), which partially satisfies a least-privilege credential story, but there is no evidence of granular scopes/permissions (e.g., restricting a key to specific endpoints or agent capabilities) or role-based access control for agents. Missing for 10: documented scope/permission granularity, per-agent key restriction, and evidence of enforcement/verification of least-privilege in practice.
- [claimed-docs] “Unlike traditional API keys, temporary tokens are ideal for real-time applications requiring secure, temporary access to Deepgram's services…”
ai-native userBuild against official SDKs
weight 2 · round to LiveKit AgentsLiveKit provides official Python and Node.js SDKs (livekit-agents), verified hands-on to pip-install and import with no key, plus documented plugin ecosystem, tool use, MCP support, testing frameworks, and telephony integration—clearly an official SDK ecosystem for AI-native building. Missing for 10: independent third-party benchmarking of SDK API stability/versioning beyond community latency complaints unrelated to SDK build story.
- [claimed-docs] “The Agents framework lets you add any Python or Node.js program to LiveKit rooms as full realtime participants.”
- [claimed-docs] “LiveKit Agents includes a large ecosystem of open source plugins for a variety of AI providers.”
- [claimed-docs] “LiveKit Agents has full support for LLM tool use. This feature allows you to create a custom library of tools to extend your agent's context”
- [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
- [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”
- [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
Evidence shows an official 'Browser Agent SDK' with composable packages, plus an OpenAPI spec, CLI, and MCP server for developer tooling, indicating some official SDK/build tooling exists. However, evidence does not mention broader server-side/language SDKs (Python, Node, Go, etc.) commonly expected for building agentic applications, nor independent corroboration of SDK quality. Missing for 10: multi-language SDK documentation, independent/hands-on developer reports, broader agentic build examples beyond browser.
- [claimed-docs] “Browser Agent SDK: add voice AI to any web application via four composable packages.”
- [probe] “PROBE openapi: HTTP 200 at https://developers.deepgram.com/openapi.json — contains "openapi" key”
- [probe] “official CLI documented at https://developers.deepgram.com/developer-tools/cli/getting-started”
- [probe] “official MCP server documented at https://developers.deepgram.com/developer-tools/agentic-tools”
ai-native userSubscribe to events via webhooks
weight 2 · round drawnLiveKit Agentsnone0/10No evidence pack items mention webhooks or an event-subscription mechanism for LiveKit Agents; the docs cover MCP tool integration, telephony, testing, and turn detection, but nothing about webhook-based event notifications.
Deepgram Voice Agentnone0/10The evidence pack shows Deepgram Voice Agent operates via a persistent WebSocket connection for real-time audio streaming, not webhook-based event subscription; no documentation mentions webhooks for event notifications (e.g., call completion, transcript ready, errors). Absence of evidence for an applicable capability yields 'none'.
Agentic features
ai-native userGet AI-generated insights and suggestions from my data inside the product
weight 2 · round drawnLiveKit Agentsnone0/10The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)
Deepgram Voice Agentnone0/10The evidence pack covers Deepgram Voice Agent's API capabilities (function calling, multi-agent orchestration, telephony, LLM providers, prompting) but contains no mention of an in-product analytics dashboard, conversation insights, or AI-generated suggestions derived from a user's own data. While such a feature (e.g., call analytics/insights) is plausible for a voice AI platform, no evidence shows Deepgram surfaces this.
ai-native userSet up automations that run autonomously in the background
weight 2 · round to LiveKit AgentsLiveKit Agents workers run as persistent 'programmatic participants' deployed to LiveKit Cloud or self-hosted infrastructure, processing realtime streams and telephony calls autonomously in the background without human intervention (docs-8, docs-20, docs-30, docs-26, gh-2), and this is verified hands-on as a real pip-installable OSS framework with a working self-hosted backbone (probe-rt-2, probe-rt-3). Missing for 10: explicit scheduling/cron-trigger mechanisms, independent third-party reports of long-running unattended stability (the one community mention is about observability tooling gaps, not autonomy failure).
- [claimed-docs] “A programmatic participant is any code that joins a LiveKit room as a participant — this includes AI agents, media processors, or custom log…”
- [claimed-docs] “The Agents framework isn't limited to AI agents. You can use it to deploy any code that needs to process realtime media and data streams as …”
- [claimed-docs] “Run your agent on LiveKit Cloud's global infrastructure.”
- [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
- [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”
- [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
- [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”
ai-native userDelegate tasks to a built-in AI assistant inside the product
weight 3 · round drawnLiveKit Agentsnone0/10The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)
ai-native userOperate the product with natural-language commands
weight 2 · round to Deepgram Voice AgentLiveKit ships an 'Agent Builder' that lets users prototype/deploy voice agents in-browser without code, and docs explicitly note the framework is 'built for coding agents like Claude Code, Cursor, and Codex' plus agent-readable llms.txt docs — suggesting some AI-native/no-code operability. However there is no evidence of a native natural-language command interface for operating LiveKit itself (e.g., NL-driven CLI, chatops control plane); the MCP support described is for agents calling external tools, not for controlling LiveKit via NL. Missing for 10: a documented NL-driven interface/CLI for configuring or operating the platform itself, and independent confirmation of the Agent Builder's no-code UX actually being NL-based rather than GUI-based.
- [claimed-docs] “LiveKit is built for coding agents like Claude Code, Cursor, and Codex.”
- [claimed-docs] “Prototype and deploy voice agents directly in your browser, without writing any code.”
- [claimed-docs] “LiveKit Agent Builder to prototype and deploy agents directly in your browser without writing code”
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.livekit.io/llms.txt # LiveKit docs > LiveKit is an open-source platform for building voice, video,…”
- [probe] “PROBE docs-md: HTTP 200 at https://docs.livekit.io/agents/.md LiveKit docs › Build Agents › Get Started › Introduction --- # Introduction …”
Deepgram Voice Agent is fundamentally a conversational system: users interact via natural spoken language, and the agent supports function calling to perform tasks, live prompting to shape behavior, and mid-call message injection—all driven by natural-language conversation rather than rigid commands. This is core, well-documented functionality (docs-2, docs-7, docs-11) directly matching the story's intent for an ai-native/agentic persona. Missing for 10: independent/hands-on validation of natural-language command accuracy and no evidence of complex multi-turn command chaining reliability from third-party sources.
- [claimed-docs] “In the context of Deepgram Voice Agents, function calling enables your agent to perform tasks during a live conversation.”
- [claimed-docs] “Prompting: write system prompts that shape live-call behavior.”
- [claimed-docs] “Inject agent message | Mid-call | InjectAgentMessage | Makes the agent speak a specific line”
- [claimed-docs] “Multi-Agent Architecture: orchestrate multiple specialized agents that hand off based on context, intent, or domain.”
Api quality
ai-native userExplore an interactive API reference with runnable examples
weight 2 · round drawnLiveKit Agentsnone0/10Evidence shows extensive quickstart guides, markdown-based docs (llms.txt), and a no-code 'Agent Builder' prototyping tool, but nothing describes an interactive API reference (e.g., embedded code sandbox, live runnable examples, or Swagger-like explorer) for exploring the SDK/API itself.
Deepgram Voice Agentnone0/10Evidence shows an OpenAPI spec exists and docs pages are available as clean Markdown, but there is no evidence of an interactive API reference with runnable/try-it-now examples (e.g., embedded playground, live code execution, or Swagger/Postman-style explorer). Missing for 10: interactive explorer UI, runnable/executable code snippets, live request/response testing.
- [probe] “PROBE openapi: HTTP 200 at https://developers.deepgram.com/openapi.json — contains "openapi" key”
- [probe] “PROBE llms.txt: HTTP 200 at https://developers.deepgram.com/llms.txt # Deepgram's Docs ## Instructions for AI Agents - For clean Markdown …”
- [probe] “PROBE docs-md: HTTP 200 at https://developers.deepgram.com/docs/voice-agent.md > For clean Markdown of any page, append .md to the page URL.…”
ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)
weight 2 · round to Deepgram Voice AgentLiveKit Agentsnone0/10The evidence pack documents LiveKit's SDKs, CLI, docs (llms.txt), and framework capabilities but contains no mention of a downloadable OpenAPI/Swagger spec or other machine-readable API description for LiveKit's server or Agents APIs. Since LiveKit exposes a real API surface (server API, cloud API), this axis applies but is unevidenced.
- [probe] “official CLI documented at https://github.com/livekit/livekit-cli”
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.livekit.io/llms.txt # LiveKit docs > LiveKit is an open-source platform for building voice, video,…”
- [probe] “PROBE docs-md: HTTP 200 at https://docs.livekit.io/agents/.md LiveKit docs › Build Agents › Get Started › Introduction --- # Introduction …”
A probe confirms Deepgram publishes a machine-readable OpenAPI spec at openapi.json containing the 'openapi' key, directly satisfying the story. Missing for 10: no first-party docs page explicitly announcing/describing the OpenAPI spec's coverage or versioning.
- [probe] “PROBE openapi: HTTP 200 at https://developers.deepgram.com/openapi.json — contains "openapi" key”
ai-native userTest against a sandbox environment without touching production data
weight 1 · round to LiveKit AgentsLiveKit provides a self-hostable dev server (`livekit-server --dev`) that boots locally with placeholder keys, separate from any production deployment, plus a built-in testing framework with behavioral tests and LLM-driven agent simulations that evaluate agent behavior without needing real production data. However, there is no explicitly branded 'sandbox environment' or staging/production data-isolation feature documented — the sandbox-like capability is inferred from dev-mode self-hosting and test simulations rather than a dedicated sandbox product feature. Missing for 10: an explicit sandbox/staging environment offering with documented separation from production data, and independent confirmation that test simulations never touch production data stores.
- [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
- [claimed-docs] “Agent simulations run end-to-end conversations between your agent and an LLM-driven user, then evaluate the results across the full interact…”
- [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes... Agent simulations run end-to-end conversations between your agent and…”
- [github] “Builtin test framework: Write tests and use judges to ensure your agent is performing as expected.”
- [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
Deepgram Voice Agentnone0/10No evidence pack item mentions a sandbox environment, test/dev API keys, or a way to test without touching production data or usage; only production endpoints, EU endpoint, and opt-out flags are documented. Missing for 10: sandbox/test-mode environment, documentation of non-production keys or test credits, any guidance on isolating test traffic from production data.
ai-native userRely on versioned APIs with a documented deprecation policy
weight 2 · round drawnLiveKit Agentsnone0/10No evidence in the pack mentions API versioning scheme, version compatibility guarantees, or a documented deprecation policy for LiveKit Agents' SDKs or APIs.
Automation depth — how much of the product can run unattendedAutomation depth
How much of the product can run unattended
ai-native userPerform bulk operations across many items at once
weight 2 · round drawnLiveKit Agentsnone0/10The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)
ai-native userDefine rules that trigger actions automatically on events
weight 3 · round drawnLiveKit Agents supports event-driven automatic behaviors — tool-calling on LLM decisions, turn-detector triggering response timing, and adaptive interruption handling triggering barge-in logic — all of which are 'events auto-trigger actions' patterns built into the agent runtime. However, the evidence shows these as built-in framework behaviors and developer-coded event handlers rather than a user-facing declarative rule engine (e.g., no evidence of a 'when X happens do Y' config surface, webhook/rule subscription API, or no-code rule builder). Missing for 10: explicit rule-definition/webhook-trigger API exposed to non-developer users, documentation of a generalized event-subscription system beyond tool calls and turn logic, and independent confirmation of custom rule automation in production use.
- [claimed-docs] “Call external APIs or lookup data for RAG.”
- [claimed-docs] “LiveKit Agents has full support for LLM tool use. This feature allows you to create a custom library of tools to extend your agent's context”
- [claimed-docs] “LiveKit Agents has full support for LLM tool use.”
- [claimed-docs] “LiveKit's `TurnDetector` is an audio model that encodes user audio directly, capturing both _what_ is said and _how_ it's said.”
- [claimed-docs] “A turn detector model can predict that they have more to say and wait for them to finish before responding.”
- [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response... to identify intentional interruptions (…”
- [claimed-docs] “the model analyzes the acoustic signals to identify intentional interruptions (barge-ins) from conversational backchanneling”
- [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response.”
Deepgram Voice Agent supports function calling that lets the agent trigger actions during a live conversation, and features like InjectAgentMessage and multi-agent handoff suggest some event-driven behavior, but this is developer-defined logic (via function calling code) rather than a declarative rules engine for automatically triggering actions on arbitrary events. missing for 10: a documented rules/trigger engine (if-this-then-that style), event subscription system, or automation workflow builder distinct from manual function-calling code.
- [claimed-docs] “In the context of Deepgram Voice Agents, function calling enables your agent to perform tasks during a live conversation.”
- [claimed-docs] “Multi-Agent Architecture: orchestrate multiple specialized agents that hand off based on context, intent, or domain.”
- [claimed-docs] “Inject agent message | Mid-call | InjectAgentMessage | Makes the agent speak a specific line”
ai-native userSchedule recurring jobs or workflows
weight 2 · round drawnLiveKit Agentsnone0/10The evidence pack covers LiveKit Agents' realtime voice/video agent capabilities, tool use, MCP integration, telephony, and testing, but contains no mention of any scheduling, cron-like recurring job, or workflow-automation trigger mechanism. This is a fair capability to ask about for an automation-focused agent framework, but no evidence supports it.
ai-native userVersion, review, and roll back my automations
weight 1 · round to LiveKit AgentsThe pricing page explicitly claims 'Instant rollback to a previous agent deployment,' and the testing/simulation framework (behavioral tests, LLM-judge simulations) supports a review step before deployment, but there's no documented versioning system, changelog, or diff/history UI for automations. Missing for 10: explicit version history/diffing of agent configs, a documented review/approval workflow beyond test suites, and independent confirmation that rollback works in practice.
- [claimed-docs] “Instant rollback to a previous agent deployment”
- [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
- [claimed-docs] “Agent simulations run end-to-end conversations between your agent and an LLM-driven user, then evaluate the results across the full interact…”
- [github] “Builtin test framework: Write tests and use judges to ensure your agent is performing as expected.”
Deepgram's Reusable Agent Configurations let you persist a named agent config and reference it by UUID, and support A/B testing two configs in parallel, which is adjacent to versioning, but there is no documented version history, diff/review workflow, or explicit rollback mechanism. missing for 10: version history tracking, review/approval workflow, explicit rollback capability, changelog or diffing between config versions.
- [claimed-docs] “Reusable Agent Configurations allow you to define and persist the agent block of your Settings message using the Deepgram API. Once created,…”
- [claimed-docs] “A/B testing voices or prompts — Run two configurations in parallel and measure conversion, CSAT, or containment rate to pick a winner—no cod…”
Compliance trust — stories about compliance trust in this arenaCompliance trust
Stories about compliance trust in this arena
Compliance
founderMeet call-recording consent and disclosure obligations with per-call recording controls and configurable data retention
weight 2 · round to Deepgram Voice AgentLiveKit Agentsnone0/10No evidence in the pack mentions call recording, consent disclosure, or data retention controls anywhere in LiveKit Agents docs, GitHub, or community sources; the pack covers telephony, tool use, MCP, testing, and turn detection but nothing about recording/retention compliance features. missing for 10: recording consent/disclosure mechanisms, per-call recording toggles, configurable data retention policies, and any documentation or hands-on evidence of these compliance controls.
Deepgram documents general data-privacy controls (per-request opt-out from retention via mip_opt_out, EU-region processing endpoint) that touch on data retention/residency, but there is no explicit documentation of per-call recording toggles, consent/disclosure workflows, or configurable retention windows tied to call recordings specifically. Missing for 10: explicit per-call recording enable/disable controls, documented retention period configuration for stored call audio/transcripts, and consent/disclosure feature support.
- [claimed-docs] “Opt out per request with mip_opt_out=true. Opted-out requests are not retained”
- [claimed-docs] “For customers requiring data processing within the European Union, Deepgram provides an EU-specific endpoint at api.eu.deepgram.com.”
platform-engineerRun regulated workloads with HIPAA/BAA support, SOC 2, and data-residency options
weight 2 · round to Deepgram Voice AgentLiveKit Agentsnone0/10The evidence pack contains no mention of HIPAA/BAA agreements, SOC 2 certification, or data-residency controls anywhere in LiveKit's docs, GitHub repo, or probes; self-hosting and TURN/security items only cover TLS/SSL and self-hosted deployment, not compliance attestations. As a platform serving enterprise/regulated workloads, this axis clearly applies, so lack of evidence yields 'none'.
- [claimed-docs] “The good news is LiveKit includes an embedded TURN server. It's a secure TURN implementation that has integrated authentication with the res…”
- [claimed-docs] “In order to have a secure LiveKit deployment, you will need a domain as well as a SSL certificate for that domain.”
Evidence shows EU data-residency endpoint and an opt-out from data retention/training, indicating some compliance/trust infrastructure, but there is no mention of HIPAA/BAA support or SOC 2 certification anywhere in the pack. missing for 10: HIPAA/BAA documentation, SOC 2 attestation evidence, broader regional residency options beyond EU.
- [claimed-docs] “Opt out per request with mip_opt_out=true. Opted-out requests are not retained”
- [claimed-docs] “For customers requiring data processing within the European Union, Deepgram provides an EU-specific endpoint at api.eu.deepgram.com.”
Deployment scale — stories about deployment scale in this arenaDeployment scale
Stories about deployment scale in this arena
Scale
platform-engineerSee documented concurrency limits and scale to many simultaneous calls without manual capacity begging
weight 2 · round to LiveKit AgentsDocs mention running agents on LiveKit Cloud's global infrastructure and self-hosting with an embedded TURN server, implying scale-out capability, but no evidence pack item cites concrete documented concurrency limits, per-instance call caps, or autoscaling guarantees; a community report even flags lack of visibility into per-call cost/latency at scale. missing for 10: published concurrency/capacity numbers, autoscaling documentation, load-testing benchmarks, and confirmation that scaling requires no manual quota requests.
- [claimed-docs] “Run your agent on LiveKit Cloud's global infrastructure.”
- [claimed-docs] “The good news is LiveKit includes an embedded TURN server. It's a secure TURN implementation that has integrated authentication with the res…”
- [community] “We've been running multiple voice AI agents on LiveKit and kept running into visibility issues — no way to measure TTFT, latency across STT …”
Deepgram Voice Agentnone0/10No evidence pack item mentions concurrency limits, rate limits, per-account call caps, scaling guidance, or capacity request processes for the Voice Agent API; the docs cover architecture, features, and integrations but nothing about scale/concurrency documentation.
Self host
platform-engineerSelf-host the voice agent runtime from open-source code on my own infrastructure
weight 3 · round to LiveKit AgentsLiveKit Agents is open source, pip/npm installable and runs the agent runtime independent of any vendor account, and the underlying WebRTC/SIP server (livekit-server) is Apache-2.0 and self-hostable, confirmed via hands-on probes (booting `livekit-server --dev` locally with no keys, and installing livekit-agents/CLI keylessly) plus official self-hosting deployment docs covering TURN, TLS/domain setup. Missing for 10: no first-party production-scale self-hosting case study or independent report of large-scale self-hosted deployment beyond dev-mode probe.
- [probe] “PROBE runtime (recorded 2026-09-05): `lk --version` printed `lk version 2.18.6` after a plain `brew install livekit-cli` — the official CLI …”
- [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
- [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”
- [claimed-docs] “The good news is LiveKit includes an embedded TURN server. It's a secure TURN implementation that has integrated authentication with the res…”
- [claimed-docs] “In order to have a secure LiveKit deployment, you will need a domain as well as a SSL certificate for that domain.”
- [claimed-docs] “A programmatic participant is any code that joins a LiveKit room as a participant — this includes AI agents, media processors, or custom log…”
Deepgram Voice Agentnone0/10Deepgram Voice Agent is a hosted, cloud-only API accessed via WebSocket; nothing in the evidence indicates open-source runtime code that can be self-hosted on customer infrastructure. All references point to Deepgram-hosted endpoints (api.deepgram.com, api.eu.deepgram.com) and a CLI/SDK for calling the cloud service, not deploying the runtime itself.
- [claimed-docs] “Deepgram's Voice Agent API collapses that stack into a single, unified API. You open one WebSocket connection, send audio in, and receive au…”
- [claimed-docs] “For customers requiring data processing within the European Union, Deepgram provides an EU-specific endpoint at api.eu.deepgram.com.”
- [claimed-docs] “Twilio streams the call audio to your server, and your server bridges that audio to the Deepgram Voice Agent API, a single WebSocket that ru…”
Latency turntaking — stories about latency turntaking in this arenaLatency turntaking
Stories about latency turntaking in this arena
Latency
platform-engineerSee documented end-to-end voice latency numbers or tuning guidance backing the platform's speed claims
weight 3 · round drawnLiveKit Agentsnone0/10The evidence pack contains no documented end-to-end latency benchmarks or explicit tuning guidance for reducing turn-taking latency — only feature descriptions (turn detector, adaptive interruption handling) without numbers or configuration guidance. A community report explicitly states users could not measure TTFT or STT→LLM→TTS latency, reinforcing that this visibility/documentation is absent.
- [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response... to identify intentional interruptions (…”
- [claimed-docs] “LiveKit's `TurnDetector` is an audio model that encodes user audio directly, capturing both _what_ is said and _how_ it's said.”
- [claimed-docs] “A turn detector model can predict that they have more to say and wait for them to finish before responding.”
- [claimed-docs] “the model analyzes the acoustic signals to identify intentional interruptions (barge-ins) from conversational backchanneling”
- [community] “We've been running multiple voice AI agents on LiveKit and kept running into visibility issues — no way to measure TTFT, latency across STT …”
Deepgram Voice Agentnone0/10The evidence pack covers architecture, features (barge-in, telephony, function calling, multi-agent), and CLI/SDK tooling, but contains no documented latency numbers (e.g., ms end-to-end) or tuning guidance for reducing turn-taking latency. missing for 10: quantified latency benchmarks, latency tuning/configuration guidance, any performance dashboard or whitepaper on speed claims.
- [claimed-docs] “Deepgram's Voice Agent API collapses that stack into a single, unified API. You open one WebSocket connection, send audio in, and receive au…”
- [claimed-docs] “That includes barge-in, the ability to talk over the agent and have it stop instantly.”
- [claimed-docs] “Twilio streams the call audio to your server, and your server bridges that audio to the Deepgram Voice Agent API, a single WebSocket that ru…”
Turn taking
developerRely on the agent to handle interruptions (barge-in) gracefully — stopping speech, updating context, and recovering the turn
weight 3 · round to LiveKit AgentsLiveKit Agents has well-documented, purpose-built features for barge-in: adaptive interruption handling distinguishes intentional interruptions from backchanneling, and the TurnDetector model analyzes acoustic and semantic signals to manage turn-taking recovery. These are first-party docs with specific technical detail on the mechanism (stopping speech, updating context, recovering turn), and the runtime probes confirm the framework is real and installable. Missing for 10: independent/hands-on evidence specifically testing barge-in behavior in production (only community evidence is about latency observability, not interruption handling itself) and no third-party benchmark of interruption accuracy.
- [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response... to identify intentional interruptions (…”
- [claimed-docs] “the model analyzes the acoustic signals to identify intentional interruptions (barge-ins) from conversational backchanneling”
- [claimed-docs] “LiveKit's `TurnDetector` is an audio model that encodes user audio directly, capturing both _what_ is said and _how_ it's said.”
- [claimed-docs] “A turn detector model can predict that they have more to say and wait for them to finish before responding.”
- [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response.”
- [claimed-docs] “LiveKit's TurnDetector is an audio model that encodes user audio directly, capturing both what is said and how it's said.”
- [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”
Docs explicitly claim barge-in support where users can talk over the agent and it stops instantly, plus mid-call context tools like InjectAgentMessage for updating conversation state. However, evidence lacks detail on how turn recovery/context updating works technically after interruption, and there's no independent/hands-on corroboration of graceful recovery in practice. missing for 10: technical details on context-state recovery post-interruption, independent verification of barge-in reliability, edge-case handling (e.g., rapid interruptions, latency of stop).
- [claimed-docs] “That includes barge-in, the ability to talk over the agent and have it stop instantly.”
- [claimed-docs] “Inject agent message | Mid-call | InjectAgentMessage | Makes the agent speak a specific line”
- [claimed-docs] “Deepgram's Voice Agent API collapses that stack into a single, unified API. You open one WebSocket connection, send audio in, and receive au…”
developerEnable noise suppression or audio filtering so the agent stays coherent on noisy real-world calls
weight 1 · round drawnLiveKit Agentsnone0/10The evidence pack covers turn detection, interruption handling, telephony, MCP tools, and testing, but contains no mention of noise suppression or audio filtering plugins/features anywhere. Missing for 10: any docs or plugin reference to noise cancellation (e.g. Krisp/BVC), background noise filtering, or audio pre-processing configuration.
developerUse model-based end-of-turn detection beyond simple VAD silence timeouts so the agent doesn't talk over slow speakers
weight 2 · round to LiveKit AgentsLiveKit's TurnDetector is a dedicated audio+semantic model (not just VAD silence) that predicts whether a user has finished speaking or has more to say, waiting accordingly, and adaptive interruption handling further distinguishes real barge-ins from backchanneling — directly addressing not talking over slow speakers. This is documented in detail across multiple first-party doc pages plus GitHub feature lists. missing for 10: independent/hands-on benchmark or community validation of the turn-detector's real-world latency/accuracy impact.
- [claimed-docs] “LiveKit's `TurnDetector` is an audio model that encodes user audio directly, capturing both _what_ is said and _how_ it's said.”
- [claimed-docs] “A turn detector model can predict that they have more to say and wait for them to finish before responding.”
- [claimed-docs] “LiveKit's TurnDetector is an audio model that encodes user audio directly, capturing both what is said and how it's said.”
- [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response... to identify intentional interruptions (…”
- [claimed-docs] “the model analyzes the acoustic signals to identify intentional interruptions (barge-ins) from conversational backchanneling”
- [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response.”
Openness — open source, data portability, and self-hosting storiesOpenness
Open source, data portability, and self-hosting stories
ai-native userDo everything through the API that I can do in the UI
weight 2 · round drawnLiveKit Agents is API/SDK-first (Python/Node.js) with a full programmatic surface for building agents, tools, telephony, and testing, and there is also a no-code 'Agent Builder' UI for prototyping. However, the evidence doesn't confirm that everything achievable in that browser-based Agent Builder UI (or LiveKit Cloud dashboard features like deployment management, rollback) is equally exposed via API/CLI, so parity between UI and API is not fully demonstrated. missing for 10: explicit evidence that Agent Builder's no-code UI actions (and Cloud dashboard deployment/rollback controls) are all reachable via API/CLI, and confirmation of full CLI/API parity with dashboard features.
- [claimed-docs] “Prototype and deploy voice agents directly in your browser, without writing any code.”
- [claimed-docs] “LiveKit Agent Builder to prototype and deploy agents directly in your browser without writing code”
- [claimed-docs] “LiveKit Agent Builder: Prototype and deploy voice agents directly in your browser, without writing any code.”
- [claimed-docs] “Instant rollback to a previous agent deployment”
- [probe] “official CLI documented at https://github.com/livekit/livekit-cli”
- [probe] “PROBE runtime (recorded 2026-09-05): `lk --version` printed `lk version 2.18.6` after a plain `brew install livekit-cli` — the official CLI …”
- [claimed-docs] “A programmatic participant is any code that joins a LiveKit room as a participant — this includes AI agents, media processors, or custom log…”
Deepgram is fundamentally API-first — the Voice Agent is a WebSocket API with full configuration (prompting, function calling, multi-agent, telephony, reusable configs) exposed programmatically, and even the CLI/MCP server let AI-native users manage things from the terminal or via agentic tools rather than a GUI. However, evidence doesn't show a full-featured UI to compare against (e.g., a dashboard/console) nor confirm that every console feature (like A/B testing dashboards, analytics views) has a documented API equivalent. missing for 10: explicit mapping of console/UI-only features (analytics dashboards, A/B test result views) to API endpoints, and confirmation no UI-exclusive functionality exists.
- [claimed-docs] “Deepgram's Voice Agent API collapses that stack into a single, unified API. You open one WebSocket connection, send audio in, and receive au…”
- [claimed-docs] “Reusable Agent Configurations allow you to define and persist the agent block of your Settings message using the Deepgram API. Once created,…”
- [claimed-docs] “The dg CLI lets you transcribe files, stream live audio, synthesize speech, analyze text, and manage your Deepgram account from the terminal…”
- [claimed-docs] “The dg CLI includes a built-in MCP (Model Context Protocol) server that gives AI coding tools direct access to Deepgram APIs.”
- [claimed-docs] “A/B testing voices or prompts — Run two configurations in parallel and measure conversion, CSAT, or containment rate to pick a winner—no cod…”
- [probe] “PROBE openapi: HTTP 200 at https://developers.deepgram.com/openapi.json — contains "openapi" key”
- [probe] “official MCP server documented at https://developers.deepgram.com/developer-tools/agentic-tools”
- [probe] “official CLI documented at https://developers.deepgram.com/developer-tools/cli/getting-started”
ai-native userExport all of my data in open formats and leave
weight 3 · round drawnLiveKit Agentsnone0/10The evidence pack documents LiveKit Agents' open-source, self-hostable nature (Apache-2.0 server, pip/npm installable framework) but contains no explicit mention of data export tooling, open data formats for conversation/session logs, or a documented exit/migration path for user data. Self-hosting mitigates lock-in in principle, but that is not the same as a documented 'export all data' capability.
- [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
- [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”
- [claimed-docs] “The good news is LiveKit includes an embedded TURN server. It's a secure TURN implementation that has integrated authentication with the res…”
Deepgram Voice Agentnone0/10No evidence describes any data export, account data portability, or open-format export/deletion workflow letting a user take their data and leave; docs cover retention opt-out and EU processing but not export tooling. missing for 10: data export mechanism, open-format export documentation, account closure/data portability guarantees.
ai-native userRead the product's source under an open license
weight 2 · round to LiveKit AgentsLiveKit Agents is hosted at github.com/livekit/agents and documented as an open-source framework, with hands-on verification that the packages install and run without any account or key (pip/npm) and that the underlying server is Apache-2.0 OSS. The llms.txt docs also explicitly describe LiveKit as 'an open-source platform.' Missing for 10: an explicit license file/badge citation specifically for the livekit/agents repo (only inferred via probe evidence and sibling repo license).
- [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.livekit.io/llms.txt # LiveKit docs > LiveKit is an open-source platform for building voice, video,…”
- [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
- [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”
- [probe] “official CLI documented at https://github.com/livekit/livekit-cli”
Deepgram Voice Agentnone0/10Deepgram Voice Agent is a proprietary cloud API/SDK product; no evidence anywhere in the pack points to an open-source license or public source repository for the core Voice Agent service. Only API docs, CLI, and SDK usage are documented, none of which imply open-licensed source availability.
ai-native userSelf-host the core product
weight 3 · round to LiveKit AgentsLiveKit provides a documented, Apache-2.0 open-source self-hosting deployment guide including embedded TURN server and SSL setup, and this was hands-on verified: `livekit-server --dev` boots with no account or key, and the open-source livekit-agents framework installs and imports keylessly via pip/npm. This directly demonstrates the core product (server + agents framework) can be self-hosted. Missing for 10: independent third-party production self-hosting case study beyond the probe verification.
- [claimed-docs] “The good news is LiveKit includes an embedded TURN server. It's a secure TURN implementation that has integrated authentication with the res…”
- [claimed-docs] “In order to have a secure LiveKit deployment, you will need a domain as well as a SSL certificate for that domain.”
- [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
- [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”
- [probe] “PROBE runtime (recorded 2026-09-05): `lk --version` printed `lk version 2.18.6` after a plain `brew install livekit-cli` — the official CLI …”
Deepgram Voice Agentnone0/10Deepgram Voice Agent is a cloud API/SaaS product; all evidence points to hosted WebSocket endpoints, EU regional endpoints, and cloud-based configuration — there is no mention of a self-hostable or on-premises deployment package for the core voice agent model/inference stack. missing for 10: any self-hosting/on-prem deployment option, downloadable model weights or container, docs on running the core service outside Deepgram's cloud.
- [claimed-docs] “Deepgram's Voice Agent API collapses that stack into a single, unified API. You open one WebSocket connection, send audio in, and receive au…”
- [claimed-docs] “For customers requiring data processing within the European Union, Deepgram provides an EU-specific endpoint at api.eu.deepgram.com.”
Pricing plans — plan structure and value — what each tier costs and what it unlocksPricing plans
Plan structure and value — what each tier costs and what it unlocks
Pricing
founderSee published per-minute or usage pricing and estimate cost per call before committing
weight 2 · round drawnLiveKit Agentsnone0/10The pack references a pricing URL (livekit.io/pricing) only for an unrelated 'rollback' feature and never quotes actual per-minute or usage rates; a community comment even highlights that operators struggled with 'cost per call' visibility, but no evidence shows published pricing figures a founder could use to estimate costs before committing.
- [claimed-docs] “Instant rollback to a previous agent deployment”
- [community] “We've been running multiple voice AI agents on LiveKit and kept running into visibility issues — no way to measure TTFT, latency across STT …”
Privacy posture — data-handling and privacy storiesPrivacy posture
Data-handling and privacy stories
ai-native userChoose where my data is stored (region/residency)
weight 2 · round to Deepgram Voice AgentLiveKit Agents/server is open-source and self-hostable (embedded TURN server, deployment docs, hands-on verified self-host boot), which lets an operator choose their own hosting region/data location, but there is no explicit documented region-selection or data-residency feature for LiveKit Cloud itself. missing for 10: explicit LiveKit Cloud region-selection/data-residency controls, compliance certifications (e.g. GDPR/SOC2 region guarantees), and any customer-facing UI for choosing storage region.
- [claimed-docs] “The good news is LiveKit includes an embedded TURN server. It's a secure TURN implementation that has integrated authentication with the res…”
- [claimed-docs] “In order to have a secure LiveKit deployment, you will need a domain as well as a SSL certificate for that domain.”
- [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
Deepgram documents an EU-specific endpoint (api.eu.deepgram.com) for customers requiring EU data processing, and a per-request opt-out for model improvement data retention, showing some regional/data-handling control. However, this is limited to EU vs. default (US) with no evidence of broader region selection (e.g., APAC, other jurisdictions) or a general residency configuration option across the platform. Missing for 10: documented list of all available regions, self-serve region selection in dashboard/API config, and independent confirmation of residency guarantees beyond docs.
- [claimed-docs] “For customers requiring data processing within the European Union, Deepgram provides an EU-specific endpoint at api.eu.deepgram.com.”
- [claimed-docs] “Opt out per request with mip_opt_out=true. Opted-out requests are not retained”
ai-native userPrevent my data from being used to train AI models
weight 3 · round to Deepgram Voice AgentLiveKit Agentsnone0/10No evidence in the pack addresses data usage for AI model training, opt-out policies, or any privacy commitments regarding training data; the evidence pack is entirely about agent framework features (tools, telephony, testing, MCP support). This is a plausible axis for a platform handling user voice/media data, but absent any documentation on training-data usage or opt-out, it cannot be credited as delivered.
Deepgram documents a per-request opt-out (mip_opt_out=true) from its Model Improvement Program, with opted-out requests not retained, directly addressing training-data use. However, the evidence only shows a per-request flag rather than an account-wide default-off setting, and there's no independent verification that opt-out requests are truly excluded from training. Missing for 10: account/org-level opt-out default, third-party audit or independent confirmation of non-training use, and clarity on default behavior when the flag is omitted.
- [claimed-docs] “Opt out per request with mip_opt_out=true. Opted-out requests are not retained”
ai-native userControl data retention and deletion
weight 2 · round to Deepgram Voice AgentLiveKit Agentsnone0/10The evidence pack contains no mention of data retention policies, recording/session deletion controls, or configurable retention windows for LiveKit Agents. While self-hosting is documented (implying infrastructure control), no explicit retention/deletion feature or API is evidenced.
Deepgram documents a per-request opt-out (mip_opt_out=true) that prevents retention, and references data-privacy compliance including an EU-specific endpoint, giving users some retention control. However, there is no documented self-service deletion mechanism for already-retained data, no stated retention periods, and no dashboard/API for managing or purging stored voice data. Missing for 10: explicit data deletion API/console, documented retention duration policy, and confirmation of deletion for non-opted-out data.
- [claimed-docs] “Opt out per request with mip_opt_out=true. Opted-out requests are not retained”
- [claimed-docs] “For customers requiring data processing within the European Union, Deepgram provides an EU-specific endpoint at api.eu.deepgram.com.”
ai-native userOpt out of telemetry and usage tracking
weight 2 · round to Deepgram Voice AgentLiveKit Agentsnone0/10No evidence pack item discusses telemetry, usage tracking, analytics collection, or an opt-out mechanism for LiveKit Agents; the framework being open-source and self-hostable is not sufficient evidence of an explicit telemetry opt-out control.
Deepgram documents an opt-out mechanism (mip_opt_out=true) for model improvement/data retention on a per-request basis, which is a form of telemetry/usage-data opt-out, but this is narrowly scoped to training data retention rather than general telemetry/usage tracking (e.g., analytics, product usage metrics). Missing for 10: documentation of a broader telemetry/usage-tracking opt-out setting, confirmation this covers all usage data beyond model-improvement retention, and independent verification that opting out has no side effects.
- [claimed-docs] “Opt out per request with mip_opt_out=true. Opted-out requests are not retained”
Telephony — stories about telephony in this arenaTelephony
Stories about telephony in this arena
Call control
developerEscalate a live call to a human with warm or blind transfer, passing context along
weight 2 · round drawnLiveKit Agentsnone0/10LiveKit Agents documents inbound/outbound telephony (SIP trunks, phone calls) but the evidence pack contains no mention of call transfer (warm or blind) or passing conversational context to a human agent during a live call — this specific escalation capability is never described.
- [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
- [claimed-docs] “Enable your voice agent to make or take phone calls.”
- [claimed-docs] “Outbound trunks are used to place outgoing calls.”
- [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”
Deepgram Voice Agentnone0/10Evidence covers telephony connectivity, function calling, multi-agent handoff between AI agents, and mid-call message injection, but nothing documents a warm/blind transfer to a human agent (e.g., SIP REFER, call transfer function, or context handoff to a live operator).
developerMy agent can send DTMF keypresses, navigate IVR menus, and detect or leave voicemail
weight 1 · round drawnLiveKit Agentsnone0/10Evidence confirms LiveKit's telephony/SIP stack supports inbound/outbound calls (docs-26, docs-31, docs-32, gh-2), but nothing in the pack mentions DTMF keypress sending, IVR menu navigation, or voicemail detection/leaving capabilities specifically. Missing for 10: any documentation or example of DTMF tone generation, automated IVR menu traversal, or voicemail detection/answering-machine-detection logic.
- [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
- [claimed-docs] “Enable your voice agent to make or take phone calls.”
- [claimed-docs] “Outbound trunks are used to place outgoing calls.”
- [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”
Campaigns
founderRun batch outbound call campaigns with scheduling and throughput controls
weight 2 · round drawnLiveKit Agentsnone0/10Evidence shows LiveKit Agents can place individual outbound calls via SIP outbound trunks (docs-26, docs-32, docs-31), but there is no evidence of any batch campaign management, call scheduling, or throughput/rate-limiting controls for running outbound calling campaigns at scale — these features are simply not documented anywhere in the pack.
- [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
- [claimed-docs] “Outbound trunks are used to place outgoing calls.”
- [claimed-docs] “Enable your voice agent to make or take phone calls.”
- [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”
Deepgram Voice Agentnone0/10Deepgram's docs confirm outbound telephony connectivity (docs-5, docs-18) and per-call features like barge-in and prompting, but there is no mention of campaign-level batch dialing, call scheduling, or throughput/rate controls for outbound campaigns — this is a real-time conversation API, not a dialer/campaign manager.
- [claimed-docs] “Telephony: connect voice agents to phone networks for inbound and outbound calls.”
- [claimed-docs] “Twilio streams the call audio to your server, and your server bridges that audio to the Deepgram Voice Agent API, a single WebSocket that ru…”
Numbers
developerProvision phone numbers and run both inbound and outbound calls through the platform's API
weight 3 · round drawnLiveKit's SIP/telephony stack clearly supports inbound and outbound calling ('LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls', 'Outbound trunks are used to place outgoing calls', 'Enable your voice agent to make or take phone calls') and integrates with the agents runtime via SIP trunks. However, the evidence never shows LiveKit itself provisioning phone numbers through its own API — telephony typically requires configuring an external SIP trunk provider, and no docs here describe a native number-provisioning endpoint. Missing for 10: explicit documentation/evidence of a LiveKit API call that provisions/purchases phone numbers directly (rather than just configuring trunks against externally-acquired numbers), and independent/hands-on confirmation of the inbound+outbound call flow.
- [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
- [claimed-docs] “Enable your voice agent to make or take phone calls.”
- [claimed-docs] “Outbound trunks are used to place outgoing calls.”
- [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”
Docs confirm Deepgram Voice Agent supports inbound/outbound calls and telephony integration (e.g., via Twilio bridging audio to a single WebSocket), but there is no evidence Deepgram itself provisions or manages phone numbers—developers must bring their own Twilio account/number and build a bridging server. Missing for 10: native phone-number provisioning API, first-party telephony number management, and evidence of outbound call initiation directly through Deepgram's API without a third-party carrier.
- [claimed-docs] “Telephony: connect voice agents to phone networks for inbound and outbound calls.”
- [claimed-docs] “Twilio streams the call audio to your server, and your server bridges that audio to the Deepgram Voice Agent API, a single WebSocket that ru…”
- [claimed-docs] “That includes barge-in, the ability to talk over the agent and have it stop instantly.”
Sip
platform-engineerConnect my own carrier or PBX via SIP trunking (or import Twilio/Telnyx numbers) instead of being locked to bundled telephony
weight 2 · round to LiveKit AgentsLiveKit's SIP-based telephony stack lets agents make/receive calls via inbound and outbound SIP trunks, which is the standard mechanism for bringing your own carrier or PBX (docs-26, docs-32, gh-2, docs-31). This is architecture-agnostic SIP, not a bundled/proprietary telephony lock-in, satisfying the platform-engineer's need to connect external trunks. Missing for 10: explicit documented walkthroughs or examples of importing Twilio/Telnyx numbers specifically, and no independent/hands-on confirmation of a real carrier trunk connection succeeding.
- [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
- [claimed-docs] “Outbound trunks are used to place outgoing calls.”
- [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”
- [claimed-docs] “Enable your voice agent to make or take phone calls.”
Deepgram Voice Agentnone0/10Evidence only documents a generic WebSocket bridge pattern for Twilio (docs-18, docs-8) and a general 'Telephony' feature bullet (docs-5), but there is no mention of SIP trunking, PBX connectivity, or Telnyx number import — the specific mechanisms a platform engineer would need to bring their own carrier.
- [claimed-docs] “Telephony: connect voice agents to phone networks for inbound and outbound calls.”
- [claimed-docs] “Twilio streams the call audio to your server, and your server bridges that audio to the Deepgram Voice Agent API, a single WebSocket that ru…”
- [claimed-docs] “That includes barge-in, the ability to talk over the agent and have it stop instantly.”
Testing analytics — stories about testing analytics in this arenaTesting analytics
Stories about testing analytics in this arena
Analytics
ai-native userThe platform's AI reviews my calls for me — scoring quality, flagging failures, and analyzing resolution automatically
weight 2 · round to LiveKit AgentsLiveKit Agentsdisputedcontradicted3/10LiveKit ships a dev-time testing framework with 'judges' and agent simulations to evaluate scripted interactions (docs-9/10/25, gh-4), but this is pre-deployment test tooling, not automatic scoring/flagging/resolution-analysis of live production calls. A hands-on community report explicitly says operators running real agents on LiveKit have no built-in way to measure latency or cost per call, let alone automated quality/resolution review, contradicting the idea that the platform reviews calls for you. Missing for 10: production call-level QA scoring, automatic failure flagging on real traffic, resolution/outcome analysis dashboards.
- [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
- [claimed-docs] “Agent simulations run end-to-end conversations between your agent and an LLM-driven user, then evaluate the results across the full interact…”
- [github] “Builtin test framework: Write tests and use judges to ensure your agent is performing as expected.”
- [community] “We've been running multiple voice AI agents on LiveKit and kept running into visibility issues — no way to measure TTFT, latency across STT …”
Deepgram Voice Agentnone0/10The evidence pack covers Deepgram Voice Agent's real-time conversation infrastructure (STT/TTS, function calling, telephony, prompting, A/B testing of voices/prompts) but contains no mention of post-call AI review, automated quality scoring, failure flagging, or resolution analysis of completed calls.
founderSee call analytics — success rates, durations, outcomes, sentiment — in dashboards without building my own
weight 2 · round drawnLiveKit Agentsnone0/10The evidence pack shows testing/simulation tools (pytest/Vitest, LLM-driven simulations) but no built-in production analytics dashboard for success rates, durations, outcomes, or sentiment. A community report explicitly confirms this gap — users running LiveKit voice agents said they had 'no way to measure TTFT, latency across STT→LLM→TTS, or even cost per call' and found debugging painful, indicating no such dashboard exists out of the box.
- [community] “We've been running multiple voice AI agents on LiveKit and kept running into visibility issues — no way to measure TTFT, latency across STT …”
- [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
- [claimed-docs] “Agent simulations run end-to-end conversations between your agent and an LLM-driven user, then evaluate the results across the full interact…”
- [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes... Agent simulations run end-to-end conversations between your agent and…”
Deepgram Voice Agentnone0/10Deepgram is a developer API/infrastructure product; evidence shows mentions of A/B testing metrics (conversion, CSAT, containment rate) as data points but no evidence of an actual built-in dashboard for call analytics, success rates, durations, sentiment, or outcomes that founders can view without building their own.
- [claimed-docs] “A/B testing voices or prompts — Run two configurations in parallel and measure conversion, CSAT, or containment rate to pick a winner—no cod…”
Monitoring
platform-engineerMonitor live calls in production and get alerts when agents misbehave or error rates spike
weight 1 · round drawnLiveKit Agentsnone0/10No evidence pack items describe production monitoring dashboards, error-rate alerting, or call-quality observability tooling for LiveKit Agents; the only related evidence is a community report explicitly describing the lack of visibility into TTFT, per-call latency, and cost — a gap, not a delivered capability.
- [community] “We've been running multiple voice AI agents on LiveKit and kept running into visibility issues — no way to measure TTFT, latency across STT …”
Testing
developerTest agents with simulated conversations or evals before putting them on real phone calls
weight 2 · round to LiveKit AgentsDocs explicitly describe both behavioral tests (pytest/Vitest) and agent simulations that run end-to-end conversations with an LLM-driven user, evaluating results before deployment, and GitHub notes a 'builtin test framework' with judges — directly matching the testing-before-real-calls story, complemented by separate telephony/SIP support for real calls. Missing for 10: independent/hands-on validation of the eval/simulation framework and explicit documentation tying test simulations to pre-phone-call validation workflows.
- [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
- [claimed-docs] “Agent simulations run end-to-end conversations between your agent and an LLM-driven user, then evaluate the results across the full interact…”
- [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes... Agent simulations run end-to-end conversations between your agent and…”
- [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes... using pytest (Python) or Vitest (Node.js)”
- [github] “Builtin test framework: Write tests and use judges to ensure your agent is performing as expected.”
- [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”
- [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
Deepgram Voice Agentnone0/10The evidence covers live-call features (function calling, prompting, telephony, A/B testing of live configs) but nothing describes a simulated-conversation or eval framework for pre-production testing of agents before real phone calls. Missing for 10: any documentation of a test/sandbox mode, conversation simulation tool, or eval harness for agents.
- [claimed-docs] “A/B testing voices or prompts — Run two configurations in parallel and measure conversion, CSAT, or containment rate to pick a winner—no cod…”
- [claimed-docs] “Reusable Agent Configurations allow you to define and persist the agent block of your Settings message using the Deepgram API. Once created,…”
Tools function calling — stories about tools function calling in this arenaTools function calling
Stories about tools function calling in this arena
Post call
developerExtract structured data from every call — outcomes, entities, dispositions — delivered via API or webhook after the call
weight 2 · round drawnLiveKit Agentsnone0/10LiveKit Agents provides tool-calling, MCP integration, and telephony hooks, but nothing in the evidence describes a built-in mechanism for automatically extracting structured call outcomes/entities/dispositions and delivering them via API or webhook after a call ends — that would have to be custom-built by the developer using the tool-calling primitives. Since no such capability is documented, this applicable axis is unmet.
- [claimed-docs] “Call external APIs or lookup data for RAG.”
- [claimed-docs] “LiveKit Agents has full support for LLM tool use. This feature allows you to create a custom library of tools to extend your agent's context”
- [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
- [claimed-docs] “Outbound trunks are used to place outgoing calls.”
Deepgram Voice Agentnone0/10The evidence pack covers live function calling during a call and various agent configuration/telephony features, but there is no mention of post-call structured data extraction (summaries, entities, dispositions) or webhook delivery of such analytics after a call ends.
Tools
ai-native userMy voice agent can plug in MCP servers as tool sources so one integration grants it whole toolsets mid-call
weight 2 · round to LiveKit AgentsLiveKit Agents has first-class, documented MCP support ("first-class support for Model Context Protocol (MCP) servers", "Integrate tools provided by MCP servers with one line of code") via `MCPToolset`, which wraps an MCP server and passes it directly to the agent's tools parameter — enabling mid-call toolset access for voice agents. This is corroborated across both docs and GitHub README. Missing for 10: independent hands-on validation of actual mid-call MCP toolset usage beyond vendor docs.
- [claimed-docs] “Wrap an MCP server in an \`MCPToolset\` and pass it to the agent's \`tools\` parameter”
- [claimed-docs] “Wrap an MCP server in an `MCPToolset` and pass it to the agent's `tools` parameter”
- [claimed-docs] “LiveKit Agents has first-class support for Model Context Protocol (MCP) servers.”
- [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
- [github] “Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
Deepgram Voice Agentnone0/10Evidence confirms mid-call function calling exists for the Voice Agent (deepgram-docs-2) but only documents an MCP *server* built into the CLI that lets coding tools access Deepgram APIs (deepgram-docs-13, deepgram-probe-4) — this is the reverse direction, not the Voice Agent acting as an MCP client to pull in external toolsets mid-call. No evidence shows the Voice Agent itself can connect to MCP servers as a tool source.
- [claimed-docs] “In the context of Deepgram Voice Agents, function calling enables your agent to perform tasks during a live conversation.”
- [claimed-docs] “The dg CLI includes a built-in MCP (Model Context Protocol) server that gives AI coding tools direct access to Deepgram APIs.”
- [probe] “official MCP server documented at https://developers.deepgram.com/developer-tools/agentic-tools”
developerMy agent can call external APIs and custom functions mid-conversation and speak the result without awkward dead air
weight 3 · round to LiveKit AgentsLiveKit Agents has full documented support for LLM tool/function calling (custom Python/Node functions calling external APIs mid-conversation) plus MCP server tool integration, combined with adaptive interruption handling and turn-detection to avoid awkward dead air while tools execute. Runtime probes confirm the framework is real and installable, corroborating the docs claims. missing for 10: independent/hands-on evidence specifically showing tool-call latency handled gracefully in a live conversation (only vendor docs cover this exact combination).
- [claimed-docs] “Call external APIs or lookup data for RAG.”
- [claimed-docs] “LiveKit Agents has full support for LLM tool use. This feature allows you to create a custom library of tools to extend your agent's context”
- [claimed-docs] “LiveKit Agents has full support for LLM tool use.”
- [claimed-docs] “LiveKit Agents has first-class support for Model Context Protocol (MCP) servers.”
- [claimed-docs] “Wrap an MCP server in an \`MCPToolset\` and pass it to the agent's \`tools\` parameter”
- [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response... to identify intentional interruptions (…”
- [claimed-docs] “the model analyzes the acoustic signals to identify intentional interruptions (barge-ins) from conversational backchanneling”
- [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”
Docs confirm function calling is a supported feature for performing tasks mid-conversation (deepgram-docs-2), and the agent architecture is a single low-latency WebSocket for audio in/out (deepgram-docs-1) with barge-in and inject-message features (deepgram-docs-8, deepgram-docs-11) suggesting attention to latency/dead-air. However there's no explicit documentation or example describing how function-call latency is masked (e.g., filler speech, streaming partial results) while awaiting an API response. Missing for 10: concrete guidance/examples on avoiding dead air during function execution, sample code showing async function calls with the agent speaking a holding phrase, and independent/hands-on verification of smooth latency handling.
- [claimed-docs] “In the context of Deepgram Voice Agents, function calling enables your agent to perform tasks during a live conversation.”
- [claimed-docs] “Deepgram's Voice Agent API collapses that stack into a single, unified API. You open one WebSocket connection, send audio in, and receive au…”
- [claimed-docs] “That includes barge-in, the ability to talk over the agent and have it stop instantly.”
- [claimed-docs] “Inject agent message | Mid-call | InjectAgentMessage | Makes the agent speak a specific line”
Transcription recording — stories about transcription recording in this arenaTranscription recording
Stories about transcription recording in this arena
Recording
platform-engineerRetrieve full call recordings and transcripts programmatically for every call
weight 2 · round drawnLiveKit Agentsnone0/10The evidence pack covers agent building, tool use, MCP, telephony, and testing but contains no mention of recording APIs, transcript storage, or programmatic retrieval of call recordings/transcripts for platform engineers.
Deepgram Voice Agentnone0/10The evidence pack covers Voice Agent architecture, function calling, telephony bridging, LLM providers, and data privacy opt-outs, but nowhere describes an API or mechanism to retrieve full call recordings or persisted transcripts after a call ends. No endpoint, storage feature, or retrieval workflow is documented for this specific capability.
Transcription
developerGet accurate real-time transcription with control over the STT provider, language models, or key terms
weight 2 · round drawnLiveKit Agents supports swapping STT/LLM providers via its open-source plugin ecosystem and lets you override API key/base URL for any provider (docs-12, docs-24, docs-35), giving real control over transcription and language-model choice. However there is no evidence of support for custom vocabulary/key-term or hotword boosting in STT, and no independent benchmark of transcription accuracy. Missing for 10: key-term/hotword/phrase-hint configuration support, accuracy benchmarks or independent corroboration of transcription quality.
- [claimed-docs] “LiveKit Agents includes a large ecosystem of open source plugins for a variety of AI providers.”
- [claimed-docs] “You can use LiveKit Inference to access many of these models directly through LiveKit Cloud, or you can use the open source plugins to conne…”
- [claimed-docs] “For any provider not included, you can override the API key and base URL at initialization for the LLM, STT, and TTS interfaces in the plugi…”
Docs confirm real-time transcription via the unified WebSocket API, selectable LLM providers (docs-9), and multilingual STT/TTS model choices (docs-10), giving developers meaningful control over STT and LLM models. However, there is no evidence of 'key terms' or keyword-boosting controls specific to the Voice Agent's transcription pipeline. Missing for 10: explicit key-term/keyword-boost configuration support, independent benchmarking of transcription accuracy.
- [claimed-docs] “Deepgram's Voice Agent API collapses that stack into a single, unified API. You open one WebSocket connection, send audio in, and receive au…”
- [claimed-docs] “Supported LLM providers | Parameter | open_ai | anthropic | aws_bedrock | google | groq | nvidia”
- [claimed-docs] “A multilingual voice agent has two model decisions: which STT model transcribes the user, and which TTS model speaks the agent.”
Voices tts — stories about voices tts in this arenaVoices tts
Stories about voices tts in this arena
Voices
founderClone a custom brand voice and use it for my agents, with a documented consent process
weight 2 · round drawnLiveKit Agentsnone0/10The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)
developerChoose from a broad voice library or plug in multiple TTS providers to get the voice I want
weight 2 · round to LiveKit AgentsDocs confirm a large open-source plugin ecosystem for TTS/STT/LLM providers, plus LiveKit Inference for provider access, and an override mechanism for API key/base URL for providers not natively included, supporting multi-provider TTS flexibility. However, there's no explicit mention of a 'voice library' (e.g., curated voice catalog/selection UI) or enumeration of specific TTS providers/voices, so the 'broad voice library' half of the story is unevidenced. Missing for 10: explicit voice catalog/list of supported TTS providers and voices, evidence of ease of switching between TTS voices, independent confirmation of provider breadth.
- [claimed-docs] “LiveKit Agents includes a large ecosystem of open source plugins for a variety of AI providers.”
- [claimed-docs] “You can use LiveKit Inference to access many of these models directly through LiveKit Cloud, or you can use the open source plugins to conne…”
- [claimed-docs] “For any provider not included, you can override the API key and base URL at initialization for the LLM, STT, and TTS interfaces in the plugi…”
Deepgram Voice Agentnone0/10Evidence confirms TTS model selection exists (docs-10 references choosing a TTS model) but there is no documentation of a broad voice library or of plugging in multiple third-party TTS providers analogous to the multi-LLM-provider list (docs-9). Missing for 10: evidence of voice catalog/library breadth, evidence of multiple supported TTS providers/vendors, and any provider-switching mechanism for voice output.
- [claimed-docs] “A multilingual voice agent has two model decisions: which STT model transcribes the user, and which TTS model speaks the agent.”