LiveKit Agents vs telli

Agent ops

1. ai-native userMy coding agent can provision a complete voice agent end to end — create the agent, attach a number, and place a call — through the API, CLI, or MCP without touching the dashboard

   weight 3 · round drawn
   LiveKit Agents~partialprobed5/10

   LiveKit provides a real CLI (`lk`), a documented telephony/SIP stack for provisioning inbound/outbound trunks and placing calls, and framework docs explicitly note it's 'built for coding agents like Claude Code, Cursor, and Codex,' suggesting API/CLI-driven workflows are possible without the dashboard. However, the MCP support documented is for the agent's own tool-calling (consuming MCP servers as tools), not an MCP server exposing agent-provisioning/number-attachment/call-placing actions to a coding agent, and no single evidenced workflow shows an agent programmatically creating an agent, attaching a number, and placing a call end-to-end via CLI/API in one flow — the Agent Builder path shown is dashboard/browser-based (no-code), which contradicts the 'without touching dashboard' framing. Missing for 10: an MCP server (or CLI/API recipe) that lets a coding agent itself create an agent, provision/attach a phone number, and place a call in one documented end-to-end sequence.

   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”
   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [claimed-docs] “Outbound trunks are used to place outgoing calls.”
   • [probe] “official CLI documented at https://github.com/livekit/livekit-cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): `lk --version` printed `lk version 2.18.6` after a plain `brew install livekit-cli` — the official CLI …”
   • [claimed-docs] “LiveKit is built for coding agents like Claude Code, Cursor, and Codex.”
   • [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
   • [claimed-docs] “LiveKit Agents has first-class support for Model Context Protocol (MCP) servers.”

   proof ↗⚑ flag

   telli~partialprobed5/10

   telli's REST API documents endpoints for scheduling calls and importing/attaching phone numbers (telli-docs-17, telli-docs-19), and the MCP server lets AI tools talk to Charlie, who can 'build and refine agents' (telli-docs-11, telli-docs-22), suggesting agent creation is possible without the dashboard. However, there is no explicit API/CLI 'create agent' endpoint documented, no CLI tool is mentioned at all, and the MCP-driven agent creation is only described as an assistant chat capability rather than a deterministic API call — so a full end-to-end create→attach-number→place-call pipeline via one consistent interface is not clearly evidenced. Missing for 10: a documented create-agent API/CLI endpoint, confirmation MCP/Charlie can fully provision agents without any dashboard step, and an example combining all three steps in one flow.

   • [claimed-docs] “Charlie: Build and refine agents, automate workflows, and analyze your calls with telli's built-in AI assistant.”
   • [claimed-docs] “API Endpoints: Use telli's REST API to programmatically manage contacts, schedule calls, and integrate with your existing systems.”
   • [claimed-docs] “Schedules a call at the earliest opportunity within the dialer window. May occur immediately or on the next business day depending on settin…”
   • [claimed-docs] “Imports a phone number from your own SIP trunk provider. This allows you to use existing phone numbers with telli by configuring custom SIP …”
   • [claimed-docs] “The telli MCP server lets AI tools like Claude, ChatGPT, and Codex talk to Charlie, telli's AI assistant, over the Model Context Protocol.”
   • [probe] “PROBE openapi: HTTP 200 at https://docs.telli.com/openapi.json — contains "openapi" key”
   • [probe] “official MCP server documented at https://docs.telli.com/integrations/mcp”

   proof ↗⚑ flag

2. ai-native userThe platform's own AI helps me author agents — generating or improving prompts, flows, and test cases from a description

   weight 1 · round to telli
   LiveKit Agents—none0/10

   Evidence shows a no-code 'Agent Builder' for browser prototyping and a testing framework, but nothing indicates the platform itself uses AI to generate or improve prompts, flows, or test cases from a natural-language description of the desired agent.

   • [claimed-docs] “LiveKit Agent Builder to prototype and deploy agents directly in your browser without writing code”
   • [claimed-docs] “LiveKit Agent Builder: Prototype and deploy voice agents directly in your browser, without writing any code.”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes... Agent simulations run end-to-end conversations between your agent and…”

   proof ↗⚑ flag

   telli~partialprobed5/10

   Charlie, telli's built-in AI assistant, is documented as helping to 'build and refine agents, automate workflows, and analyze your calls,' which supports the core idea of AI-assisted agent authoring, and the MCP server lets external AI tools interact with Charlie too. However there's no concrete evidence of generating test cases from a description, or detailed examples of prompt/flow generation output. missing for 10: documented examples of Charlie generating/improving prompts or flows from a natural-language description, evidence of AI-generated test case creation, and independent/hands-on confirmation of these capabilities in practice.

   • [claimed-docs] “Charlie: Build and refine agents, automate workflows, and analyze your calls with telli's built-in AI assistant.”
   • [claimed-docs] “The telli MCP server lets AI tools like Claude, ChatGPT, and Codex talk to Charlie, telli's AI assistant, over the Model Context Protocol.”
   • [probe] “official MCP server documented at https://docs.telli.com/integrations/mcp”

   proof ↗⚑ flag

Build

1. developerBuild a working phone voice agent — prompt, voice, and phone number — and take my first live call within an hour

   weight 3 · round to LiveKit Agents
   LiveKit Agents✓fullprobed7/10

   Docs show a <10‑minute voice-assistant quickstart (prompt+voice) plus a dedicated telephony/SIP stack for inbound/outbound calls (trunks, phone number), and runtime probes confirm the framework and self-hosted server actually install and boot without extra keys. Together these cover prompt, voice, and phone number needed for a first live call within an hour. Missing for 10: a single unified, hands-on-verified tutorial that walks through phone-number provisioning and first live call end-to-end (currently voice-quickstart and telephony docs are separate), and independent (non-vendor) confirmation of the 'within an hour' timeline.

   • [claimed-docs] “Build and deploy a simple voice assistant in less than 10 minutes.”
   • [claimed-docs] “Build and deploy a simple voice assistant with Python or Node.js in less than 10 minutes.”
   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [claimed-docs] “Enable your voice agent to make or take phone calls.”
   • [claimed-docs] “Outbound trunks are used to place outgoing calls.”
   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `lk --version` printed `lk version 2.18.6` after a plain `brew install livekit-cli` — the official CLI …”
   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”

   proof ↗⚑ flag

   telli~partialclaimed6/10

   telli provides agent builder, test call feature, voice cloning, and phone number setup (forwarding or SIP trunk import) which together support building and testing a voice agent with prompt, voice, and number, but there's no explicit walkthrough or evidence confirming a full end-to-end setup can be completed within an hour or that phone number provisioning is instant/self-serve for new numbers. missing for 10: a documented quick-start/time-to-first-call benchmark, evidence of instant/native number provisioning (vs SIP/forwarding setup), and independent/hands-on confirmation of the full flow's speed.

   • [claimed-docs] “In the agent builder, click Test call to open the dialog. Pick the agent version (your draft with unsaved changes or any saved revision)...”
   • [claimed-docs] “Use the side-by-side view to listen to the recording while following along in the transcript.”
   • [claimed-docs] “Voice cloning lets you give your agent a custom voice (your own, a colleague's, or a professional voice actor's), so the agent sounds exactl…”
   • [claimed-docs] “Keep your public business number and forward incoming calls to a telli number. You configure the forwarding with your phone provider or PBX.”
   • [claimed-docs] “Connect an existing number and phone system for incoming and outgoing calls. You need SIP trunk access from your provider.”
   • [claimed-docs] “Imports a phone number from your own SIP trunk provider. This allows you to use existing phone numbers with telli by configuring custom SIP …”

   proof ↗⚑ flag

2. developerRun conversations in multiple languages, including detecting and switching language mid-call

   weight 2 · round drawn
   LiveKit Agents—none0/10

   No evidence in the pack addresses multilingual conversation support, language detection, or dynamic language switching mid-call; the docs cover turn detection, interruption handling, tool use, MCP, and telephony but never mention STT/TTS language selection or switching logic. This is a fair axis for a voice-agent framework (many STT/TTS providers support multi-language), so absence of evidence means 'none' rather than 'na'.

   ⚑ flag

   telli—none0/10

   No evidence in the pack mentions multilingual support, language detection, or mid-call language switching for telli's voice agents; all documented features concern scheduling, webhooks, tools, analytics, and MCP integration.

   ⚑ flag

3. founderDesign multi-step conversation flows in a visual builder with branching, states, and handoffs without writing code

   weight 2 · round to telli
   LiveKit Agents~partialclaimed3/10

   LiveKit mentions a browser-based 'Agent Builder' for no-code prototyping of voice agents, but the evidence never describes visual branching, explicit states, or handoff design — the core framework is fundamentally code-first (Python/Node.js) with tools, MCP, and turn-detection logic. missing for 10: documentation of branching/state UI, handoff modeling, or any screenshots/examples of the Agent Builder's flow-design capabilities beyond a vague no-code prototyping claim.

   • [claimed-docs] “Prototype and deploy voice agents directly in your browser, without writing any code.”
   • [claimed-docs] “LiveKit Agent Builder to prototype and deploy agents directly in your browser without writing code”
   • [claimed-docs] “LiveKit Agent Builder: Prototype and deploy voice agents directly in your browser, without writing any code.”

   proof ↗⚑ flag

   telli~partialclaimed4/10

   telli has an 'agent builder' with test-call and versioning, and workflow triggers (scheduled, SMS/WhatsApp) suggesting some no-code workflow configuration, plus tool-calls like transfer to another agent implying handoffs. But there's no evidence of a visual flow builder with branching/state diagrams — evidence only shows agent settings, tools, and triggers, not a drag-and-drop conversation flow canvas. missing for 10: explicit visual builder UI with branching/state nodes, evidence of designing multi-step flows with conditional paths, and documentation of handoff logic beyond simple 'transfer tool'.

   • [claimed-docs] “In the agent builder, click Test call to open the dialog. Pick the agent version (your draft with unsaved changes or any saved revision)...”
   • [claimed-docs] “Transfer tools | Move the call to another agent or number.”
   • [claimed-docs] “Choose SMS and WhatsApp workflow recipients from contact properties, collected values, or a fixed number”
   • [claimed-docs] “Workflows can now run automatically on a recurring schedule. Add a Scheduled trigger, choose the interval and start time, and let the workfl…”

   proof ↗⚑ flag

Personalization

1. developerInject dynamic variables and per-caller context at call time so each conversation is personalized

   weight 2 · round drawn
   LiveKit Agents—none0/10

   The evidence pack covers tool use, MCP, telephony, testing, and turn detection, but nothing addresses injecting dynamic variables or per-caller context (e.g., participant/room metadata, job context) into an agent's prompt or session at call start. Missing for 10: any documentation of job/participant metadata APIs, per-call context injection, or dynamic prompt personalization.

   ⚑ flag

   telli—none0/10

   The evidence describes contact properties and collected values being used to select SMS/WhatsApp recipients (telli-docs-5) and APIs to manage contacts and schedule calls (telli-docs-12, telli-docs-17), but nowhere does it document injecting dynamic variables or per-caller context into an agent's voice call script/persona at call time. Missing for 10: explicit documentation of variable placeholders in agent prompts, a mechanism for passing per-call custom data via the schedule-call API, and any example of personalized call content driven by caller-specific fields.

   • [claimed-docs] “Choose SMS and WhatsApp workflow recipients from contact properties, collected values, or a fixed number”
   • [claimed-docs] “API Endpoints: Use telli's REST API to programmatically manage contacts, schedule calls, and integrate with your existing systems.”
   • [claimed-docs] “Schedules a call at the earliest opportunity within the dialer window. May occur immediately or on the next business day depending on settin…”

   proof ↗⚑ flag

2. developerGround the agent on my documents with a built-in knowledge base or RAG so it answers from my content

   weight 2 · round to LiveKit Agents
   LiveKit Agents~partialclaimed4/10

   LiveKit Agents supports calling external APIs/tools that a developer can use to implement RAG ("Call external APIs or lookup data for RAG"), and has full LLM tool-use and MCP integration for wiring in retrieval systems, but there is no built-in knowledge base, document ingestion, or vector-store/RAG pipeline shipped by the framework itself — developers must bring their own RAG implementation via the tools API. Missing for 10: built-in document indexing/vector store, out-of-the-box knowledge-base feature, first-party RAG pipeline or example showing document grounding end-to-end.

   • [claimed-docs] “Call external APIs or lookup data for RAG.”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use. This feature allows you to create a custom library of tools to extend your agent's context”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use.”
   • [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
   • [claimed-docs] “Wrap an MCP server in an \`MCPToolset\` and pass it to the agent's \`tools\` parameter”

   proof ↗⚑ flag

   telli—none0/10

   No evidence of a built-in knowledge base or RAG feature for grounding agents on uploaded documents; the docs describe custom tools for API integration and a web_search tool for public information, but nothing about ingesting documents or content for retrieval-based answers.

   ⚑ flag

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to telli
   LiveKit Agents✓fullprobed8/10

   LiveKit publishes a working llms.txt (verified live at docs.livekit.io/llms.txt, HTTP 200) and agent-friendly markdown doc endpoints (e.g. /agents/.md), plus explicit docs noting the platform is 'built for coding agents like Claude Code, Cursor, and Codex,' confirming intentional support for AI-native doc consumption. missing for 10: no independent third-party confirmation that external agents successfully consume these endpoints in practice, and no dedicated documentation page explaining the llms.txt/agent-doc strategy itself.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.livekit.io/llms.txt # LiveKit docs > LiveKit is an open-source platform for building voice, video,…”
   • [probe] “PROBE docs-md: HTTP 200 at https://docs.livekit.io/agents/.md LiveKit docs › Build Agents › Get Started › Introduction --- # Introduction …”
   • [claimed-docs] “LiveKit is built for coding agents like Claude Code, Cursor, and Codex.”

   proof ↗⚑ flag

   telli✓fullprobed9/10

   telli hosts a live llms.txt file (confirmed via probe, HTTP 200) plus machine-readable OpenAPI spec explicitly intended for coding agents, and docs also mention agent-oriented context for downloading API specs. missing for 10: no independent/third-party confirmation of an agent successfully consuming llms.txt in practice.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.telli.com/llms.txt # telli Documentation > telli is the AI voice platform for agents that sell, su…”
   • [claimed-docs] “Download them to generate client libraries, import the API into tools like Postman, or give coding agents machine-readable API context”
   • [probe] “PROBE openapi: HTTP 200 at https://docs.telli.com/openapi.json — contains "openapi" key”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to LiveKit Agents
   LiveKit Agents✓fullprobed8/10

   LiveKit Agents is a Python/Node.js SDK plus self-hostable server that runs as a headless programmatic participant with no UI dependency; it's pip/npm-installable, has a built-in test framework (pytest/Vitest, agent simulations) suitable for CI, and the server/CLI were verified runtime to work keylessly. missing for 10: no explicit documented CI pipeline example (e.g., GitHub Actions config) or headless-mode confirmation beyond inference from server/testing docs.

   • [claimed-docs] “A programmatic participant is any code that joins a LiveKit room as a participant — this includes AI agents, media processors, or custom log…”
   • [claimed-docs] “The Agents framework isn't limited to AI agents. You can use it to deploy any code that needs to process realtime media and data streams as …”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
   • [claimed-docs] “Agent simulations run end-to-end conversations between your agent and an LLM-driven user, then evaluate the results across the full interact…”
   • [github] “Builtin test framework: Write tests and use judges to ensure your agent is performing as expected.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”

   proof ↗⚑ flag

   telli~partialprobed6/10

   telli exposes a full REST API (with OpenAPI spec) to programmatically schedule calls, manage contacts, and trigger workflows, plus webhooks for event-driven automation and scheduled/recurring workflow triggers — all of which support headless, non-UI automation suitable for CI-like pipelines. However there is no explicit CI/CD example, CLI tool, or documented automated pipeline integration beyond the API/webhook primitives. Missing for 10: dedicated CI/CD documentation or examples, a CLI for scripted invocation, and independent confirmation of headless use in production pipelines.

   • [claimed-docs] “API Endpoints: Use telli's REST API to programmatically manage contacts, schedule calls, and integrate with your existing systems.”
   • [claimed-docs] “Schedules a call at the earliest opportunity within the dialer window. May occur immediately or on the next business day depending on settin…”
   • [claimed-docs] “Workflows can now run automatically on a recurring schedule. Add a Scheduled trigger, choose the interval and start time, and let the workfl…”
   • [claimed-docs] “Webhooks push data from telli to your systems in real time. When something happens on a call, telli sends a POST request to the endpoint you…”
   • [probe] “PROBE openapi: HTTP 200 at https://docs.telli.com/openapi.json — contains "openapi" key”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round to LiveKit Agents
   LiveKit Agents✓fullclaimed8/10

   Docs and GitHub explicitly state 'first-class support for Model Context Protocol (MCP) servers' with a documented pattern to wrap an MCP server in an MCPToolset and pass it to the agent's tools parameter, described as one line of code integration. This is first-party documentation without independent hands-on verification of MCP-specific usage. Missing for 10: independent/community confirmation of MCP tool integration working end-to-end, and more detail on multi-server or auth configurations.

   • [claimed-docs] “Wrap an MCP server in an \`MCPToolset\` and pass it to the agent's \`tools\` parameter”
   • [claimed-docs] “Wrap an MCP server in an `MCPToolset` and pass it to the agent's `tools` parameter”
   • [claimed-docs] “LiveKit Agents has first-class support for Model Context Protocol (MCP) servers.”
   • [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
   • [github] “Native support for MCP. Integrate tools provided by MCP servers with one line of code.”

   proof ↗⚑ flag

   telli—none0/10

   Telli's documented MCP support only runs in the opposite direction — it exposes an MCP server so external AI tools (Claude, ChatGPT, Codex) can talk to its Charlie assistant (telli-docs-22, telli-probe-3), not a mechanism for telli's own voice agents to consume external MCP servers' tools. Custom Tools (telli-docs-14, telli-docs-20) let agents call external APIs but are described as custom integrations, not MCP-based tool ingestion.

   • [claimed-docs] “The telli MCP server lets AI tools like Claude, ChatGPT, and Codex talk to Charlie, telli's AI assistant, over the Model Context Protocol.”
   • [probe] “official MCP server documented at https://docs.telli.com/integrations/mcp”
   • [claimed-docs] “Custom Tools: Extend telli's capabilities with custom integrations and workflows.”
   • [claimed-docs] “Custom tools allow your AI agents to access external information by connecting to your APIs.”

   proof ↗⚑ flag

4. ai-native userConnect an agent via an official MCP server

   weight 3 · round to telli
   LiveKit Agents—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   telli✓fullprobed8/10

   telli documents an official MCP server that lets external AI tools (Claude, ChatGPT, Codex) connect over the Model Context Protocol to interact with Charlie, telli's assistant, confirmed both in docs and via a live probe of the integrations/mcp page. Missing for 10: no independent/hands-on corroboration beyond first-party docs and the probe, and no detail on server setup/auth specifics.

   • [claimed-docs] “The telli MCP server lets AI tools like Claude, ChatGPT, and Codex talk to Charlie, telli's AI assistant, over the Model Context Protocol.”
   • [probe] “official MCP server documented at https://docs.telli.com/integrations/mcp”

   proof ↗⚑ flag

5. ai-native userUse an official CLI

   weight 2 · round to LiveKit Agents
   LiveKit Agents✓fullprobed9/10

   LiveKit ships an official CLI (livekit-cli), documented on GitHub and verified hands-on to install and run (`lk --version` works via brew install), confirming it's a real, functional official CLI supporting the agent workflow. missing for 10: no deep documentation of full CLI command surface for agent-specific workflows within the evidence pack.

   • [probe] “official CLI documented at https://github.com/livekit/livekit-cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): `lk --version` printed `lk version 2.18.6` after a plain `brew install livekit-cli` — the official CLI …”

   proof ↗⚑ flag

   telli—none0/10

   telli offers a REST API, webhooks, and an MCP server for AI assistants, but there is no mention anywhere in the evidence of an official CLI tool for telli. missing for 10: any documentation or reference to a telli CLI, its commands, or installation instructions.

   ⚑ flag

6. ai-native userDrive the product through a documented public API

   weight 3 · round to telli
   LiveKit Agents✓fullprobed8/10

   LiveKit Agents ships a well-documented, pip/npm-installable Python and Node.js SDK with extensive public API surface (tools, MCP, turn detection, telephony, testing), plus a llms.txt and hands-on verified installs/runtime confirming the API is real and usable by AI-native developers. Missing for 10: independent third-party API reference/versioning audit and deeper evidence of API stability guarantees beyond docs and probes.

   • [claimed-docs] “The Agents framework lets you add any Python or Node.js program to LiveKit rooms as full realtime participants.”
   • [claimed-docs] “A programmatic participant is any code that joins a LiveKit room as a participant — this includes AI agents, media processors, or custom log…”
   • [claimed-docs] “The Agents framework isn't limited to AI agents. You can use it to deploy any code that needs to process realtime media and data streams as …”
   • [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use. This feature allows you to create a custom library of tools to extend your agent's context”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.livekit.io/llms.txt # LiveKit docs > LiveKit is an open-source platform for building voice, video,…”
   • [probe] “PROBE docs-md: HTTP 200 at https://docs.livekit.io/agents/.md LiveKit docs › Build Agents › Get Started › Introduction --- # Introduction …”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”

   proof ↗⚑ flag

   telli✓fullprobed9/10

   telli documents a REST API with endpoints (schedule-call, import-phone-number), API key/bearer auth, webhooks, and a downloadable OpenAPI spec (confirmed live at openapi.json), plus an MCP server for AI tool access — enabling full programmatic/agentic control. missing for 10: independent third-party validation of API robustness beyond vendor docs.

   • [claimed-docs] “API Endpoints: Use telli's REST API to programmatically manage contacts, schedule calls, and integrate with your existing systems.”
   • [claimed-docs] “Webhooks: Receive real-time notifications about call events, status changes, and outcomes.”
   • [claimed-docs] “All endpoints authenticate with an API key from the telli app (Settings > Developer), sent as a bearer token”
   • [claimed-docs] “Download them to generate client libraries, import the API into tools like Postman, or give coding agents machine-readable API context”
   • [claimed-docs] “Schedules a call at the earliest opportunity within the dialer window. May occur immediately or on the next business day depending on settin…”
   • [claimed-docs] “Imports a phone number from your own SIP trunk provider. This allows you to use existing phone numbers with telli by configuring custom SIP …”
   • [claimed-docs] “Webhooks push data from telli to your systems in real time. When something happens on a call, telli sends a POST request to the endpoint you…”
   • [claimed-docs] “The telli MCP server lets AI tools like Claude, ChatGPT, and Codex talk to Charlie, telli's AI assistant, over the Model Context Protocol.”
   • [probe] “PROBE openapi: HTTP 200 at https://docs.telli.com/openapi.json — contains "openapi" key”
   • [probe] “official MCP server documented at https://docs.telli.com/integrations/mcp”

   proof ↗⚑ flag

7. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round drawn
   LiveKit Agents—none0/10

   The evidence pack covers voice/agent features, tool-calling, MCP, testing, and telephony, but contains no mention of scoped or least-privilege API key/token issuance for agents (e.g., LiveKit's grant-based access tokens or credential scoping). Since LiveKit is a platform with API keys and would plausibly support such scoping, this is an applicable but undocumented axis in the given evidence.

   ⚑ flag

   telli—none0/10

   telli documents a single API key (bearer token) generated in Settings > Developer, but there's no evidence of scoped permissions, role-based keys, or least-privilege credential issuance for agents specifically.

   • [claimed-docs] “All endpoints authenticate with an API key from the telli app (Settings > Developer), sent as a bearer token”

   proof ↗⚑ flag

8. ai-native userBuild against official SDKs

   weight 2 · round to LiveKit Agents
   LiveKit Agents✓fullprobed9/10

   LiveKit provides official Python and Node.js SDKs (livekit-agents), verified hands-on to pip-install and import with no key, plus documented plugin ecosystem, tool use, MCP support, testing frameworks, and telephony integration—clearly an official SDK ecosystem for AI-native building. Missing for 10: independent third-party benchmarking of SDK API stability/versioning beyond community latency complaints unrelated to SDK build story.

   • [claimed-docs] “The Agents framework lets you add any Python or Node.js program to LiveKit rooms as full realtime participants.”
   • [claimed-docs] “LiveKit Agents includes a large ecosystem of open source plugins for a variety of AI providers.”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use. This feature allows you to create a custom library of tools to extend your agent's context”
   • [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”

   proof ↗⚑ flag

   telli~partialprobed5/10

   telli provides a REST API with an OpenAPI spec that can be used to generate client libraries, plus webhooks, custom tools, and an MCP server for agent integration, but there is no evidence of telli-maintained official SDKs in specific languages (e.g., Python/JS packages published by telli) — only self-generated clients from the OpenAPI spec. missing for 10: dedicated first-party SDK packages/repos, language-specific SDK documentation, versioning/release notes for such SDKs.

   • [claimed-docs] “API Endpoints: Use telli's REST API to programmatically manage contacts, schedule calls, and integrate with your existing systems.”
   • [claimed-docs] “All endpoints authenticate with an API key from the telli app (Settings > Developer), sent as a bearer token”
   • [claimed-docs] “Download them to generate client libraries, import the API into tools like Postman, or give coding agents machine-readable API context”
   • [probe] “PROBE openapi: HTTP 200 at https://docs.telli.com/openapi.json — contains "openapi" key”

   proof ↗⚑ flag

9. ai-native userSubscribe to events via webhooks

   weight 2 · round to telli
   LiveKit Agents—none0/10

   No evidence pack items mention webhooks or an event-subscription mechanism for LiveKit Agents; the docs cover MCP tool integration, telephony, testing, and turn detection, but nothing about webhook-based event notifications.

   ⚑ flag

   telli✓fullclaimed8/10

   telli documents a webhooks system that pushes real-time POST notifications about call events, status changes, and outcomes, explicitly described as event-driven subscription for external systems (telli-docs-13, telli-docs-21). Missing for 10: no evidence of configurable event-type filtering/subscription granularity or independent third-party confirmation of reliability.

   • [claimed-docs] “Webhooks: Receive real-time notifications about call events, status changes, and outcomes.”
   • [claimed-docs] “Webhooks push data from telli to your systems in real time. When something happens on a call, telli sends a POST request to the endpoint you…”
   • [claimed-docs] “API Endpoints: Use telli's REST API to programmatically manage contacts, schedule calls, and integrate with your existing systems.”

   proof ↗⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round to telli
   LiveKit Agents—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   telli~partialclaimed6/10

   telli ships a built-in AI assistant (Charlie) that can 'analyze your calls' and help build/refine agents, plus dashboards for reach rate, call duration, success rate, and per-call transcript/outcome analysis, which together provide AI-driven insight into product data. However, the docs don't detail proactive, autonomous 'suggestions' surfaced to the user (e.g., recommended actions, anomaly detection) beyond on-demand analysis via Charlie or manual dashboard review. Missing for 10: concrete examples of Charlie proactively surfacing insights/recommendations, independent user corroboration of insight quality.

   • [claimed-docs] “Charlie: Build and refine agents, automate workflows, and analyze your calls with telli's built-in AI assistant.”
   • [claimed-docs] “Track key metrics like reach rate, call duration, and success rate.”
   • [claimed-docs] “Analyze individual calls, transcripts, and call outcome fields.”
   • [claimed-docs] “Submit feedback on calls to continuously improve your agent.”

   proof ↗⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round drawn
   LiveKit Agents✓fullprobed7/10

   LiveKit Agents workers run as persistent 'programmatic participants' deployed to LiveKit Cloud or self-hosted infrastructure, processing realtime streams and telephony calls autonomously in the background without human intervention (docs-8, docs-20, docs-30, docs-26, gh-2), and this is verified hands-on as a real pip-installable OSS framework with a working self-hosted backbone (probe-rt-2, probe-rt-3). Missing for 10: explicit scheduling/cron-trigger mechanisms, independent third-party reports of long-running unattended stability (the one community mention is about observability tooling gaps, not autonomy failure).

   • [claimed-docs] “A programmatic participant is any code that joins a LiveKit room as a participant — this includes AI agents, media processors, or custom log…”
   • [claimed-docs] “The Agents framework isn't limited to AI agents. You can use it to deploy any code that needs to process realtime media and data streams as …”
   • [claimed-docs] “Run your agent on LiveKit Cloud's global infrastructure.”
   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”

   proof ↗⚑ flag

   telli✓fullclaimed7/10

   telli supports Scheduled triggers so workflows run automatically without a call event, plus retry/dialing-window configuration, webhooks, and an AI assistant (Charlie) to build/automate workflows — together enabling background autonomous automations. Missing for 10: independent/hands-on verification of scheduled automations running reliably in production, and more detail on monitoring/error-handling for autonomous runs beyond docs claims.

   • [claimed-docs] “Workflows can now run automatically on a recurring schedule. Add a Scheduled trigger, choose the interval and start time, and let the workfl…”
   • [claimed-docs] “Configure dialing windows, calling strategies, and retry behavior.”
   • [claimed-docs] “Charlie: Build and refine agents, automate workflows, and analyze your calls with telli's built-in AI assistant.”
   • [claimed-docs] “Webhooks: Receive real-time notifications about call events, status changes, and outcomes.”
   • [claimed-docs] “Schedules a call at the earliest opportunity within the dialer window. May occur immediately or on the next business day depending on settin…”

   proof ↗⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round to telli
   LiveKit Agents—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   telli✓fullprobed8/10

   telli documents "Charlie," a built-in AI assistant that can build/refine agents, automate workflows, and analyze calls, and this assistant is also exposed via an MCP server for external AI tools to interact with. This directly matches delegating tasks to a built-in assistant within the product. Missing for 10: independent/hands-on verification of Charlie's task delegation in practice and more detail on the scope of tasks it can autonomously execute.

   • [claimed-docs] “Charlie: Build and refine agents, automate workflows, and analyze your calls with telli's built-in AI assistant.”
   • [claimed-docs] “The telli MCP server lets AI tools like Claude, ChatGPT, and Codex talk to Charlie, telli's AI assistant, over the Model Context Protocol.”
   • [probe] “official MCP server documented at https://docs.telli.com/integrations/mcp”

   proof ↗⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round to telli
   LiveKit Agents~partialprobed4/10

   LiveKit ships an 'Agent Builder' that lets users prototype/deploy voice agents in-browser without code, and docs explicitly note the framework is 'built for coding agents like Claude Code, Cursor, and Codex' plus agent-readable llms.txt docs — suggesting some AI-native/no-code operability. However there is no evidence of a native natural-language command interface for operating LiveKit itself (e.g., NL-driven CLI, chatops control plane); the MCP support described is for agents calling external tools, not for controlling LiveKit via NL. Missing for 10: a documented NL-driven interface/CLI for configuring or operating the platform itself, and independent confirmation of the Agent Builder's no-code UX actually being NL-based rather than GUI-based.

   • [claimed-docs] “LiveKit is built for coding agents like Claude Code, Cursor, and Codex.”
   • [claimed-docs] “Prototype and deploy voice agents directly in your browser, without writing any code.”
   • [claimed-docs] “LiveKit Agent Builder to prototype and deploy agents directly in your browser without writing code”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.livekit.io/llms.txt # LiveKit docs > LiveKit is an open-source platform for building voice, video,…”
   • [probe] “PROBE docs-md: HTTP 200 at https://docs.livekit.io/agents/.md LiveKit docs › Build Agents › Get Started › Introduction --- # Introduction …”

   proof ↗⚑ flag

   telli~partialprobed6/10

   telli offers Charlie, a built-in AI assistant that can build/refine agents, automate workflows, and analyze calls via natural language, plus an MCP server letting external AI tools (Claude, ChatGPT, Codex) converse with Charlie — both point to natural-language operation. However, most of the product surface (workflows, scheduling, dialing windows, phone number setup) is described as configured through UI/API rather than conversational commands, and there's no independent/hands-on evidence of Charlie's NL command scope or reliability. missing for 10: hands-on evidence of Charlie executing complex natural-language commands, documentation detailing the breadth of actions controllable purely via NL, independent corroboration beyond vendor docs.

   • [claimed-docs] “Charlie: Build and refine agents, automate workflows, and analyze your calls with telli's built-in AI assistant.”
   • [claimed-docs] “The telli MCP server lets AI tools like Claude, ChatGPT, and Codex talk to Charlie, telli's AI assistant, over the Model Context Protocol.”
   • [probe] “official MCP server documented at https://docs.telli.com/integrations/mcp”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round to telli
   LiveKit Agents—none0/10

   Evidence shows extensive quickstart guides, markdown-based docs (llms.txt), and a no-code 'Agent Builder' prototyping tool, but nothing describes an interactive API reference (e.g., embedded code sandbox, live runnable examples, or Swagger-like explorer) for exploring the SDK/API itself.

   ⚑ flag

   telli~partialprobed5/10

   telli publishes an OpenAPI spec (openapi.json) and API docs with downloadable schemas for generating client libraries or feeding coding agents, but there is no evidence of an interactive reference with runnable/'try it' examples in-browser. missing for 10: an interactive try-it console, live runnable code samples, or sandboxed request execution within the docs site.

   • [claimed-docs] “Download them to generate client libraries, import the API into tools like Postman, or give coding agents machine-readable API context”
   • [probe] “PROBE openapi: HTTP 200 at https://docs.telli.com/openapi.json — contains "openapi" key”
   • [claimed-docs] “All endpoints authenticate with an API key from the telli app (Settings > Developer), sent as a bearer token”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round to telli
   LiveKit Agents—none0/10

   The evidence pack documents LiveKit's SDKs, CLI, docs (llms.txt), and framework capabilities but contains no mention of a downloadable OpenAPI/Swagger spec or other machine-readable API description for LiveKit's server or Agents APIs. Since LiveKit exposes a real API surface (server API, cloud API), this axis applies but is unevidenced.

   • [probe] “official CLI documented at https://github.com/livekit/livekit-cli”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.livekit.io/llms.txt # LiveKit docs > LiveKit is an open-source platform for building voice, video,…”
   • [probe] “PROBE docs-md: HTTP 200 at https://docs.livekit.io/agents/.md LiveKit docs › Build Agents › Get Started › Introduction --- # Introduction …”

   proof ↗⚑ flag

   telli✓fullprobed9/10

   telli provides a downloadable OpenAPI spec confirmed live at /openapi.json, and docs explicitly state it can be downloaded to generate client libraries, import into tools, or give coding agents machine-readable API context. Missing for 10: no independent third-party confirmation of spec completeness/versioning beyond first-party docs and probe.

   • [claimed-docs] “Download them to generate client libraries, import the API into tools like Postman, or give coding agents machine-readable API context”
   • [probe] “PROBE openapi: HTTP 200 at https://docs.telli.com/openapi.json — contains "openapi" key”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round to LiveKit Agents
   LiveKit Agents~partialprobed6/10

   LiveKit provides a self-hostable dev server (`livekit-server --dev`) that boots locally with placeholder keys, separate from any production deployment, plus a built-in testing framework with behavioral tests and LLM-driven agent simulations that evaluate agent behavior without needing real production data. However, there is no explicitly branded 'sandbox environment' or staging/production data-isolation feature documented — the sandbox-like capability is inferred from dev-mode self-hosting and test simulations rather than a dedicated sandbox product feature. Missing for 10: an explicit sandbox/staging environment offering with documented separation from production data, and independent confirmation that test simulations never touch production data stores.

   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
   • [claimed-docs] “Agent simulations run end-to-end conversations between your agent and an LLM-driven user, then evaluate the results across the full interact…”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes... Agent simulations run end-to-end conversations between your agent and…”
   • [github] “Builtin test framework: Write tests and use judges to ensure your agent is performing as expected.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”

   proof ↗⚑ flag

   telli~partialclaimed4/10

   telli's agent builder includes a 'Test call' feature that lets users dial a draft or saved agent version and review the recording/transcript before going live, which functions as a lightweight test mode (telli-docs-28, telli-docs-29). However, there is no documentation of a dedicated sandbox environment, separate test dataset, or explicit guarantee that test calls don't write to production contacts/analytics. missing for 10: explicit sandbox/staging environment separate from production, confirmation that test calls don't affect production data/metrics, and any mention of test API keys or isolated test accounts.

   • [claimed-docs] “In the agent builder, click Test call to open the dialog. Pick the agent version (your draft with unsaved changes or any saved revision)...”
   • [claimed-docs] “Use the side-by-side view to listen to the recording while following along in the transcript.”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   LiveKit Agents—none0/10

   No evidence in the pack mentions API versioning scheme, version compatibility guarantees, or a documented deprecation policy for LiveKit Agents' SDKs or APIs.

   ⚑ flag

   telli—none0/10

   Evidence shows an OpenAPI spec and API key auth exist, but nothing in the pack mentions API versioning (e.g., v1/v2 paths, version headers) or any documented deprecation/sunset policy for endpoints. Missing for 10: explicit API version scheme, changelog of breaking changes, and a stated deprecation/sunset policy.

   • [claimed-docs] “API Endpoints: Use telli's REST API to programmatically manage contacts, schedule calls, and integrate with your existing systems.”
   • [claimed-docs] “All endpoints authenticate with an API key from the telli app (Settings > Developer), sent as a bearer token”
   • [probe] “PROBE openapi: HTTP 200 at https://docs.telli.com/openapi.json — contains "openapi" key”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to telli
   LiveKit Agents—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   telli~partialprobed6/10

   Telli's platform is built around calling large contact lists (dialing windows, calling strategies, retry behavior, reach-rate metrics) and its REST API lets users 'programmatically manage contacts, schedule calls' at scale, implying bulk campaign operations. However, there is no explicit documentation of a bulk-upload/bulk-schedule endpoint or UI for acting on many items simultaneously — the schedule-call endpoint documented is single-call oriented. Missing for 10: explicit bulk-contact import/schedule endpoint, documented batch size limits, and independent evidence of large-scale bulk execution.

   • [claimed-docs] “Configure dialing windows, calling strategies, and retry behavior.”
   • [claimed-docs] “Track key metrics like reach rate, call duration, and success rate.”
   • [claimed-docs] “API Endpoints: Use telli's REST API to programmatically manage contacts, schedule calls, and integrate with your existing systems.”
   • [claimed-docs] “Schedules a call at the earliest opportunity within the dialer window. May occur immediately or on the next business day depending on settin…”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.telli.com/llms.txt # telli Documentation > telli is the AI voice platform for agents that sell, su…”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round drawn
   LiveKit Agents~partialclaimed5/10

   LiveKit Agents supports event-driven automatic behaviors — tool-calling on LLM decisions, turn-detector triggering response timing, and adaptive interruption handling triggering barge-in logic — all of which are 'events auto-trigger actions' patterns built into the agent runtime. However, the evidence shows these as built-in framework behaviors and developer-coded event handlers rather than a user-facing declarative rule engine (e.g., no evidence of a 'when X happens do Y' config surface, webhook/rule subscription API, or no-code rule builder). Missing for 10: explicit rule-definition/webhook-trigger API exposed to non-developer users, documentation of a generalized event-subscription system beyond tool calls and turn logic, and independent confirmation of custom rule automation in production use.

   • [claimed-docs] “Call external APIs or lookup data for RAG.”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use. This feature allows you to create a custom library of tools to extend your agent's context”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use.”
   • [claimed-docs] “LiveKit's `TurnDetector` is an audio model that encodes user audio directly, capturing both _what_ is said and _how_ it's said.”
   • [claimed-docs] “A turn detector model can predict that they have more to say and wait for them to finish before responding.”
   • [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response... to identify intentional interruptions (…”
   • [claimed-docs] “the model analyzes the acoustic signals to identify intentional interruptions (barge-ins) from conversational backchanneling”
   • [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response.”

   proof ↗⚑ flag

   telli~partialclaimed5/10

   telli's workflows can be triggered by call events (implied default) and now by a recurring schedule, and webhooks notify external systems on call events, showing some event-driven automation. However there's no documented rule-definition interface (conditions, event-type selection, if/then logic) for arbitrary events beyond calls/schedule. Missing for 10: explicit rule/condition builder, broader event-type coverage beyond calls, and evidence of user-defined trigger logic rather than fixed schedule/call triggers.

   • [claimed-docs] “Workflows can now run automatically on a recurring schedule. Add a Scheduled trigger, choose the interval and start time, and let the workfl…”
   • [claimed-docs] “Webhooks: Receive real-time notifications about call events, status changes, and outcomes.”
   • [claimed-docs] “Webhooks push data from telli to your systems in real time. When something happens on a call, telli sends a POST request to the endpoint you…”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round to telli
   LiveKit Agents—none0/10

   The evidence pack covers LiveKit Agents' realtime voice/video agent capabilities, tool use, MCP integration, telephony, and testing, but contains no mention of any scheduling, cron-like recurring job, or workflow-automation trigger mechanism. This is a fair capability to ask about for an automation-focused agent framework, but no evidence supports it.

   ⚑ flag

   telli✓fullclaimed7/10

   telli documents scheduled triggers that let workflows run recurringly on an interval/start time without a call event, directly matching the recurring-workflow story, and also supports scheduling individual calls via API. missing for 10: independent/hands-on corroboration of recurrence reliability and detail on editing/monitoring scheduled workflow runs.

   • [claimed-docs] “Workflows can now run automatically on a recurring schedule. Add a Scheduled trigger, choose the interval and start time, and let the workfl…”
   • [claimed-docs] “Schedules a call at the earliest opportunity within the dialer window. May occur immediately or on the next business day depending on settin…”
   • [claimed-docs] “Optional flag to bypass dialing window restrictions for the scheduled call. When true, calls at the scheduled time regardless of dialing win…”

   proof ↗⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round to LiveKit Agents
   LiveKit Agents~partialclaimed4/10

   The pricing page explicitly claims 'Instant rollback to a previous agent deployment,' and the testing/simulation framework (behavioral tests, LLM-judge simulations) supports a review step before deployment, but there's no documented versioning system, changelog, or diff/history UI for automations. Missing for 10: explicit version history/diffing of agent configs, a documented review/approval workflow beyond test suites, and independent confirmation that rollback works in practice.

   • [claimed-docs] “Instant rollback to a previous agent deployment”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
   • [claimed-docs] “Agent simulations run end-to-end conversations between your agent and an LLM-driven user, then evaluate the results across the full interact…”
   • [github] “Builtin test framework: Write tests and use judges to ensure your agent is performing as expected.”

   proof ↗⚑ flag

   telli~partialclaimed3/10

   The only hint of versioning is in agent testing, where you can pick 'your draft with unsaved changes or any saved revision' (telli-docs-28), implying saved agent versions exist. However there is no documented review interface for comparing versions, no explicit rollback/restore action, and no versioning mentioned for broader automations/workflows (only agents). Missing for 10: explicit rollback UI/API, version diff/review tooling, and versioning coverage for workflows beyond agent test calls.

   • [claimed-docs] “In the agent builder, click Test call to open the dialog. Pick the agent version (your draft with unsaved changes or any saved revision)...”

   proof ↗⚑ flag

Compliance

1. founderMeet call-recording consent and disclosure obligations with per-call recording controls and configurable data retention

   weight 2 · round to telli
   LiveKit Agents—none0/10

   No evidence in the pack mentions call recording, consent disclosure, or data retention controls anywhere in LiveKit Agents docs, GitHub, or community sources; the pack covers telephony, tool use, MCP, testing, and turn detection but nothing about recording/retention compliance features. missing for 10: recording consent/disclosure mechanisms, per-call recording toggles, configurable data retention policies, and any documentation or hands-on evidence of these compliance controls.

   ⚑ flag

   telli~partialclaimed6/10

   telli documents conditional recording based on caller consent and a configurable data-retention policy that auto-deletes contacts and call data after a defined age, directly addressing the founder's compliance need. However, evidence lacks detail on per-call granular controls (e.g., pause/resume, jurisdiction-specific disclosure scripts), audit logging of consent events, or independent verification that these features work as described. Missing for 10: per-call recording toggle/audit trail details, jurisdiction-specific disclosure templates, retention configuration granularity (e.g., per-data-type settings), and independent/hands-on confirmation of compliance behavior.

   • [claimed-docs] “Data retention deletes contacts and call data automatically once they reach an age you define.”
   • [claimed-docs] “In telli, you can easily make audio recording conditional on the caller's consent — so a call is only recorded once they agree.”

   proof ↗⚑ flag

2. platform-engineerRun regulated workloads with HIPAA/BAA support, SOC 2, and data-residency options

   weight 2 · round drawn
   LiveKit Agents—none0/10

   The evidence pack contains no mention of HIPAA/BAA agreements, SOC 2 certification, or data-residency controls anywhere in LiveKit's docs, GitHub repo, or probes; self-hosting and TURN/security items only cover TLS/SSL and self-hosted deployment, not compliance attestations. As a platform serving enterprise/regulated workloads, this axis clearly applies, so lack of evidence yields 'none'.

   • [claimed-docs] “The good news is LiveKit includes an embedded TURN server. It's a secure TURN implementation that has integrated authentication with the res…”
   • [claimed-docs] “In order to have a secure LiveKit deployment, you will need a domain as well as a SSL certificate for that domain.”

   proof ↗⚑ flag

   telli—none0/10

   No evidence in the pack mentions HIPAA, BAA, SOC 2, or data-residency options; only data retention and consent-based recording are covered, which don't establish formal compliance certifications.

   ⚑ flag

Scale

1. platform-engineerSee documented concurrency limits and scale to many simultaneous calls without manual capacity begging

   weight 2 · round to LiveKit Agents
   LiveKit Agents~partialcommunity3/10

   Docs mention running agents on LiveKit Cloud's global infrastructure and self-hosting with an embedded TURN server, implying scale-out capability, but no evidence pack item cites concrete documented concurrency limits, per-instance call caps, or autoscaling guarantees; a community report even flags lack of visibility into per-call cost/latency at scale. missing for 10: published concurrency/capacity numbers, autoscaling documentation, load-testing benchmarks, and confirmation that scaling requires no manual quota requests.

   • [claimed-docs] “Run your agent on LiveKit Cloud's global infrastructure.”
   • [claimed-docs] “The good news is LiveKit includes an embedded TURN server. It's a secure TURN implementation that has integrated authentication with the res…”
   • [community] “We've been running multiple voice AI agents on LiveKit and kept running into visibility issues — no way to measure TTFT, latency across STT …”

   proof ↗⚑ flag

   telli—none0/10

   No evidence pack item documents concurrency limits, per-account call caps, or scaling guarantees for simultaneous calls; docs mention dialing windows/retry strategy but not concurrency ceilings or auto-scaling behavior. This is a fair question for a voice-calling platform (scaling many simultaneous calls is core to the product's value), so 'none' applies rather than 'na'.

   ⚑ flag

Self host

1. platform-engineerSelf-host the voice agent runtime from open-source code on my own infrastructure

   weight 3 · round to LiveKit Agents
   LiveKit Agents✓fullprobed9/10

   LiveKit Agents is open source, pip/npm installable and runs the agent runtime independent of any vendor account, and the underlying WebRTC/SIP server (livekit-server) is Apache-2.0 and self-hostable, confirmed via hands-on probes (booting `livekit-server --dev` locally with no keys, and installing livekit-agents/CLI keylessly) plus official self-hosting deployment docs covering TURN, TLS/domain setup. Missing for 10: no first-party production-scale self-hosting case study or independent report of large-scale self-hosted deployment beyond dev-mode probe.

   • [probe] “PROBE runtime (recorded 2026-09-05): `lk --version` printed `lk version 2.18.6` after a plain `brew install livekit-cli` — the official CLI …”
   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”
   • [claimed-docs] “The good news is LiveKit includes an embedded TURN server. It's a secure TURN implementation that has integrated authentication with the res…”
   • [claimed-docs] “In order to have a secure LiveKit deployment, you will need a domain as well as a SSL certificate for that domain.”
   • [claimed-docs] “A programmatic participant is any code that joins a LiveKit room as a participant — this includes AI agents, media processors, or custom log…”

   proof ↗⚑ flag

   telli—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

Latency

1. platform-engineerSee documented end-to-end voice latency numbers or tuning guidance backing the platform's speed claims

   weight 3 · round drawn
   LiveKit Agents—none0/10

   The evidence pack contains no documented end-to-end latency benchmarks or explicit tuning guidance for reducing turn-taking latency — only feature descriptions (turn detector, adaptive interruption handling) without numbers or configuration guidance. A community report explicitly states users could not measure TTFT or STT→LLM→TTS latency, reinforcing that this visibility/documentation is absent.

   • [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response... to identify intentional interruptions (…”
   • [claimed-docs] “LiveKit's `TurnDetector` is an audio model that encodes user audio directly, capturing both _what_ is said and _how_ it's said.”
   • [claimed-docs] “A turn detector model can predict that they have more to say and wait for them to finish before responding.”
   • [claimed-docs] “the model analyzes the acoustic signals to identify intentional interruptions (barge-ins) from conversational backchanneling”
   • [community] “We've been running multiple voice AI agents on LiveKit and kept running into visibility issues — no way to measure TTFT, latency across STT …”

   proof ↗⚑ flag

   telli—none0/10

   No evidence pack item mentions latency numbers, turn-taking performance, or tuning guidance for voice response speed; documentation covers workflows, API, webhooks, and agent building but nothing on end-to-end latency benchmarks or tuning.

   ⚑ flag

Turn taking

1. developerRely on the agent to handle interruptions (barge-in) gracefully — stopping speech, updating context, and recovering the turn

   weight 3 · round to LiveKit Agents
   LiveKit Agents✓fullprobed8/10

   LiveKit Agents has well-documented, purpose-built features for barge-in: adaptive interruption handling distinguishes intentional interruptions from backchanneling, and the TurnDetector model analyzes acoustic and semantic signals to manage turn-taking recovery. These are first-party docs with specific technical detail on the mechanism (stopping speech, updating context, recovering turn), and the runtime probes confirm the framework is real and installable. Missing for 10: independent/hands-on evidence specifically testing barge-in behavior in production (only community evidence is about latency observability, not interruption handling itself) and no third-party benchmark of interruption accuracy.

   • [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response... to identify intentional interruptions (…”
   • [claimed-docs] “the model analyzes the acoustic signals to identify intentional interruptions (barge-ins) from conversational backchanneling”
   • [claimed-docs] “LiveKit's `TurnDetector` is an audio model that encodes user audio directly, capturing both _what_ is said and _how_ it's said.”
   • [claimed-docs] “A turn detector model can predict that they have more to say and wait for them to finish before responding.”
   • [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response.”
   • [claimed-docs] “LiveKit's TurnDetector is an audio model that encodes user audio directly, capturing both what is said and how it's said.”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”

   proof ↗⚑ flag

   telli—none0/10

   No evidence in the pack addresses barge-in, interruption handling, speech-stopping, or turn recovery mechanics; the docs cover scheduling, workflows, analytics, and integrations but never mention conversational turn-taking behavior.

   ⚑ flag

2. developerEnable noise suppression or audio filtering so the agent stays coherent on noisy real-world calls

   weight 1 · round drawn
   LiveKit Agents—none0/10

   The evidence pack covers turn detection, interruption handling, telephony, MCP tools, and testing, but contains no mention of noise suppression or audio filtering plugins/features anywhere. Missing for 10: any docs or plugin reference to noise cancellation (e.g. Krisp/BVC), background noise filtering, or audio pre-processing configuration.

   ⚑ flag

   telli—none0/10

   No evidence pack item mentions noise suppression, audio filtering, or any mechanism for handling noisy call environments; documentation covers scheduling, workflows, analytics, and voice cloning but not audio quality/noise handling.

   ⚑ flag

3. developerUse model-based end-of-turn detection beyond simple VAD silence timeouts so the agent doesn't talk over slow speakers

   weight 2 · round to LiveKit Agents
   LiveKit Agents✓fullclaimed9/10

   LiveKit's TurnDetector is a dedicated audio+semantic model (not just VAD silence) that predicts whether a user has finished speaking or has more to say, waiting accordingly, and adaptive interruption handling further distinguishes real barge-ins from backchanneling — directly addressing not talking over slow speakers. This is documented in detail across multiple first-party doc pages plus GitHub feature lists. missing for 10: independent/hands-on benchmark or community validation of the turn-detector's real-world latency/accuracy impact.

   • [claimed-docs] “LiveKit's `TurnDetector` is an audio model that encodes user audio directly, capturing both _what_ is said and _how_ it's said.”
   • [claimed-docs] “A turn detector model can predict that they have more to say and wait for them to finish before responding.”
   • [claimed-docs] “LiveKit's TurnDetector is an audio model that encodes user audio directly, capturing both what is said and how it's said.”
   • [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response... to identify intentional interruptions (…”
   • [claimed-docs] “the model analyzes the acoustic signals to identify intentional interruptions (barge-ins) from conversational backchanneling”
   • [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response.”

   proof ↗⚑ flag

   telli—none0/10

   No evidence in the pack discusses turn-taking, end-of-turn detection, VAD, or interruption handling in telli's voice agent; the docs cover scheduling, workflows, tools, and integrations but not conversational latency/turn-taking mechanics.

   ⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round drawn
   LiveKit Agents~partialprobed6/10

   LiveKit Agents is API/SDK-first (Python/Node.js) with a full programmatic surface for building agents, tools, telephony, and testing, and there is also a no-code 'Agent Builder' UI for prototyping. However, the evidence doesn't confirm that everything achievable in that browser-based Agent Builder UI (or LiveKit Cloud dashboard features like deployment management, rollback) is equally exposed via API/CLI, so parity between UI and API is not fully demonstrated. missing for 10: explicit evidence that Agent Builder's no-code UI actions (and Cloud dashboard deployment/rollback controls) are all reachable via API/CLI, and confirmation of full CLI/API parity with dashboard features.

   • [claimed-docs] “Prototype and deploy voice agents directly in your browser, without writing any code.”
   • [claimed-docs] “LiveKit Agent Builder to prototype and deploy agents directly in your browser without writing code”
   • [claimed-docs] “LiveKit Agent Builder: Prototype and deploy voice agents directly in your browser, without writing any code.”
   • [claimed-docs] “Instant rollback to a previous agent deployment”
   • [probe] “official CLI documented at https://github.com/livekit/livekit-cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): `lk --version` printed `lk version 2.18.6` after a plain `brew install livekit-cli` — the official CLI …”
   • [claimed-docs] “A programmatic participant is any code that joins a LiveKit room as a participant — this includes AI agents, media processors, or custom log…”

   proof ↗⚑ flag

   telli~partialprobed6/10

   telli exposes a REST API (with OpenAPI spec) covering contacts, scheduling calls, phone number import, custom tools, and webhooks, and also offers an MCP server so external AI tools can interact with Charlie. However, several UI-native features—agent builder/test-call dialog, live monitoring dashboard, side-by-side transcript/recording view, voice cloning setup, workflow trigger/recurring schedule configuration UI, and analytics dashboards—are not documented as API-accessible endpoints, so full UI/API parity is not evidenced. missing for 10: API endpoints for agent building/testing, live monitoring, voice cloning, workflow scheduling configuration, and analytics/reporting equivalent to UI dashboards.

   • [claimed-docs] “API Endpoints: Use telli's REST API to programmatically manage contacts, schedule calls, and integrate with your existing systems.”
   • [claimed-docs] “All endpoints authenticate with an API key from the telli app (Settings > Developer), sent as a bearer token”
   • [claimed-docs] “Download them to generate client libraries, import the API into tools like Postman, or give coding agents machine-readable API context”
   • [claimed-docs] “Schedules a call at the earliest opportunity within the dialer window. May occur immediately or on the next business day depending on settin…”
   • [claimed-docs] “Imports a phone number from your own SIP trunk provider. This allows you to use existing phone numbers with telli by configuring custom SIP …”
   • [claimed-docs] “Custom tools allow your AI agents to access external information by connecting to your APIs.”
   • [claimed-docs] “Webhooks push data from telli to your systems in real time. When something happens on a call, telli sends a POST request to the endpoint you…”
   • [claimed-docs] “The telli MCP server lets AI tools like Claude, ChatGPT, and Codex talk to Charlie, telli's AI assistant, over the Model Context Protocol.”
   • [probe] “PROBE openapi: HTTP 200 at https://docs.telli.com/openapi.json — contains "openapi" key”
   • [probe] “official MCP server documented at https://docs.telli.com/integrations/mcp”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round drawn
   LiveKit Agents—none0/10

   The evidence pack documents LiveKit Agents' open-source, self-hostable nature (Apache-2.0 server, pip/npm installable framework) but contains no explicit mention of data export tooling, open data formats for conversation/session logs, or a documented exit/migration path for user data. Self-hosting mitigates lock-in in principle, but that is not the same as a documented 'export all data' capability.

   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”
   • [claimed-docs] “The good news is LiveKit includes an embedded TURN server. It's a secure TURN implementation that has integrated authentication with the res…”

   proof ↗⚑ flag

   telli—none0/10

   No evidence of a bulk data-export feature or open-format data portability; only API endpoints to manage/read specific records and a data-retention deletion policy are documented, not a full self-service export/leave capability. Missing for 10: documented full-account export tool/endpoint, open-format (CSV/JSON) bulk export, and any explicit data-portability statement.

   • [claimed-docs] “API Endpoints: Use telli's REST API to programmatically manage contacts, schedule calls, and integrate with your existing systems.”
   • [claimed-docs] “Data retention deletes contacts and call data automatically once they reach an age you define.”

   proof ↗⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round to LiveKit Agents
   LiveKit Agents✓fullprobed8/10

   LiveKit Agents is hosted at github.com/livekit/agents and documented as an open-source framework, with hands-on verification that the packages install and run without any account or key (pip/npm) and that the underlying server is Apache-2.0 OSS. The llms.txt docs also explicitly describe LiveKit as 'an open-source platform.' Missing for 10: an explicit license file/badge citation specifically for the livekit/agents repo (only inferred via probe evidence and sibling repo license).

   • [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.livekit.io/llms.txt # LiveKit docs > LiveKit is an open-source platform for building voice, video,…”
   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”
   • [probe] “official CLI documented at https://github.com/livekit/livekit-cli”

   proof ↗⚑ flag

   telli—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round to LiveKit Agents
   LiveKit Agents✓fullprobed9/10

   LiveKit provides a documented, Apache-2.0 open-source self-hosting deployment guide including embedded TURN server and SSL setup, and this was hands-on verified: `livekit-server --dev` boots with no account or key, and the open-source livekit-agents framework installs and imports keylessly via pip/npm. This directly demonstrates the core product (server + agents framework) can be self-hosted. Missing for 10: independent third-party production self-hosting case study beyond the probe verification.

   • [claimed-docs] “The good news is LiveKit includes an embedded TURN server. It's a secure TURN implementation that has integrated authentication with the res…”
   • [claimed-docs] “In order to have a secure LiveKit deployment, you will need a domain as well as a SSL certificate for that domain.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”
   • [probe] “PROBE runtime (recorded 2026-09-05): `lk --version` printed `lk version 2.18.6` after a plain `brew install livekit-cli` — the official CLI …”

   proof ↗⚑ flag

   telli—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

Pricing

1. founderSee published per-minute or usage pricing and estimate cost per call before committing

   weight 2 · round drawn
   LiveKit Agents—none0/10

   The pack references a pricing URL (livekit.io/pricing) only for an unrelated 'rollback' feature and never quotes actual per-minute or usage rates; a community comment even highlights that operators struggled with 'cost per call' visibility, but no evidence shows published pricing figures a founder could use to estimate costs before committing.

   • [claimed-docs] “Instant rollback to a previous agent deployment”
   • [community] “We've been running multiple voice AI agents on LiveKit and kept running into visibility issues — no way to measure TTFT, latency across STT …”

   proof ↗⚑ flag

   telli—none0/10

   No evidence in the pack mentions pricing, per-minute rates, usage costs, or any cost calculator/estimator; the docs cover features (workflows, appointments, API, MCP, etc.) but nothing about published pricing.

   ⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round to LiveKit Agents
   LiveKit Agents~partialprobed5/10

   LiveKit Agents/server is open-source and self-hostable (embedded TURN server, deployment docs, hands-on verified self-host boot), which lets an operator choose their own hosting region/data location, but there is no explicit documented region-selection or data-residency feature for LiveKit Cloud itself. missing for 10: explicit LiveKit Cloud region-selection/data-residency controls, compliance certifications (e.g. GDPR/SOC2 region guarantees), and any customer-facing UI for choosing storage region.

   • [claimed-docs] “The good news is LiveKit includes an embedded TURN server. It's a secure TURN implementation that has integrated authentication with the res…”
   • [claimed-docs] “In order to have a secure LiveKit deployment, you will need a domain as well as a SSL certificate for that domain.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”

   proof ↗⚑ flag

   telli—none0/10

   No evidence in the pack mentions data residency, region selection, or data storage location controls; data retention (deletion timing) is covered but that is a different capability from choosing storage region.

   ⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   LiveKit Agents—none0/10

   No evidence in the pack addresses data usage for AI model training, opt-out policies, or any privacy commitments regarding training data; the evidence pack is entirely about agent framework features (tools, telephony, testing, MCP support). This is a plausible axis for a platform handling user voice/media data, but absent any documentation on training-data usage or opt-out, it cannot be credited as delivered.

   ⚑ flag

   telli—none0/10

   No evidence in the pack addresses AI training data usage or opt-out policies; data retention docs cover deletion of contacts/call data but not model-training exclusion.

   ⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round to telli
   LiveKit Agents—none0/10

   The evidence pack contains no mention of data retention policies, recording/session deletion controls, or configurable retention windows for LiveKit Agents. While self-hosting is documented (implying infrastructure control), no explicit retention/deletion feature or API is evidenced.

   ⚑ flag

   telli~partialclaimed6/10

   telli documents automatic data retention deletion (contacts and call data purged once they reach a defined age) and conditional consent-based recording, giving users some control over data lifecycle and privacy. However, there's no evidence of user-initiated manual deletion, export/right-to-access tools, or granular per-record deletion controls beyond the age-based policy. missing for 10: manual/on-demand deletion controls, data export capability, independent verification of retention enforcement.

   • [claimed-docs] “Data retention deletes contacts and call data automatically once they reach an age you define.”
   • [claimed-docs] “In telli, you can easily make audio recording conditional on the caller's consent — so a call is only recorded once they agree.”

   proof ↗⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   LiveKit Agents—none0/10

   No evidence pack item discusses telemetry, usage tracking, analytics collection, or an opt-out mechanism for LiveKit Agents; the framework being open-source and self-hostable is not sufficient evidence of an explicit telemetry opt-out control.

   ⚑ flag

   telli—none0/10

   telli is a voice-agent platform, not a developer tool where telemetry opt-out is typically discussed, but the axis is still fair to ask (does the product track usage, and can a user disable it). The evidence pack (data retention, consent-based recording, etc.) never mentions any telemetry/usage-tracking opt-out mechanism for the product itself. Missing for 10: any documentation of telemetry collection practices or an opt-out setting/toggle.

   ⚑ flag

Call control

1. developerEscalate a live call to a human with warm or blind transfer, passing context along

   weight 2 · round to telli
   LiveKit Agents—none0/10

   LiveKit Agents documents inbound/outbound telephony (SIP trunks, phone calls) but the evidence pack contains no mention of call transfer (warm or blind) or passing conversational context to a human agent during a live call — this specific escalation capability is never described.

   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [claimed-docs] “Enable your voice agent to make or take phone calls.”
   • [claimed-docs] “Outbound trunks are used to place outgoing calls.”
   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”

   proof ↗⚑ flag

   telli~partialclaimed4/10

   Docs confirm a transfer tool that can move a call to another agent or number, matching the core escalation-to-human idea, but there's no mention of distinct warm vs. blind transfer modes or explicit context-passing to the human recipient. missing for 10: explicit warm/blind transfer distinction, evidence of context/data hand-off during transfer, hands-on confirmation of the feature in practice.

   • [claimed-docs] “Transfer tools | Move the call to another agent or number.”

   proof ↗⚑ flag

2. developerMy agent can send DTMF keypresses, navigate IVR menus, and detect or leave voicemail

   weight 1 · round drawn
   LiveKit Agents—none0/10

   Evidence confirms LiveKit's telephony/SIP stack supports inbound/outbound calls (docs-26, docs-31, docs-32, gh-2), but nothing in the pack mentions DTMF keypress sending, IVR menu navigation, or voicemail detection/leaving capabilities specifically. Missing for 10: any documentation or example of DTMF tone generation, automated IVR menu traversal, or voicemail detection/answering-machine-detection logic.

   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [claimed-docs] “Enable your voice agent to make or take phone calls.”
   • [claimed-docs] “Outbound trunks are used to place outgoing calls.”
   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”

   proof ↗⚑ flag

   telli—none0/10

   No evidence in the pack mentions DTMF keypress sending, IVR menu navigation, or voicemail detection/leaving; telli's docs cover scheduling, transfers, web search tool, and webhooks but nothing about touch-tone or answering machine handling.

   ⚑ flag

Campaigns

1. founderRun batch outbound call campaigns with scheduling and throughput controls

   weight 2 · round to telli
   LiveKit Agents—none0/10

   Evidence shows LiveKit Agents can place individual outbound calls via SIP outbound trunks (docs-26, docs-32, docs-31), but there is no evidence of any batch campaign management, call scheduling, or throughput/rate-limiting controls for running outbound calling campaigns at scale — these features are simply not documented anywhere in the pack.

   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [claimed-docs] “Outbound trunks are used to place outgoing calls.”
   • [claimed-docs] “Enable your voice agent to make or take phone calls.”
   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”

   proof ↗⚑ flag

   telli~partialclaimed6/10

   telli documents dialing windows, calling strategies, and retry behavior (telli-docs-9) plus a schedule-call API endpoint that respects dialer windows or can bypass them (telli-docs-17, telli-docs-18) and reach-rate/success metrics (telli-docs-7), which together support scheduled, throughput-controlled outbound calling. However there is no explicit mention of a 'campaign' concept, bulk contact upload, or concurrency/rate limits for large batches. Missing for 10: explicit batch/campaign creation UI or endpoint, bulk contact import for calling, and documented concurrency/throughput limits beyond dialing windows.

   • [claimed-docs] “Configure dialing windows, calling strategies, and retry behavior.”
   • [claimed-docs] “Schedules a call at the earliest opportunity within the dialer window. May occur immediately or on the next business day depending on settin…”
   • [claimed-docs] “Optional flag to bypass dialing window restrictions for the scheduled call. When true, calls at the scheduled time regardless of dialing win…”
   • [claimed-docs] “Track key metrics like reach rate, call duration, and success rate.”
   • [claimed-docs] “Workflows can now run automatically on a recurring schedule. Add a Scheduled trigger, choose the interval and start time, and let the workfl…”

   proof ↗⚑ flag

Numbers

1. developerProvision phone numbers and run both inbound and outbound calls through the platform's API

   weight 3 · round to telli
   LiveKit Agents~partialclaimed6/10

   LiveKit's SIP/telephony stack clearly supports inbound and outbound calling ('LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls', 'Outbound trunks are used to place outgoing calls', 'Enable your voice agent to make or take phone calls') and integrates with the agents runtime via SIP trunks. However, the evidence never shows LiveKit itself provisioning phone numbers through its own API — telephony typically requires configuring an external SIP trunk provider, and no docs here describe a native number-provisioning endpoint. Missing for 10: explicit documentation/evidence of a LiveKit API call that provisions/purchases phone numbers directly (rather than just configuring trunks against externally-acquired numbers), and independent/hands-on confirmation of the inbound+outbound call flow.

   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [claimed-docs] “Enable your voice agent to make or take phone calls.”
   • [claimed-docs] “Outbound trunks are used to place outgoing calls.”
   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”

   proof ↗⚑ flag

   telli~partialprobed7/10

   telli's docs show a REST API with a schedule-call endpoint for outbound calls, an import-phone-number endpoint to bring SIP-trunk numbers under API control, webhooks for call events, and an OpenAPI spec confirming the API surface. However, inbound handling is documented mainly as manual forwarding/SIP trunk setup rather than a clear 'provision a new number via API' flow, and there's no explicit doc of purchasing/creating a fresh telli-hosted number through the API. missing for 10: a documented endpoint to provision/purchase brand-new phone numbers (not just import existing SIP trunk numbers), and clearer API-driven inbound call handling beyond forwarding.

   • [claimed-docs] “API Endpoints: Use telli's REST API to programmatically manage contacts, schedule calls, and integrate with your existing systems.”
   • [claimed-docs] “Schedules a call at the earliest opportunity within the dialer window. May occur immediately or on the next business day depending on settin…”
   • [claimed-docs] “Imports a phone number from your own SIP trunk provider. This allows you to use existing phone numbers with telli by configuring custom SIP …”
   • [claimed-docs] “Webhooks push data from telli to your systems in real time. When something happens on a call, telli sends a POST request to the endpoint you…”
   • [claimed-docs] “Keep your public business number and forward incoming calls to a telli number. You configure the forwarding with your phone provider or PBX.”
   • [claimed-docs] “Connect an existing number and phone system for incoming and outgoing calls. You need SIP trunk access from your provider.”
   • [probe] “PROBE openapi: HTTP 200 at https://docs.telli.com/openapi.json — contains "openapi" key”

   proof ↗⚑ flag

Sip

1. platform-engineerConnect my own carrier or PBX via SIP trunking (or import Twilio/Telnyx numbers) instead of being locked to bundled telephony

   weight 2 · round to telli
   LiveKit Agents✓fullclaimed7/10

   LiveKit's SIP-based telephony stack lets agents make/receive calls via inbound and outbound SIP trunks, which is the standard mechanism for bringing your own carrier or PBX (docs-26, docs-32, gh-2, docs-31). This is architecture-agnostic SIP, not a bundled/proprietary telephony lock-in, satisfying the platform-engineer's need to connect external trunks. Missing for 10: explicit documented walkthroughs or examples of importing Twilio/Telnyx numbers specifically, and no independent/hands-on confirmation of a real carrier trunk connection succeeding.

   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [claimed-docs] “Outbound trunks are used to place outgoing calls.”
   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”
   • [claimed-docs] “Enable your voice agent to make or take phone calls.”

   proof ↗⚑ flag

   telli✓fullclaimed8/10

   Docs explicitly support importing numbers from a customer's own SIP trunk provider via a dedicated API endpoint, plus general SIP trunk connection for incoming/outgoing calls and call forwarding from an existing PBX/provider number. This directly matches the platform-engineer's need to bring their own carrier rather than being locked into bundled telephony. Missing for 10: no explicit mention of native Twilio/Telnyx number import (only generic SIP trunk import), and no independent/hands-on confirmation of the SIP trunk setup working in practice.

   • [claimed-docs] “Imports a phone number from your own SIP trunk provider. This allows you to use existing phone numbers with telli by configuring custom SIP …”
   • [claimed-docs] “Keep your public business number and forward incoming calls to a telli number. You configure the forwarding with your phone provider or PBX.”
   • [claimed-docs] “Connect an existing number and phone system for incoming and outgoing calls. You need SIP trunk access from your provider.”

   proof ↗⚑ flag

Analytics

1. ai-native userThe platform's AI reviews my calls for me — scoring quality, flagging failures, and analyzing resolution automatically

   weight 2 · round to telli
   LiveKit Agents!disputedcontradicted3/10

   LiveKit ships a dev-time testing framework with 'judges' and agent simulations to evaluate scripted interactions (docs-9/10/25, gh-4), but this is pre-deployment test tooling, not automatic scoring/flagging/resolution-analysis of live production calls. A hands-on community report explicitly says operators running real agents on LiveKit have no built-in way to measure latency or cost per call, let alone automated quality/resolution review, contradicting the idea that the platform reviews calls for you. Missing for 10: production call-level QA scoring, automatic failure flagging on real traffic, resolution/outcome analysis dashboards.

   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
   • [claimed-docs] “Agent simulations run end-to-end conversations between your agent and an LLM-driven user, then evaluate the results across the full interact…”
   • [github] “Builtin test framework: Write tests and use judges to ensure your agent is performing as expected.”
   • [community] “We've been running multiple voice AI agents on LiveKit and kept running into visibility issues — no way to measure TTFT, latency across STT …”

   proof ↗⚑ flag

   telli~partialclaimed5/10

   telli provides analytics like reach rate, success rate, call outcome fields, transcripts, and manual feedback submission for agent improvement, but there's no evidence of automated AI-driven quality scoring, failure flagging, or resolution analysis performed autonomously by the platform's AI — feedback and analysis appear human-driven rather than AI-automated review. missing for 10: automated AI scoring of call quality, automatic failure/anomaly flagging, AI-generated resolution analysis without human review.

   • [claimed-docs] “Track key metrics like reach rate, call duration, and success rate.”
   • [claimed-docs] “Analyze individual calls, transcripts, and call outcome fields.”
   • [claimed-docs] “Submit feedback on calls to continuously improve your agent.”

   proof ↗⚑ flag

2. founderSee call analytics — success rates, durations, outcomes, sentiment — in dashboards without building my own

   weight 2 · round to telli
   LiveKit Agents—none0/10

   The evidence pack shows testing/simulation tools (pytest/Vitest, LLM-driven simulations) but no built-in production analytics dashboard for success rates, durations, outcomes, or sentiment. A community report explicitly confirms this gap — users running LiveKit voice agents said they had 'no way to measure TTFT, latency across STT→LLM→TTS, or even cost per call' and found debugging painful, indicating no such dashboard exists out of the box.

   • [community] “We've been running multiple voice AI agents on LiveKit and kept running into visibility issues — no way to measure TTFT, latency across STT …”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
   • [claimed-docs] “Agent simulations run end-to-end conversations between your agent and an LLM-driven user, then evaluate the results across the full interact…”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes... Agent simulations run end-to-end conversations between your agent and…”

   proof ↗⚑ flag

   telli~partialclaimed6/10

   telli's docs describe built-in dashboards tracking reach rate, call duration, and success rate, plus per-call analysis of transcripts and outcome fields, and a live monitoring overview—covering most of the story without needing custom tooling. However, there is no explicit mention of sentiment analysis in dashboards. missing for 10: sentiment scoring/analysis in dashboards, independent/hands-on corroboration of dashboard usability.

   • [claimed-docs] “Track key metrics like reach rate, call duration, and success rate.”
   • [claimed-docs] “Analyze individual calls, transcripts, and call outcome fields.”
   • [claimed-docs] “Live Monitoring > Real-time call overview of your contacts”

   proof ↗⚑ flag

Monitoring

1. platform-engineerMonitor live calls in production and get alerts when agents misbehave or error rates spike

   weight 1 · round to telli
   LiveKit Agents—none0/10

   No evidence pack items describe production monitoring dashboards, error-rate alerting, or call-quality observability tooling for LiveKit Agents; the only related evidence is a community report explicitly describing the lack of visibility into TTFT, per-call latency, and cost — a gap, not a delivered capability.

   • [community] “We've been running multiple voice AI agents on LiveKit and kept running into visibility issues — no way to measure TTFT, latency across STT …”

   proof ↗⚑ flag

   telli~partialclaimed5/10

   telli offers a real-time Live Monitoring call overview (telli-docs-32) and webhooks that push real-time notifications on call events/status/outcomes (telli-docs-13, telli-docs-21), which a platform engineer could wire into an alerting pipeline, plus metric tracking like success/reach rate (telli-docs-7). However there is no documented built-in alerting mechanism, error-rate-spike detection, or agent-misbehavior alerting system. Missing for 10: native alert/notification thresholds, anomaly or error-rate-spike detection, and evidence of alert delivery (email/Slack/PagerDuty) tied to live monitoring.

   • [claimed-docs] “Live Monitoring > Real-time call overview of your contacts”
   • [claimed-docs] “Webhooks: Receive real-time notifications about call events, status changes, and outcomes.”
   • [claimed-docs] “Webhooks push data from telli to your systems in real time. When something happens on a call, telli sends a POST request to the endpoint you…”
   • [claimed-docs] “Track key metrics like reach rate, call duration, and success rate.”

   proof ↗⚑ flag

Testing

1. developerTest agents with simulated conversations or evals before putting them on real phone calls

   weight 2 · round to LiveKit Agents
   LiveKit Agents✓fullclaimed8/10

   Docs explicitly describe both behavioral tests (pytest/Vitest) and agent simulations that run end-to-end conversations with an LLM-driven user, evaluating results before deployment, and GitHub notes a 'builtin test framework' with judges — directly matching the testing-before-real-calls story, complemented by separate telephony/SIP support for real calls. Missing for 10: independent/hands-on validation of the eval/simulation framework and explicit documentation tying test simulations to pre-phone-call validation workflows.

   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
   • [claimed-docs] “Agent simulations run end-to-end conversations between your agent and an LLM-driven user, then evaluate the results across the full interact…”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes... Agent simulations run end-to-end conversations between your agent and…”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes... using pytest (Python) or Vitest (Node.js)”
   • [github] “Builtin test framework: Write tests and use judges to ensure your agent is performing as expected.”
   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”
   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”

   proof ↗⚑ flag

   telli~partialclaimed5/10

   telli's agent builder includes a 'Test call' feature to trial a draft or saved agent version and review the recording/transcript side-by-side before going live, which covers pre-production testing, and feedback submission helps iterate on agent quality. However, there is no evidence of simulated/synthetic conversation runs, automated eval suites, or scripted test scenarios independent of an actual phone call. Missing for 10: automated evals framework, non-call simulated conversation testing, batch/regression testing tools.

   • [claimed-docs] “In the agent builder, click Test call to open the dialog. Pick the agent version (your draft with unsaved changes or any saved revision)...”
   • [claimed-docs] “Use the side-by-side view to listen to the recording while following along in the transcript.”
   • [claimed-docs] “Submit feedback on calls to continuously improve your agent.”

   proof ↗⚑ flag

Post call

1. developerExtract structured data from every call — outcomes, entities, dispositions — delivered via API or webhook after the call

   weight 2 · round to telli
   LiveKit Agents—none0/10

   LiveKit Agents provides tool-calling, MCP integration, and telephony hooks, but nothing in the evidence describes a built-in mechanism for automatically extracting structured call outcomes/entities/dispositions and delivering them via API or webhook after a call ends — that would have to be custom-built by the developer using the tool-calling primitives. Since no such capability is documented, this applicable axis is unmet.

   • [claimed-docs] “Call external APIs or lookup data for RAG.”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use. This feature allows you to create a custom library of tools to extend your agent's context”
   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [claimed-docs] “Outbound trunks are used to place outgoing calls.”

   proof ↗⚑ flag

   telli~partialclaimed6/10

   telli documents webhooks pushing real-time call event/status/outcome data and a REST API for programmatic access, plus conversation views showing appointment/outcome fields and call outcome analysis, supporting structured post-call data delivery. However, there's no explicit documentation of a structured schema for entities/dispositions extraction or example payloads showing the full breadth of structured fields promised. missing for 10: example webhook/API payload showing structured entities and disposition fields, documentation of a dedicated 'call outcomes' data model, independent/hands-on confirmation of data completeness.

   • [claimed-docs] “Webhooks: Receive real-time notifications about call events, status changes, and outcomes.”
   • [claimed-docs] “Webhooks push data from telli to your systems in real time. When something happens on a call, telli sends a POST request to the endpoint you…”
   • [claimed-docs] “Analyze individual calls, transcripts, and call outcome fields.”
   • [claimed-docs] “Conversations now show appointments saved during a call, including the time, status, hosts, and provider link.”
   • [claimed-docs] “API Endpoints: Use telli's REST API to programmatically manage contacts, schedule calls, and integrate with your existing systems.”

   proof ↗⚑ flag

Tools

1. ai-native userMy voice agent can plug in MCP servers as tool sources so one integration grants it whole toolsets mid-call

   weight 2 · round to LiveKit Agents
   LiveKit Agents✓fullclaimed9/10

   LiveKit Agents has first-class, documented MCP support ("first-class support for Model Context Protocol (MCP) servers", "Integrate tools provided by MCP servers with one line of code") via `MCPToolset`, which wraps an MCP server and passes it directly to the agent's tools parameter — enabling mid-call toolset access for voice agents. This is corroborated across both docs and GitHub README. Missing for 10: independent hands-on validation of actual mid-call MCP toolset usage beyond vendor docs.

   • [claimed-docs] “Wrap an MCP server in an \`MCPToolset\` and pass it to the agent's \`tools\` parameter”
   • [claimed-docs] “Wrap an MCP server in an `MCPToolset` and pass it to the agent's `tools` parameter”
   • [claimed-docs] “LiveKit Agents has first-class support for Model Context Protocol (MCP) servers.”
   • [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
   • [github] “Native support for MCP. Integrate tools provided by MCP servers with one line of code.”

   proof ↗⚑ flag

   telli—none0/10

   telli documents custom tools connecting to external APIs (telli-docs-20) and even an MCP server (telli-docs-22, telli-probe-3), but that MCP server lets external AI tools like Claude/ChatGPT talk to telli's Charlie assistant — it is telli acting as an MCP server, not the voice agent consuming external MCP servers as a tool source mid-call. There is no evidence the voice agent itself can plug in MCP servers to gain toolsets during a call.

   • [claimed-docs] “The telli MCP server lets AI tools like Claude, ChatGPT, and Codex talk to Charlie, telli's AI assistant, over the Model Context Protocol.”
   • [probe] “official MCP server documented at https://docs.telli.com/integrations/mcp”
   • [claimed-docs] “Custom tools allow your AI agents to access external information by connecting to your APIs.”

   proof ↗⚑ flag

2. developerMy agent can call external APIs and custom functions mid-conversation and speak the result without awkward dead air

   weight 3 · round to LiveKit Agents
   LiveKit Agents✓fullprobed9/10

   LiveKit Agents has full documented support for LLM tool/function calling (custom Python/Node functions calling external APIs mid-conversation) plus MCP server tool integration, combined with adaptive interruption handling and turn-detection to avoid awkward dead air while tools execute. Runtime probes confirm the framework is real and installable, corroborating the docs claims. missing for 10: independent/hands-on evidence specifically showing tool-call latency handled gracefully in a live conversation (only vendor docs cover this exact combination).

   • [claimed-docs] “Call external APIs or lookup data for RAG.”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use. This feature allows you to create a custom library of tools to extend your agent's context”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use.”
   • [claimed-docs] “LiveKit Agents has first-class support for Model Context Protocol (MCP) servers.”
   • [claimed-docs] “Wrap an MCP server in an \`MCPToolset\` and pass it to the agent's \`tools\` parameter”
   • [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response... to identify intentional interruptions (…”
   • [claimed-docs] “the model analyzes the acoustic signals to identify intentional interruptions (barge-ins) from conversational backchanneling”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”

   proof ↗⚑ flag

   telli~partialclaimed6/10

   Docs confirm custom tools let agents call external APIs mid-call (telli-docs-20) and built-in tool calls like web_search operate during live calls (telli-docs-30), plus transfer tools show mid-call action execution (telli-docs-31). However, there is no documentation addressing latency masking or filler-speech to avoid 'dead air' while awaiting API responses. missing for 10: explicit evidence of latency-hiding/filler speech during tool calls, independent hands-on confirmation of smooth voice continuity.

   • [claimed-docs] “Custom tools allow your AI agents to access external information by connecting to your APIs.”
   • [claimed-docs] “For a Duo agent, open Tools > System and enable web_search to let the agent search the web for current public information during calls.”
   • [claimed-docs] “Transfer tools | Move the call to another agent or number.”

   proof ↗⚑ flag

Recording

1. platform-engineerRetrieve full call recordings and transcripts programmatically for every call

   weight 2 · round to telli
   LiveKit Agents—none0/10

   The evidence pack covers agent building, tool use, MCP, telephony, and testing but contains no mention of recording APIs, transcript storage, or programmatic retrieval of call recordings/transcripts for platform engineers.

   ⚑ flag

   telli~partialprobed5/10

   Docs confirm transcripts and recordings exist and can be reviewed in-app (side-by-side view, consent-based recording) and telli exposes a general REST API plus webhooks for call events, but no endpoint is documented that explicitly returns full call recordings or transcripts programmatically for every call. missing for 10: a documented GET/recording or GET/transcript API endpoint, confirmation of bulk/enumerable retrieval, and independent verification that recordings are accessible via the API rather than only in the UI.

   • [claimed-docs] “Analyze individual calls, transcripts, and call outcome fields.”
   • [claimed-docs] “Use the side-by-side view to listen to the recording while following along in the transcript.”
   • [claimed-docs] “In telli, you can easily make audio recording conditional on the caller's consent — so a call is only recorded once they agree.”
   • [claimed-docs] “API Endpoints: Use telli's REST API to programmatically manage contacts, schedule calls, and integrate with your existing systems.”
   • [claimed-docs] “Webhooks: Receive real-time notifications about call events, status changes, and outcomes.”
   • [claimed-docs] “Webhooks push data from telli to your systems in real time. When something happens on a call, telli sends a POST request to the endpoint you…”
   • [probe] “PROBE openapi: HTTP 200 at https://docs.telli.com/openapi.json — contains "openapi" key”

   proof ↗⚑ flag

Transcription

1. developerGet accurate real-time transcription with control over the STT provider, language models, or key terms

   weight 2 · round to LiveKit Agents
   LiveKit Agents~partialclaimed6/10

   LiveKit Agents supports swapping STT/LLM providers via its open-source plugin ecosystem and lets you override API key/base URL for any provider (docs-12, docs-24, docs-35), giving real control over transcription and language-model choice. However there is no evidence of support for custom vocabulary/key-term or hotword boosting in STT, and no independent benchmark of transcription accuracy. Missing for 10: key-term/hotword/phrase-hint configuration support, accuracy benchmarks or independent corroboration of transcription quality.

   • [claimed-docs] “LiveKit Agents includes a large ecosystem of open source plugins for a variety of AI providers.”
   • [claimed-docs] “You can use LiveKit Inference to access many of these models directly through LiveKit Cloud, or you can use the open source plugins to conne…”
   • [claimed-docs] “For any provider not included, you can override the API key and base URL at initialization for the LLM, STT, and TTS interfaces in the plugi…”

   proof ↗⚑ flag

   telli—none0/10

   telli documents transcript viewing and side-by-side playback (telli-docs-29) but provides no evidence of developer-facing control over STT provider selection, choice of underlying language models, or custom key-term/vocabulary boosting for transcription accuracy.

   • [claimed-docs] “Use the side-by-side view to listen to the recording while following along in the transcript.”

   proof ↗⚑ flag

Voices

1. founderClone a custom brand voice and use it for my agents, with a documented consent process

   weight 2 · round to telli
   LiveKit Agents—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   telli~partialclaimed5/10

   Voice cloning is documented as a real feature (telli-docs-27) letting founders give agents a custom voice, but the pack's only consent-related documentation concerns call recording consent (telli-docs-26), not a consent process specifically for cloning someone's voice. Missing for 10: explicit documented consent/authorization workflow for the voice being cloned (e.g., proof of rights from the voice owner), and any legal/compliance guardrails for cloning third-party voices.

   • [claimed-docs] “Voice cloning lets you give your agent a custom voice (your own, a colleague's, or a professional voice actor's), so the agent sounds exactl…”
   • [claimed-docs] “In telli, you can easily make audio recording conditional on the caller's consent — so a call is only recorded once they agree.”

   proof ↗⚑ flag

2. developerChoose from a broad voice library or plug in multiple TTS providers to get the voice I want

   weight 2 · round to LiveKit Agents
   LiveKit Agents~partialclaimed6/10

   Docs confirm a large open-source plugin ecosystem for TTS/STT/LLM providers, plus LiveKit Inference for provider access, and an override mechanism for API key/base URL for providers not natively included, supporting multi-provider TTS flexibility. However, there's no explicit mention of a 'voice library' (e.g., curated voice catalog/selection UI) or enumeration of specific TTS providers/voices, so the 'broad voice library' half of the story is unevidenced. Missing for 10: explicit voice catalog/list of supported TTS providers and voices, evidence of ease of switching between TTS voices, independent confirmation of provider breadth.

   • [claimed-docs] “LiveKit Agents includes a large ecosystem of open source plugins for a variety of AI providers.”
   • [claimed-docs] “You can use LiveKit Inference to access many of these models directly through LiveKit Cloud, or you can use the open source plugins to conne…”
   • [claimed-docs] “For any provider not included, you can override the API key and base URL at initialization for the LLM, STT, and TTS interfaces in the plugi…”

   proof ↗⚑ flag

   telli~partialclaimed3/10

   Docs mention voice cloning to give an agent a custom voice, but there is no evidence of a curated voice library to choose from or the ability to plug in multiple external TTS providers as the story requires. missing for 10: evidence of a selectable voice library, multi-provider TTS integration, developer API/config for switching TTS engines.

   • [claimed-docs] “Voice cloning lets you give your agent a custom voice (your own, a colleague's, or a professional voice actor's), so the agent sounds exactl…”

   proof ↗⚑ flag