Bland vs LiveKit Agents

Agent ops

1. ai-native userMy coding agent can provision a complete voice agent end to end — create the agent, attach a number, and place a call — through the API, CLI, or MCP without touching the dashboard

   weight 3 · round to Bland
   Bland✓fullprobed8/10

   Bland's docs show all three provisioning steps available programmatically: pathway/agent creation via API (bland-docs-1, bland-docs-18), phone number acquisition/porting/Twilio and SIP attachment (bland-docs-3, bland-docs-16, bland-docs-17), and call placement via API or batch calls (bland-docs-6, bland-docs-23). Both the CLI ('make calls, build and test pathways, configure phone numbers' — bland-docs-12) and the MCP server ('place and inspect calls, build and validate pathways, manage agents' — bland-docs-11/20/27) explicitly cover the full create-agent/attach-number/place-call lifecycle without the dashboard, and runtime probes confirm both the CLI and hosted MCP endpoint are live and functional (bland-probe-rt-1, bland-probe-rt-2). Missing for 10: a single consolidated end-to-end tutorial/example walking through create→attach→call in one flow, and independent (non-vendor) confirmation of the full pipeline working end-to-end.

   • [claimed-docs] “See the Pathways API reference to create and manage pathways programmatically.”
   • [claimed-docs] “Users can connect their own Twilio account to Bland, easily bring over your existing phone numbers to use within the platform.”
   • [claimed-docs] “Batch calls let you initiate high-volume call campaigns by uploading a list of recipients via CSV.”
   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
   • [claimed-docs] “Manage your entire Bland account from the terminal: make calls, build and test pathways, configure phone numbers, and more.”
   • [claimed-docs] “Number porting to bring existing numbers to Bland”
   • [claimed-docs] “Bland supports inbound and outbound SIP. Point calls from your carrier or PBX at Bland to be answered by an agent, or have Bland place calls…”
   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y bland-cli --version` printed `0.6.2` keylessly — the official npm CLI (bland-cli, with dev term…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP endpoint https://api.bland.ai/v1/mcp returned HTTP 4…”

   proof ↗⚑ flag

   LiveKit Agents~partialprobed5/10

   LiveKit provides a real CLI (`lk`), a documented telephony/SIP stack for provisioning inbound/outbound trunks and placing calls, and framework docs explicitly note it's 'built for coding agents like Claude Code, Cursor, and Codex,' suggesting API/CLI-driven workflows are possible without the dashboard. However, the MCP support documented is for the agent's own tool-calling (consuming MCP servers as tools), not an MCP server exposing agent-provisioning/number-attachment/call-placing actions to a coding agent, and no single evidenced workflow shows an agent programmatically creating an agent, attaching a number, and placing a call end-to-end via CLI/API in one flow — the Agent Builder path shown is dashboard/browser-based (no-code), which contradicts the 'without touching dashboard' framing. Missing for 10: an MCP server (or CLI/API recipe) that lets a coding agent itself create an agent, provision/attach a phone number, and place a call in one documented end-to-end sequence.

   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”
   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [claimed-docs] “Outbound trunks are used to place outgoing calls.”
   • [probe] “official CLI documented at https://github.com/livekit/livekit-cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): `lk --version` printed `lk version 2.18.6` after a plain `brew install livekit-cli` — the official CLI …”
   • [claimed-docs] “LiveKit is built for coding agents like Claude Code, Cursor, and Codex.”
   • [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
   • [claimed-docs] “LiveKit Agents has first-class support for Model Context Protocol (MCP) servers.”

   proof ↗⚑ flag

2. ai-native userThe platform's own AI helps me author agents — generating or improving prompts, flows, and test cases from a description

   weight 1 · round to Bland
   Bland~partialclaimed4/10

   Bland's MCP server lets external AI coding agents 'build and validate pathways ... through natural language' (bland-docs-11/20/27), and the testbed/evals tools support iterating on and grading prompts, which loosely supports AI-assisted authoring. However there is no evidence of a built-in, first-party generative feature where Bland's own platform AI drafts a full pathway/flow/test-cases from a plain-language description inside the product itself — the closest capability requires an external AI agent connecting via MCP. Missing for 10: a native 'describe your agent, we generate the pathway/prompts/tests' feature, in-product prompt-improvement AI, and independent/hands-on confirmation of AI-generated flows.

   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
   • [claimed-docs] “The testbed lets you take any call ... isolate a specific node interaction, edit the prompt, and run it multiple times to see how the output…”
   • [claimed-docs] “The testbed lets you take any call — whether it's a historical call from call logs or a test chat from the pathway editor — isolate a specif…”
   • [claimed-docs] “Evals let you measure the quality of your calls with LLM judges.”
   • [claimed-docs] “Evals let you measure the quality of your calls with LLM judges. You define eval agents, each of which grades one dimension of a call”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   Evidence shows a no-code 'Agent Builder' for browser prototyping and a testing framework, but nothing indicates the platform itself uses AI to generate or improve prompts, flows, or test cases from a natural-language description of the desired agent.

   • [claimed-docs] “LiveKit Agent Builder to prototype and deploy agents directly in your browser without writing code”
   • [claimed-docs] “LiveKit Agent Builder: Prototype and deploy voice agents directly in your browser, without writing any code.”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes... Agent simulations run end-to-end conversations between your agent and…”

   proof ↗⚑ flag

Build

1. developerBuild a working phone voice agent — prompt, voice, and phone number — and take my first live call within an hour

   weight 3 · round to Bland
   Bland✓fullprobed8/10

   Docs cover the whole first-call workflow: creating a pathway/prompt (bland-docs-18, bland-docs-26, bland-docs-30), voice cloning/selection (bland-docs-10), phone number setup via own Twilio or new inbound numbers (bland-docs-3, bland-docs-24), and dispatching outbound calls (bland-docs-23), plus a CLI and MCP server confirmed live at runtime (bland-probe-rt-1, bland-probe-rt-2) that let a developer configure and place calls quickly. Missing for 10: no independent hands-on account of a developer actually completing a first call within an hour, and one community comment notes cost concerns rather than time-to-first-call, so onboarding speed is only documented, not externally verified.

   • [claimed-docs] “Conversational pathways are our new way of prompting Bland that has led to major breakthroughs in realism. Give your agent instructions on h…”
   • [claimed-docs] “Conversational pathways are our new way of prompting Bland that has led to major breakthroughs in realism.”
   • [claimed-docs] “Give your agent instructions on how it should respond at specific points of the conversation. Choose between prompting or fixed sentences. E…”
   • [claimed-docs] “A clone needs one clean sample of about ten seconds. Quality of the sample sets the ceiling on quality of the voice”
   • [claimed-docs] “Users can connect their own Twilio account to Bland, easily bring over your existing phone numbers to use within the platform.”
   • [claimed-docs] “Create inbound phone numbers for customer support, etc.”
   • [claimed-docs] “Dispatch AI phone calls to call customers, leads, and to streamline operations.”
   • [claimed-docs] “The API integration lets you connect your AI agent to any HTTP endpoint.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y bland-cli --version` printed `0.6.2` keylessly — the official npm CLI (bland-cli, with dev term…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP endpoint https://api.bland.ai/v1/mcp returned HTTP 4…”

   proof ↗⚑ flag

   LiveKit Agents✓fullprobed7/10

   Docs show a <10‑minute voice-assistant quickstart (prompt+voice) plus a dedicated telephony/SIP stack for inbound/outbound calls (trunks, phone number), and runtime probes confirm the framework and self-hosted server actually install and boot without extra keys. Together these cover prompt, voice, and phone number needed for a first live call within an hour. Missing for 10: a single unified, hands-on-verified tutorial that walks through phone-number provisioning and first live call end-to-end (currently voice-quickstart and telephony docs are separate), and independent (non-vendor) confirmation of the 'within an hour' timeline.

   • [claimed-docs] “Build and deploy a simple voice assistant in less than 10 minutes.”
   • [claimed-docs] “Build and deploy a simple voice assistant with Python or Node.js in less than 10 minutes.”
   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [claimed-docs] “Enable your voice agent to make or take phone calls.”
   • [claimed-docs] “Outbound trunks are used to place outgoing calls.”
   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `lk --version` printed `lk version 2.18.6` after a plain `brew install livekit-cli` — the official CLI …”
   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”

   proof ↗⚑ flag

2. developerRun conversations in multiple languages, including detecting and switching language mid-call

   weight 2 · round drawn
   Bland—none0/10

   The evidence pack contains no mention of multi-language support, language detection, or mid-call language switching anywhere in Bland's docs; only pathways, TTS voice cloning, and infrastructure features are documented. Missing for 10: any documentation of multilingual conversation support, automatic language detection, or dynamic language switching mid-call.

   ⚑ flag

   LiveKit Agents—none0/10

   No evidence in the pack addresses multilingual conversation support, language detection, or dynamic language switching mid-call; the docs cover turn detection, interruption handling, tool use, MCP, and telephony but never mention STT/TTS language selection or switching logic. This is a fair axis for a voice-agent framework (many STT/TTS providers support multi-language), so absence of evidence means 'none' rather than 'na'.

   ⚑ flag

3. founderDesign multi-step conversation flows in a visual builder with branching, states, and handoffs without writing code

   weight 2 · round to Bland
   Bland✓fullclaimed7/10

   Bland's 'Conversational Pathways' feature is explicitly a node-based flow builder where founders give instructions at specific conversation points, test individual node interactions in a pathway editor/testbed, and publish drafts separately from production—matching branching/states/handoffs without code (bland-docs-18, 26, 30, 28, 19). Missing for 10: explicit confirmation of a drag-and-drop visual canvas UI (docs describe 'nodes' and 'pathway editor' but no screenshot/UI walkthrough) and independent hands-on corroboration beyond vendor docs.

   • [claimed-docs] “Conversational pathways are our new way of prompting Bland that has led to major breakthroughs in realism. Give your agent instructions on h…”
   • [claimed-docs] “Conversational pathways are our new way of prompting Bland that has led to major breakthroughs in realism.”
   • [claimed-docs] “Give your agent instructions on how it should respond at specific points of the conversation. Choose between prompting or fixed sentences. E…”
   • [claimed-docs] “The testbed lets you take any call — whether it's a historical call from call logs or a test chat from the pathway editor — isolate a specif…”
   • [claimed-docs] “When you edit a pathway, you are working on a draft. Live calls are not affected. They keep using the published production version.”
   • [claimed-docs] “See the Pathways API reference to create and manage pathways programmatically.”

   proof ↗⚑ flag

   LiveKit Agents~partialclaimed3/10

   LiveKit mentions a browser-based 'Agent Builder' for no-code prototyping of voice agents, but the evidence never describes visual branching, explicit states, or handoff design — the core framework is fundamentally code-first (Python/Node.js) with tools, MCP, and turn-detection logic. missing for 10: documentation of branching/state UI, handoff modeling, or any screenshots/examples of the Agent Builder's flow-design capabilities beyond a vague no-code prototyping claim.

   • [claimed-docs] “Prototype and deploy voice agents directly in your browser, without writing any code.”
   • [claimed-docs] “LiveKit Agent Builder to prototype and deploy agents directly in your browser without writing code”
   • [claimed-docs] “LiveKit Agent Builder: Prototype and deploy voice agents directly in your browser, without writing any code.”

   proof ↗⚑ flag

Personalization

1. developerInject dynamic variables and per-caller context at call time so each conversation is personalized

   weight 2 · round to Bland
   Bland~partialclaimed5/10

   Evidence shows personalization mechanisms exist—Memory for per-caller context (bland-docs-9), batch calls that likely carry per-recipient data (bland-docs-6), and pathway webhooks/API integrations that could fetch live data (bland-docs-4, bland-docs-30)—but there is no explicit documentation of a 'dynamic variables' injection API or call-time variable substitution mechanism. missing for 10: explicit dynamic-variable/request_data injection documentation, examples of per-call variable interpolation into prompts, independent confirmation of personalization working in practice.

   • [claimed-docs] “Memory lets Bland agents remember people across conversations, including calls, SMS, and other channels, so each interaction feels continuou…”
   • [claimed-docs] “Batch calls let you initiate high-volume call campaigns by uploading a list of recipients via CSV.”
   • [claimed-docs] “The API integration lets you connect your AI agent to any HTTP endpoint.”
   • [claimed-docs] “Give your agent instructions on how it should respond at specific points of the conversation. Choose between prompting or fixed sentences. E…”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   The evidence pack covers tool use, MCP, telephony, testing, and turn detection, but nothing addresses injecting dynamic variables or per-caller context (e.g., participant/room metadata, job context) into an agent's prompt or session at call start. Missing for 10: any documentation of job/participant metadata APIs, per-call context injection, or dynamic prompt personalization.

   ⚑ flag

2. developerGround the agent on my documents with a built-in knowledge base or RAG so it answers from my content

   weight 2 · round to LiveKit Agents
   Bland—none0/10

   Bland is a voice AI/phone agent platform; evidence covers pathways, memory, tools, MCP, CLI, etc., but nowhere mentions a document knowledge base or RAG capability for grounding agent responses in uploaded content. Since a voice/conversational agent platform could plausibly ship this (many competitors do), absence of evidence makes this 'none' rather than 'na'.

   ⚑ flag

   LiveKit Agents~partialclaimed4/10

   LiveKit Agents supports calling external APIs/tools that a developer can use to implement RAG ("Call external APIs or lookup data for RAG"), and has full LLM tool-use and MCP integration for wiring in retrieval systems, but there is no built-in knowledge base, document ingestion, or vector-store/RAG pipeline shipped by the framework itself — developers must bring their own RAG implementation via the tools API. Missing for 10: built-in document indexing/vector store, out-of-the-box knowledge-base feature, first-party RAG pipeline or example showing document grounding end-to-end.

   • [claimed-docs] “Call external APIs or lookup data for RAG.”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use. This feature allows you to create a custom library of tools to extend your agent's context”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use.”
   • [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
   • [claimed-docs] “Wrap an MCP server in an \`MCPToolset\` and pass it to the agent's \`tools\` parameter”

   proof ↗⚑ flag

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round drawn
   Bland✓fullprobed8/10

   Bland has a confirmed live llms.txt at docs.bland.ai/llms.txt (HTTP 200) providing agent-oriented documentation, plus an official MCP server and CLI explicitly designed for AI coding agents to interact with the platform via natural language. missing for 10: independent third-party confirmation that agents actually consume llms.txt successfully in practice, and broader agent-oriented docs beyond the single llms.txt file.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.bland.ai/llms.txt # Bland Documentation Bland is an enterprise voice AI platform for high-volume, …”
   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
   • [probe] “official MCP server documented at https://docs.bland.ai/integrations/mcp/overview”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP endpoint https://api.bland.ai/v1/mcp returned HTTP 4…”

   proof ↗⚑ flag

   LiveKit Agents✓fullprobed8/10

   LiveKit publishes a working llms.txt (verified live at docs.livekit.io/llms.txt, HTTP 200) and agent-friendly markdown doc endpoints (e.g. /agents/.md), plus explicit docs noting the platform is 'built for coding agents like Claude Code, Cursor, and Codex,' confirming intentional support for AI-native doc consumption. missing for 10: no independent third-party confirmation that external agents successfully consume these endpoints in practice, and no dedicated documentation page explaining the llms.txt/agent-doc strategy itself.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.livekit.io/llms.txt # LiveKit docs > LiveKit is an open-source platform for building voice, video,…”
   • [probe] “PROBE docs-md: HTTP 200 at https://docs.livekit.io/agents/.md LiveKit docs › Build Agents › Get Started › Introduction --- # Introduction …”
   • [claimed-docs] “LiveKit is built for coding agents like Claude Code, Cursor, and Codex.”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round drawn
   Bland✓fullprobed8/10

   Bland offers a full programmatic/REST API for pathways, calls, batch calls, webhooks, and evals, plus an official CLI (verified runtime installable keylessly via npx) for terminal-based automation, enabling headless/CI usage. missing for 10: no explicit CI/CD pipeline example (e.g., GitHub Actions) or independent case study confirming CI usage beyond docs and CLI probe.

   • [claimed-docs] “See the Pathways API reference to create and manage pathways programmatically.”
   • [claimed-docs] “Batch calls let you initiate high-volume call campaigns by uploading a list of recipients via CSV.”
   • [claimed-docs] “Manage your entire Bland account from the terminal: make calls, build and test pathways, configure phone numbers, and more.”
   • [probe] “official CLI documented at https://docs.bland.ai/sdks/cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y bland-cli --version` printed `0.6.2` keylessly — the official npm CLI (bland-cli, with dev term…”
   • [claimed-docs] “Evals let you measure the quality of your calls with LLM judges.”

   proof ↗⚑ flag

   LiveKit Agents✓fullprobed8/10

   LiveKit Agents is a Python/Node.js SDK plus self-hostable server that runs as a headless programmatic participant with no UI dependency; it's pip/npm-installable, has a built-in test framework (pytest/Vitest, agent simulations) suitable for CI, and the server/CLI were verified runtime to work keylessly. missing for 10: no explicit documented CI pipeline example (e.g., GitHub Actions config) or headless-mode confirmation beyond inference from server/testing docs.

   • [claimed-docs] “A programmatic participant is any code that joins a LiveKit room as a participant — this includes AI agents, media processors, or custom log…”
   • [claimed-docs] “The Agents framework isn't limited to AI agents. You can use it to deploy any code that needs to process realtime media and data streams as …”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
   • [claimed-docs] “Agent simulations run end-to-end conversations between your agent and an LLM-driven user, then evaluate the results across the full interact…”
   • [github] “Builtin test framework: Write tests and use judges to ensure your agent is performing as expected.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round to LiveKit Agents
   Bland—none0/10

   Bland's evidence only documents it exposing an outbound MCP server so that external AI coding agents can call Bland's own tools (docs-11, docs-20, docs-27, probe-rt-2) — the reverse direction of this story. There is no evidence that Bland itself can consume/plug in third-party MCP servers as a client; its tool integration story is limited to custom HTTP API endpoints and webhooks (bland-docs-4, bland-docs-25).

   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
   • [claimed-docs] “The API integration lets you connect your AI agent to any HTTP endpoint.”
   • [claimed-docs] “Connect external APIs and take live actions during phone calls.”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP endpoint https://api.bland.ai/v1/mcp returned HTTP 4…”

   proof ↗⚑ flag

   LiveKit Agents✓fullclaimed8/10

   Docs and GitHub explicitly state 'first-class support for Model Context Protocol (MCP) servers' with a documented pattern to wrap an MCP server in an MCPToolset and pass it to the agent's tools parameter, described as one line of code integration. This is first-party documentation without independent hands-on verification of MCP-specific usage. Missing for 10: independent/community confirmation of MCP tool integration working end-to-end, and more detail on multi-server or auth configurations.

   • [claimed-docs] “Wrap an MCP server in an \`MCPToolset\` and pass it to the agent's \`tools\` parameter”
   • [claimed-docs] “Wrap an MCP server in an `MCPToolset` and pass it to the agent's `tools` parameter”
   • [claimed-docs] “LiveKit Agents has first-class support for Model Context Protocol (MCP) servers.”
   • [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
   • [github] “Native support for MCP. Integrate tools provided by MCP servers with one line of code.”

   proof ↗⚑ flag

4. ai-native userConnect an agent via an official MCP server

   weight 3 · round to Bland
   Bland✓fullprobed8/10

   Bland ships an official MCP server (docs and runtime probe confirm it's live and gated by API key) that lets AI coding agents place/inspect calls, build pathways, manage agents, query analytics, run evals, and search docs — directly fulfilling the story. Missing for 10: independent third-party hands-on review of the MCP server beyond vendor docs/probe.

   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
   • [probe] “official MCP server documented at https://docs.bland.ai/integrations/mcp/overview”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP endpoint https://api.bland.ai/v1/mcp returned HTTP 4…”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

5. ai-native userUse an official CLI

   weight 2 · round drawn
   Bland✓fullprobed9/10

   Bland ships an official CLI (bland-cli) documented to manage the entire account from the terminal, and a runtime probe confirms it installs and runs via npx keylessly. missing for 10: independent third-party review/usage reports of the CLI beyond the official docs and one probe run.

   • [claimed-docs] “Manage your entire Bland account from the terminal: make calls, build and test pathways, configure phone numbers, and more.”
   • [probe] “official CLI documented at https://docs.bland.ai/sdks/cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y bland-cli --version` printed `0.6.2` keylessly — the official npm CLI (bland-cli, with dev term…”

   proof ↗⚑ flag

   LiveKit Agents✓fullprobed9/10

   LiveKit ships an official CLI (livekit-cli), documented on GitHub and verified hands-on to install and run (`lk --version` works via brew install), confirming it's a real, functional official CLI supporting the agent workflow. missing for 10: no deep documentation of full CLI command surface for agent-specific workflows within the evidence pack.

   • [probe] “official CLI documented at https://github.com/livekit/livekit-cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): `lk --version` printed `lk version 2.18.6` after a plain `brew install livekit-cli` — the official CLI …”

   proof ↗⚑ flag

6. ai-native userDrive the product through a documented public API

   weight 3 · round to Bland
   Bland✓fullprobed9/10

   Bland documents a comprehensive public API (pathways, calls, tools, webhooks, evals, batch calls) plus SDKs, CLI, and an official MCP server, and runtime probes confirm the CLI installs and the hosted MCP endpoint is live and gated as documented, showing agentic programmatic control. Missing for 10: a formally published OpenAPI/Swagger spec (probe found 404s at standard OpenAPI paths), so machine-readable spec discoverability is unconfirmed.

   • [claimed-docs] “See the Pathways API reference to create and manage pathways programmatically.”
   • [claimed-docs] “The API integration lets you connect your AI agent to any HTTP endpoint.”
   • [claimed-docs] “Batch calls let you initiate high-volume call campaigns by uploading a list of recipients via CSV.”
   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
   • [claimed-docs] “Manage your entire Bland account from the terminal: make calls, build and test pathways, configure phone numbers, and more.”
   • [probe] “official MCP server documented at https://docs.bland.ai/integrations/mcp/overview”
   • [probe] “official CLI documented at https://docs.bland.ai/sdks/cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y bland-cli --version` printed `0.6.2` keylessly — the official npm CLI (bland-cli, with dev term…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP endpoint https://api.bland.ai/v1/mcp returned HTTP 4…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.bland.ai/openapi.json, https://docs.bland.ai/swagger.json, https://docs.bland.ai/api/op…”

   proof ↗⚑ flag

   LiveKit Agents✓fullprobed8/10

   LiveKit Agents ships a well-documented, pip/npm-installable Python and Node.js SDK with extensive public API surface (tools, MCP, turn detection, telephony, testing), plus a llms.txt and hands-on verified installs/runtime confirming the API is real and usable by AI-native developers. Missing for 10: independent third-party API reference/versioning audit and deeper evidence of API stability guarantees beyond docs and probes.

   • [claimed-docs] “The Agents framework lets you add any Python or Node.js program to LiveKit rooms as full realtime participants.”
   • [claimed-docs] “A programmatic participant is any code that joins a LiveKit room as a participant — this includes AI agents, media processors, or custom log…”
   • [claimed-docs] “The Agents framework isn't limited to AI agents. You can use it to deploy any code that needs to process realtime media and data streams as …”
   • [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use. This feature allows you to create a custom library of tools to extend your agent's context”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.livekit.io/llms.txt # LiveKit docs > LiveKit is an open-source platform for building voice, video,…”
   • [probe] “PROBE docs-md: HTTP 200 at https://docs.livekit.io/agents/.md LiveKit docs › Build Agents › Get Started › Introduction --- # Introduction …”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”

   proof ↗⚑ flag

7. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round drawn
   Bland—none0/10

   Evidence shows generic API-key auth and JWT-based webhook verification (bland-docs-15), but nothing about issuing scoped or least-privilege credentials specific to an agent's permissions (e.g., role-based API keys, scoped tokens limiting call/pathway/account access). Missing for 10: documentation of scoped API key creation, permission levels, or per-agent credential restriction.

   • [claimed-docs] “JWT signatures eliminate these risks through asymmetric cryptography - you verify requests using our public JWKS endpoint without storing an…”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   The evidence pack covers voice/agent features, tool-calling, MCP, testing, and telephony, but contains no mention of scoped or least-privilege API key/token issuance for agents (e.g., LiveKit's grant-based access tokens or credential scoping). Since LiveKit is a platform with API keys and would plausibly support such scoping, this is an applicable but undocumented axis in the given evidence.

   ⚑ flag

8. ai-native userBuild against official SDKs

   weight 2 · round to LiveKit Agents
   Bland~partialprobed6/10

   Bland documents a Web Agent SDK for embedding voice agents (React/Vanilla JS/Node), a CLI, and a REST API used throughout tutorials, giving AI-native developers concrete building blocks; runtime probes confirm the CLI installs and runs. However, no dedicated 'official SDK' page for server-side languages (Python/Node backend SDK) is evidenced, and openapi/swagger spec endpoints all 404, suggesting the API reference isn't machine-consumable in a standard SDK-generation format. Missing for 10: a clearly documented multi-language backend SDK (Python/Node) beyond the browser widget SDK, and a working OpenAPI spec for auto-generating clients.

   • [claimed-docs] “Embed a Bland voice agent into any web application (React, Vanilla JS, or Node). The SDK handles secure authentication between your server a…”
   • [claimed-docs] “Manage your entire Bland account from the terminal: make calls, build and test pathways, configure phone numbers, and more.”
   • [probe] “official CLI documented at https://docs.bland.ai/sdks/cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y bland-cli --version` printed `0.6.2` keylessly — the official npm CLI (bland-cli, with dev term…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.bland.ai/openapi.json, https://docs.bland.ai/swagger.json, https://docs.bland.ai/api/op…”
   • [claimed-docs] “See the Pathways API reference to create and manage pathways programmatically.”

   proof ↗⚑ flag

   LiveKit Agents✓fullprobed9/10

   LiveKit provides official Python and Node.js SDKs (livekit-agents), verified hands-on to pip-install and import with no key, plus documented plugin ecosystem, tool use, MCP support, testing frameworks, and telephony integration—clearly an official SDK ecosystem for AI-native building. Missing for 10: independent third-party benchmarking of SDK API stability/versioning beyond community latency complaints unrelated to SDK build story.

   • [claimed-docs] “The Agents framework lets you add any Python or Node.js program to LiveKit rooms as full realtime participants.”
   • [claimed-docs] “LiveKit Agents includes a large ecosystem of open source plugins for a variety of AI providers.”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use. This feature allows you to create a custom library of tools to extend your agent's context”
   • [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”

   proof ↗⚑ flag

9. ai-native userSubscribe to events via webhooks

   weight 2 · round to Bland
   Bland~partialclaimed6/10

   Bland documents post-call webhooks (automatic HTTP notifications sent after a call completes) and pathway-level webhook execution at specific conversation points, showing genuine event-driven webhook support tied to call lifecycle. However, evidence only covers call-related events (completion, in-call triggers) — missing for 10: a general-purpose event subscription/webhook management API covering other account events (e.g., evals, batch campaign status, pathway publishes), and any independent confirmation of webhook reliability/configuration options.

   • [claimed-docs] “Post-call webhooks are HTTP notifications that Bland AI automatically sends to your server after a phone call completes.”
   • [claimed-docs] “Give your agent instructions on how it should respond at specific points of the conversation. Choose between prompting or fixed sentences. E…”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   No evidence pack items mention webhooks or an event-subscription mechanism for LiveKit Agents; the docs cover MCP tool integration, telephony, testing, and turn detection, but nothing about webhook-based event notifications.

   ⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round to Bland
   Bland~partialclaimed4/10

   Bland offers LLM-judge Evals to grade call quality and an MCP integration that can 'query analytics' on your account data, which are AI-generated evaluative outputs derived from your call data, but there's no dedicated insights/suggestions dashboard or proactive recommendation feature described. Missing for 10: a native analytics/insights UI, evidence of proactive suggestions surfaced to users, and independent confirmation of these AI-generated insights in practice.

   • [claimed-docs] “Evals let you measure the quality of your calls with LLM judges.”
   • [claimed-docs] “Evals let you measure the quality of your calls with LLM judges. You define eval agents, each of which grades one dimension of a call”
   • [claimed-docs] “Memory lets Bland agents remember people across conversations, including calls, SMS, and other channels, so each interaction feels continuou…”
   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to LiveKit Agents
   Bland~partialclaimed6/10

   Bland supports batch calls, webhooks, pathways, and scheduled/triggered call campaigns that run without manual intervention, which constitute a form of autonomous background automation for voice workflows. However, this is scoped to phone-call automation only, not general-purpose background task/agent scheduling. missing for 10: evidence of a generic scheduler/cron-like trigger system, independent hands-on validation of unattended background runs, and confirmation of failure handling/monitoring for long-running autonomous automations.

   • [claimed-docs] “Batch calls let you initiate high-volume call campaigns by uploading a list of recipients via CSV.”
   • [claimed-docs] “Post-call webhooks are HTTP notifications that Bland AI automatically sends to your server after a phone call completes.”
   • [claimed-docs] “Conversational pathways are our new way of prompting Bland that has led to major breakthroughs in realism. Give your agent instructions on h…”
   • [claimed-docs] “Give your agent instructions on how it should respond at specific points of the conversation. Choose between prompting or fixed sentences. E…”
   • [claimed-docs] “Dispatch AI phone calls to call customers, leads, and to streamline operations.”
   • [claimed-docs] “Connect external APIs and take live actions during phone calls.”

   proof ↗⚑ flag

   LiveKit Agents✓fullprobed7/10

   LiveKit Agents workers run as persistent 'programmatic participants' deployed to LiveKit Cloud or self-hosted infrastructure, processing realtime streams and telephony calls autonomously in the background without human intervention (docs-8, docs-20, docs-30, docs-26, gh-2), and this is verified hands-on as a real pip-installable OSS framework with a working self-hosted backbone (probe-rt-2, probe-rt-3). Missing for 10: explicit scheduling/cron-trigger mechanisms, independent third-party reports of long-running unattended stability (the one community mention is about observability tooling gaps, not autonomy failure).

   • [claimed-docs] “A programmatic participant is any code that joins a LiveKit room as a participant — this includes AI agents, media processors, or custom log…”
   • [claimed-docs] “The Agents framework isn't limited to AI agents. You can use it to deploy any code that needs to process realtime media and data streams as …”
   • [claimed-docs] “Run your agent on LiveKit Cloud's global infrastructure.”
   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”

   proof ↗⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round to Bland
   Bland✓fullclaimed8/10

   Bland's core product is a built-in AI voice agent that users delegate tasks to (placing/answering calls, executing pathways, calling APIs, remembering context) rather than a separate feature bolted on — e.g., 'Dispatch AI phone calls to call customers, leads, and to streamline operations' and pathway/tool/memory docs show rich task delegation to the built-in agent. Missing for 10: independent/hands-on evidence of real-world task delegation outcomes beyond vendor docs.

   • [claimed-docs] “Dispatch AI phone calls to call customers, leads, and to streamline operations.”
   • [claimed-docs] “Conversational pathways are our new way of prompting Bland that has led to major breakthroughs in realism. Give your agent instructions on h…”
   • [claimed-docs] “Connect external APIs and take live actions during phone calls.”
   • [claimed-docs] “Memory lets Bland agents remember people across conversations, including calls, SMS, and other channels, so each interaction feels continuou…”
   • [claimed-docs] “Give your agent instructions on how it should respond at specific points of the conversation. Choose between prompting or fixed sentences. E…”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round to Bland
   Bland✓fullprobed8/10

   Bland offers an official MCP server that lets AI agents operate the entire account (calls, pathways, agents, analytics, evals) via natural language, plus an official CLI, both confirmed live via runtime probes. missing for 10: independent/hands-on third-party review of the MCP/CLI natural-language experience beyond vendor docs and probes.

   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
   • [probe] “official MCP server documented at https://docs.bland.ai/integrations/mcp/overview”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP endpoint https://api.bland.ai/v1/mcp returned HTTP 4…”
   • [claimed-docs] “Manage your entire Bland account from the terminal: make calls, build and test pathways, configure phone numbers, and more.”
   • [probe] “official CLI documented at https://docs.bland.ai/sdks/cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y bland-cli --version` printed `0.6.2` keylessly — the official npm CLI (bland-cli, with dev term…”

   proof ↗⚑ flag

   LiveKit Agents~partialprobed4/10

   LiveKit ships an 'Agent Builder' that lets users prototype/deploy voice agents in-browser without code, and docs explicitly note the framework is 'built for coding agents like Claude Code, Cursor, and Codex' plus agent-readable llms.txt docs — suggesting some AI-native/no-code operability. However there is no evidence of a native natural-language command interface for operating LiveKit itself (e.g., NL-driven CLI, chatops control plane); the MCP support described is for agents calling external tools, not for controlling LiveKit via NL. Missing for 10: a documented NL-driven interface/CLI for configuring or operating the platform itself, and independent confirmation of the Agent Builder's no-code UX actually being NL-based rather than GUI-based.

   • [claimed-docs] “LiveKit is built for coding agents like Claude Code, Cursor, and Codex.”
   • [claimed-docs] “Prototype and deploy voice agents directly in your browser, without writing any code.”
   • [claimed-docs] “LiveKit Agent Builder to prototype and deploy agents directly in your browser without writing code”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.livekit.io/llms.txt # LiveKit docs > LiveKit is an open-source platform for building voice, video,…”
   • [probe] “PROBE docs-md: HTTP 200 at https://docs.livekit.io/agents/.md LiveKit docs › Build Agents › Get Started › Introduction --- # Introduction …”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   Bland—none0/10

   Docs reference an API reference for pathways but no evidence of an interactive playground with runnable examples; a probe for OpenAPI/Swagger specs at standard paths returned 404s, suggesting no such interactive reference exists.

   • [claimed-docs] “See the Pathways API reference to create and manage pathways programmatically.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.bland.ai/openapi.json, https://docs.bland.ai/swagger.json, https://docs.bland.ai/api/op…”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   Evidence shows extensive quickstart guides, markdown-based docs (llms.txt), and a no-code 'Agent Builder' prototyping tool, but nothing describes an interactive API reference (e.g., embedded code sandbox, live runnable examples, or Swagger-like explorer) for exploring the SDK/API itself.

   ⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   Bland—none0/10

   A direct probe found no OpenAPI/Swagger spec at any expected location (all 404s), and no evidence pack item shows a downloadable machine-readable API spec despite extensive API documentation.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.bland.ai/openapi.json, https://docs.bland.ai/swagger.json, https://docs.bland.ai/api/op…”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   The evidence pack documents LiveKit's SDKs, CLI, docs (llms.txt), and framework capabilities but contains no mention of a downloadable OpenAPI/Swagger spec or other machine-readable API description for LiveKit's server or Agents APIs. Since LiveKit exposes a real API surface (server API, cloud API), this axis applies but is unevidenced.

   • [probe] “official CLI documented at https://github.com/livekit/livekit-cli”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.livekit.io/llms.txt # LiveKit docs > LiveKit is an open-source platform for building voice, video,…”
   • [probe] “PROBE docs-md: HTTP 200 at https://docs.livekit.io/agents/.md LiveKit docs › Build Agents › Get Started › Introduction --- # Introduction …”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round drawn
   Bland~partialclaimed6/10

   Bland provides draft/staging pathway editing where live production calls are unaffected, plus a testbed for isolated node testing, canary deployments, and staged version adoption — all functioning as sandbox-like mechanisms distinct from production. However, there's no explicit 'sandbox environment' or dedicated test account/data isolation concept described, and testing still appears to involve real calls/production infrastructure rather than a fully isolated non-production environment. missing for 10: a dedicated sandbox/test-mode account distinct from production billing and phone infrastructure, explicit documentation of synthetic/non-production test data, and independent confirmation that testbed/draft testing never touches real production call data or costs.

   • [claimed-docs] “You can also promote a version to staging to test it before it goes live, or send an individual call against a specific”
   • [claimed-docs] “When you edit a pathway, you are working on a draft. Live calls are not affected. They keep using the published production version.”
   • [claimed-docs] “The testbed lets you take any call ... isolate a specific node interaction, edit the prompt, and run it multiple times to see how the output…”
   • [claimed-docs] “The testbed lets you take any call — whether it's a historical call from call logs or a test chat from the pathway editor — isolate a specif…”
   • [claimed-docs] “Bland supports canary deployments — a way to run a new release on a separate set of containers alongside your production infrastructure and …”
   • [claimed-docs] “This is how you A/B test a new agent release against your live production version, with real calls, before committing to a full rollout.”

   proof ↗⚑ flag

   LiveKit Agents~partialprobed6/10

   LiveKit provides a self-hostable dev server (`livekit-server --dev`) that boots locally with placeholder keys, separate from any production deployment, plus a built-in testing framework with behavioral tests and LLM-driven agent simulations that evaluate agent behavior without needing real production data. However, there is no explicitly branded 'sandbox environment' or staging/production data-isolation feature documented — the sandbox-like capability is inferred from dev-mode self-hosting and test simulations rather than a dedicated sandbox product feature. Missing for 10: an explicit sandbox/staging environment offering with documented separation from production data, and independent confirmation that test simulations never touch production data stores.

   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
   • [claimed-docs] “Agent simulations run end-to-end conversations between your agent and an LLM-driven user, then evaluate the results across the full interact…”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes... Agent simulations run end-to-end conversations between your agent and…”
   • [github] “Builtin test framework: Write tests and use judges to ensure your agent is performing as expected.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round to Bland
   Bland~partialprobed4/10

   Bland documents infrastructure versioning concepts (staged/canary rollouts, choosing when to adopt a new release, draft vs. production pathway versions) but there is no evidence of a documented API versioning scheme (e.g., v1/v2 endpoints) or an explicit deprecation policy for its APIs, and the OpenAPI spec probe returned 404s. missing for 10: documented API version numbering/endpoints, an explicit deprecation/sunset policy for APIs, published OpenAPI spec, and independent confirmation of versioning practices.

   • [claimed-docs] “You choose when to adopt a new release. Your infrastructure stays on the version you've selected until you decide to move forward.”
   • [claimed-docs] “Bland supports canary deployments — a way to run a new release on a separate set of containers alongside your production infrastructure and …”
   • [claimed-docs] “You choose when to adopt a new release... Bland supports canary deployments — a way to run a new release on a separate set of containers alo…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.bland.ai/openapi.json, https://docs.bland.ai/swagger.json, https://docs.bland.ai/api/op…”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   No evidence in the pack mentions API versioning scheme, version compatibility guarantees, or a documented deprecation policy for LiveKit Agents' SDKs or APIs.

   ⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Bland
   Bland✓fullprobed7/10

   Batch calls let users upload a CSV of recipients to initiate high-volume call campaigns, directly enabling bulk operations across many items, and the CLI/MCP server extend programmatic/bulk management of pathways, agents, and calls. Missing for 10: independent/hands-on verification of batch call performance at scale, and documentation of bulk operations beyond calls (e.g., bulk pathway or number management).

   • [claimed-docs] “Batch calls let you initiate high-volume call campaigns by uploading a list of recipients via CSV.”
   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
   • [claimed-docs] “Manage your entire Bland account from the terminal: make calls, build and test pathways, configure phone numbers, and more.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y bland-cli --version` printed `0.6.2` keylessly — the official npm CLI (bland-cli, with dev term…”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to Bland
   Bland✓fullclaimed7/10

   Bland's pathways let users define conditional logic that executes webhooks/API calls at specific conversation nodes, and post-call webhooks automatically fire HTTP notifications when a call-completion event occurs — this is a documented rules-trigger-action-on-event mechanism. missing for 10: independent/hands-on verification of the webhook triggering in production, and evidence of event types beyond call-based ones (e.g., generic account-level event automation).

   • [claimed-docs] “Post-call webhooks are HTTP notifications that Bland AI automatically sends to your server after a phone call completes.”
   • [claimed-docs] “Give your agent instructions on how it should respond at specific points of the conversation. Choose between prompting or fixed sentences. E…”
   • [claimed-docs] “Connect external APIs and take live actions during phone calls.”
   • [claimed-docs] “Conversational pathways are our new way of prompting Bland that has led to major breakthroughs in realism. Give your agent instructions on h…”

   proof ↗⚑ flag

   LiveKit Agents~partialclaimed5/10

   LiveKit Agents supports event-driven automatic behaviors — tool-calling on LLM decisions, turn-detector triggering response timing, and adaptive interruption handling triggering barge-in logic — all of which are 'events auto-trigger actions' patterns built into the agent runtime. However, the evidence shows these as built-in framework behaviors and developer-coded event handlers rather than a user-facing declarative rule engine (e.g., no evidence of a 'when X happens do Y' config surface, webhook/rule subscription API, or no-code rule builder). Missing for 10: explicit rule-definition/webhook-trigger API exposed to non-developer users, documentation of a generalized event-subscription system beyond tool calls and turn logic, and independent confirmation of custom rule automation in production use.

   • [claimed-docs] “Call external APIs or lookup data for RAG.”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use. This feature allows you to create a custom library of tools to extend your agent's context”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use.”
   • [claimed-docs] “LiveKit's `TurnDetector` is an audio model that encodes user audio directly, capturing both _what_ is said and _how_ it's said.”
   • [claimed-docs] “A turn detector model can predict that they have more to say and wait for them to finish before responding.”
   • [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response... to identify intentional interruptions (…”
   • [claimed-docs] “the model analyzes the acoustic signals to identify intentional interruptions (barge-ins) from conversational backchanneling”
   • [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response.”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round drawn
   Bland—none0/10

   Bland's docs cover batch calls, pathways, webhooks, and API integrations, but nothing describes scheduling recurring jobs or workflows (e.g., cron-like triggers for calls or campaigns) — batch calls are one-off CSV uploads, not recurring schedules.

   • [claimed-docs] “Batch calls let you initiate high-volume call campaigns by uploading a list of recipients via CSV.”
   • [claimed-docs] “Dispatch AI phone calls to call customers, leads, and to streamline operations.”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   The evidence pack covers LiveKit Agents' realtime voice/video agent capabilities, tool use, MCP integration, telephony, and testing, but contains no mention of any scheduling, cron-like recurring job, or workflow-automation trigger mechanism. This is a fair capability to ask about for an automation-focused agent framework, but no evidence supports it.

   ⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round to Bland
   Bland~partialclaimed6/10

   Bland pathways support draft/staging/production versioning, A/B testing new releases against live production, and canary/staged rollout with adoption control (bland-docs-2, bland-docs-14, bland-docs-19, bland-docs-21, bland-docs-29, bland-docs-31), plus a testbed and evals for reviewing behavior before shipping (bland-docs-7, bland-docs-8, bland-docs-22, bland-docs-28). However, there is no explicit documentation of an automated 'rollback' mechanism to revert a live pathway/release to a prior version — only forward-adoption and canary controls are described. Missing for 10: explicit rollback/revert-to-previous-version capability, version history/diff view, and independent confirmation of rollback in practice.

   • [claimed-docs] “You can also promote a version to staging to test it before it goes live, or send an individual call against a specific”
   • [claimed-docs] “You choose when to adopt a new release. Your infrastructure stays on the version you've selected until you decide to move forward.”
   • [claimed-docs] “When you edit a pathway, you are working on a draft. Live calls are not affected. They keep using the published production version.”
   • [claimed-docs] “This is how you A/B test a new agent release against your live production version, with real calls, before committing to a full rollout.”
   • [claimed-docs] “Bland supports canary deployments — a way to run a new release on a separate set of containers alongside your production infrastructure and …”
   • [claimed-docs] “You choose when to adopt a new release... Bland supports canary deployments — a way to run a new release on a separate set of containers alo…”
   • [claimed-docs] “Evals let you measure the quality of your calls with LLM judges.”
   • [claimed-docs] “The testbed lets you take any call ... isolate a specific node interaction, edit the prompt, and run it multiple times to see how the output…”
   • [claimed-docs] “Evals let you measure the quality of your calls with LLM judges. You define eval agents, each of which grades one dimension of a call”
   • [claimed-docs] “The testbed lets you take any call — whether it's a historical call from call logs or a test chat from the pathway editor — isolate a specif…”

   proof ↗⚑ flag

   LiveKit Agents~partialclaimed4/10

   The pricing page explicitly claims 'Instant rollback to a previous agent deployment,' and the testing/simulation framework (behavioral tests, LLM-judge simulations) supports a review step before deployment, but there's no documented versioning system, changelog, or diff/history UI for automations. Missing for 10: explicit version history/diffing of agent configs, a documented review/approval workflow beyond test suites, and independent confirmation that rollback works in practice.

   • [claimed-docs] “Instant rollback to a previous agent deployment”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
   • [claimed-docs] “Agent simulations run end-to-end conversations between your agent and an LLM-driven user, then evaluate the results across the full interact…”
   • [github] “Builtin test framework: Write tests and use judges to ensure your agent is performing as expected.”

   proof ↗⚑ flag

Compliance

1. founderMeet call-recording consent and disclosure obligations with per-call recording controls and configurable data retention

   weight 2 · round drawn
   Bland—none0/10

   The evidence pack contains no mention of call recording consent/disclosure features, per-call recording toggles, or configurable data retention policies anywhere in Bland's docs; while this is a fair compliance axis for a voice AI platform, nothing in the pack substantiates it.

   ⚑ flag

   LiveKit Agents—none0/10

   No evidence in the pack mentions call recording, consent disclosure, or data retention controls anywhere in LiveKit Agents docs, GitHub, or community sources; the pack covers telephony, tool use, MCP, testing, and turn detection but nothing about recording/retention compliance features. missing for 10: recording consent/disclosure mechanisms, per-call recording toggles, configurable data retention policies, and any documentation or hands-on evidence of these compliance controls.

   ⚑ flag

2. platform-engineerRun regulated workloads with HIPAA/BAA support, SOC 2, and data-residency options

   weight 2 · round drawn
   Bland—none0/10

   The evidence pack contains no mention of HIPAA, BAA, SOC 2 certification, or data-residency options anywhere in the docs or probes; only a generic tagline calling Bland an 'enterprise' platform for 'regulated' workflows without specifics. Missing for 10: HIPAA/BAA documentation, SOC 2 report or certification evidence, data-residency/region controls, and any compliance attestations.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.bland.ai/llms.txt # Bland Documentation Bland is an enterprise voice AI platform for high-volume, …”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   The evidence pack contains no mention of HIPAA/BAA agreements, SOC 2 certification, or data-residency controls anywhere in LiveKit's docs, GitHub repo, or probes; self-hosting and TURN/security items only cover TLS/SSL and self-hosted deployment, not compliance attestations. As a platform serving enterprise/regulated workloads, this axis clearly applies, so lack of evidence yields 'none'.

   • [claimed-docs] “The good news is LiveKit includes an embedded TURN server. It's a secure TURN implementation that has integrated authentication with the res…”
   • [claimed-docs] “In order to have a secure LiveKit deployment, you will need a domain as well as a SSL certificate for that domain.”

   proof ↗⚑ flag

Scale

1. platform-engineerSee documented concurrency limits and scale to many simultaneous calls without manual capacity begging

   weight 2 · round drawn
   Bland~partialclaimed3/10

   Docs mention batch calling for high-volume campaigns and enterprise infrastructure controls (canary releases, staged rollout), implying some capacity for scale, but there is no documented concurrency limit, rate ceiling, or auto-scaling guarantee, and no evidence that capacity increases don't require contacting sales/support. Missing for 10: explicit concurrent-call limits, auto-scaling documentation, and evidence that scaling doesn't require manual requests to Bland.

   • [claimed-docs] “Batch calls let you initiate high-volume call campaigns by uploading a list of recipients via CSV.”
   • [claimed-docs] “Bland supports canary deployments — a way to run a new release on a separate set of containers alongside your production infrastructure and …”
   • [claimed-docs] “You choose when to adopt a new release... Bland supports canary deployments — a way to run a new release on a separate set of containers alo…”

   proof ↗⚑ flag

   LiveKit Agents~partialcommunity3/10

   Docs mention running agents on LiveKit Cloud's global infrastructure and self-hosting with an embedded TURN server, implying scale-out capability, but no evidence pack item cites concrete documented concurrency limits, per-instance call caps, or autoscaling guarantees; a community report even flags lack of visibility into per-call cost/latency at scale. missing for 10: published concurrency/capacity numbers, autoscaling documentation, load-testing benchmarks, and confirmation that scaling requires no manual quota requests.

   • [claimed-docs] “Run your agent on LiveKit Cloud's global infrastructure.”
   • [claimed-docs] “The good news is LiveKit includes an embedded TURN server. It's a secure TURN implementation that has integrated authentication with the res…”
   • [community] “We've been running multiple voice AI agents on LiveKit and kept running into visibility issues — no way to measure TTFT, latency across STT …”

   proof ↗⚑ flag

Self host

1. platform-engineerSelf-host the voice agent runtime from open-source code on my own infrastructure

   weight 3 · round to LiveKit Agents
   Bland—none0/10

   Bland is presented throughout as a hosted enterprise SaaS platform (managed infrastructure, release adoption controls, canary deployments on Bland's own containers) with no mention of open-source code or a self-hostable runtime; a community post even shows a user asking for an open-source alternative because Bland itself isn't one.

   • [claimed-docs] “You choose when to adopt a new release. Your infrastructure stays on the version you've selected until you decide to move forward.”
   • [claimed-docs] “Bland supports canary deployments — a way to run a new release on a separate set of containers alongside your production infrastructure and …”
   • [community] “I want to experiment with building my own phone agent. Currently experimented with bland.ai but it gets expensive. Any open source alternati…”

   proof ↗⚑ flag

   LiveKit Agents✓fullprobed9/10

   LiveKit Agents is open source, pip/npm installable and runs the agent runtime independent of any vendor account, and the underlying WebRTC/SIP server (livekit-server) is Apache-2.0 and self-hostable, confirmed via hands-on probes (booting `livekit-server --dev` locally with no keys, and installing livekit-agents/CLI keylessly) plus official self-hosting deployment docs covering TURN, TLS/domain setup. Missing for 10: no first-party production-scale self-hosting case study or independent report of large-scale self-hosted deployment beyond dev-mode probe.

   • [probe] “PROBE runtime (recorded 2026-09-05): `lk --version` printed `lk version 2.18.6` after a plain `brew install livekit-cli` — the official CLI …”
   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”
   • [claimed-docs] “The good news is LiveKit includes an embedded TURN server. It's a secure TURN implementation that has integrated authentication with the res…”
   • [claimed-docs] “In order to have a secure LiveKit deployment, you will need a domain as well as a SSL certificate for that domain.”
   • [claimed-docs] “A programmatic participant is any code that joins a LiveKit room as a participant — this includes AI agents, media processors, or custom log…”

   proof ↗⚑ flag

Latency

1. platform-engineerSee documented end-to-end voice latency numbers or tuning guidance backing the platform's speed claims

   weight 3 · round drawn
   Bland—none0/10

   No evidence pack items mention latency numbers, benchmarks, or tuning guidance for end-to-end voice response time; documentation covers pathways, SIP, MCP, CLI, and enterprise features but nothing about speed/latency metrics or optimization guidance.

   ⚑ flag

   LiveKit Agents—none0/10

   The evidence pack contains no documented end-to-end latency benchmarks or explicit tuning guidance for reducing turn-taking latency — only feature descriptions (turn detector, adaptive interruption handling) without numbers or configuration guidance. A community report explicitly states users could not measure TTFT or STT→LLM→TTS latency, reinforcing that this visibility/documentation is absent.

   • [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response... to identify intentional interruptions (…”
   • [claimed-docs] “LiveKit's `TurnDetector` is an audio model that encodes user audio directly, capturing both _what_ is said and _how_ it's said.”
   • [claimed-docs] “A turn detector model can predict that they have more to say and wait for them to finish before responding.”
   • [claimed-docs] “the model analyzes the acoustic signals to identify intentional interruptions (barge-ins) from conversational backchanneling”
   • [community] “We've been running multiple voice AI agents on LiveKit and kept running into visibility issues — no way to measure TTFT, latency across STT …”

   proof ↗⚑ flag

Turn taking

1. developerRely on the agent to handle interruptions (barge-in) gracefully — stopping speech, updating context, and recovering the turn

   weight 3 · round to LiveKit Agents
   Bland—none0/10

   The evidence pack covers pathways, webhooks, batch calls, evals, MCP/CLI integrations, and infrastructure features, but contains no mention of barge-in, interruption handling, stopping TTS mid-utterance, or turn recovery logic — a core voice-agent capability that would be a fair and expected axis for this product type. missing for 10: any documentation or claim about detecting user interruptions, halting agent speech, updating context after a barge-in, and resuming/recovering the conversational turn.

   ⚑ flag

   LiveKit Agents✓fullprobed8/10

   LiveKit Agents has well-documented, purpose-built features for barge-in: adaptive interruption handling distinguishes intentional interruptions from backchanneling, and the TurnDetector model analyzes acoustic and semantic signals to manage turn-taking recovery. These are first-party docs with specific technical detail on the mechanism (stopping speech, updating context, recovering turn), and the runtime probes confirm the framework is real and installable. Missing for 10: independent/hands-on evidence specifically testing barge-in behavior in production (only community evidence is about latency observability, not interruption handling itself) and no third-party benchmark of interruption accuracy.

   • [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response... to identify intentional interruptions (…”
   • [claimed-docs] “the model analyzes the acoustic signals to identify intentional interruptions (barge-ins) from conversational backchanneling”
   • [claimed-docs] “LiveKit's `TurnDetector` is an audio model that encodes user audio directly, capturing both _what_ is said and _how_ it's said.”
   • [claimed-docs] “A turn detector model can predict that they have more to say and wait for them to finish before responding.”
   • [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response.”
   • [claimed-docs] “LiveKit's TurnDetector is an audio model that encodes user audio directly, capturing both what is said and how it's said.”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”

   proof ↗⚑ flag

2. developerEnable noise suppression or audio filtering so the agent stays coherent on noisy real-world calls

   weight 1 · round drawn
   Bland—none0/10

   No evidence pack item mentions noise suppression, audio filtering, or handling of noisy real-world call environments; documentation covers pathways, integrations, MCP, CLI, SIP, and enterprise release features but nothing about audio quality/noise handling.

   ⚑ flag

   LiveKit Agents—none0/10

   The evidence pack covers turn detection, interruption handling, telephony, MCP tools, and testing, but contains no mention of noise suppression or audio filtering plugins/features anywhere. Missing for 10: any docs or plugin reference to noise cancellation (e.g. Krisp/BVC), background noise filtering, or audio pre-processing configuration.

   ⚑ flag

3. developerUse model-based end-of-turn detection beyond simple VAD silence timeouts so the agent doesn't talk over slow speakers

   weight 2 · round to LiveKit Agents
   Bland—none0/10

   The evidence pack contains no mention of turn-taking, end-of-turn detection, VAD, or interruption handling mechanisms of any kind — nothing addresses how Bland decides when a speaker has finished talking. This is a fair and applicable axis for a voice AI platform, but no capability is documented; missing for 10: any mention of end-of-turn/turn-taking model, VAD configuration, or handling of slow speakers/pauses.

   ⚑ flag

   LiveKit Agents✓fullclaimed9/10

   LiveKit's TurnDetector is a dedicated audio+semantic model (not just VAD silence) that predicts whether a user has finished speaking or has more to say, waiting accordingly, and adaptive interruption handling further distinguishes real barge-ins from backchanneling — directly addressing not talking over slow speakers. This is documented in detail across multiple first-party doc pages plus GitHub feature lists. missing for 10: independent/hands-on benchmark or community validation of the turn-detector's real-world latency/accuracy impact.

   • [claimed-docs] “LiveKit's `TurnDetector` is an audio model that encodes user audio directly, capturing both _what_ is said and _how_ it's said.”
   • [claimed-docs] “A turn detector model can predict that they have more to say and wait for them to finish before responding.”
   • [claimed-docs] “LiveKit's TurnDetector is an audio model that encodes user audio directly, capturing both what is said and how it's said.”
   • [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response... to identify intentional interruptions (…”
   • [claimed-docs] “the model analyzes the acoustic signals to identify intentional interruptions (barge-ins) from conversational backchanneling”
   • [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response.”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Bland
   Bland✓fullprobed7/10

   Bland's docs show extensive programmatic control mirroring UI features: pathway creation/versioning, testbed, evals, batch calls, webhooks, SIP/phone number management, and a CLI/MCP server that explicitly lets users 'manage your entire Bland account from the terminal' and perform the same actions (calls, pathways, agents, analytics) as the UI. This breadth strongly supports API parity, though there's no explicit first-party statement guaranteeing 100% feature parity and no discoverable OpenAPI spec (all candidate URLs 404), so full parity isn't independently confirmed. Missing for 10: an explicit parity guarantee/documentation and a machine-readable OpenAPI spec, plus independent hands-on confirmation that every UI action has an API equivalent.

   • [claimed-docs] “See the Pathways API reference to create and manage pathways programmatically.”
   • [claimed-docs] “Manage your entire Bland account from the terminal: make calls, build and test pathways, configure phone numbers, and more.”
   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
   • [claimed-docs] “The testbed lets you take any call ... isolate a specific node interaction, edit the prompt, and run it multiple times to see how the output…”
   • [claimed-docs] “Batch calls let you initiate high-volume call campaigns by uploading a list of recipients via CSV.”
   • [claimed-docs] “Evals let you measure the quality of your calls with LLM judges.”
   • [probe] “official CLI documented at https://docs.bland.ai/sdks/cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): `npx -y bland-cli --version` printed `0.6.2` keylessly — the official npm CLI (bland-cli, with dev term…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.bland.ai/openapi.json, https://docs.bland.ai/swagger.json, https://docs.bland.ai/api/op…”

   proof ↗⚑ flag

   LiveKit Agents~partialprobed6/10

   LiveKit Agents is API/SDK-first (Python/Node.js) with a full programmatic surface for building agents, tools, telephony, and testing, and there is also a no-code 'Agent Builder' UI for prototyping. However, the evidence doesn't confirm that everything achievable in that browser-based Agent Builder UI (or LiveKit Cloud dashboard features like deployment management, rollback) is equally exposed via API/CLI, so parity between UI and API is not fully demonstrated. missing for 10: explicit evidence that Agent Builder's no-code UI actions (and Cloud dashboard deployment/rollback controls) are all reachable via API/CLI, and confirmation of full CLI/API parity with dashboard features.

   • [claimed-docs] “Prototype and deploy voice agents directly in your browser, without writing any code.”
   • [claimed-docs] “LiveKit Agent Builder to prototype and deploy agents directly in your browser without writing code”
   • [claimed-docs] “LiveKit Agent Builder: Prototype and deploy voice agents directly in your browser, without writing any code.”
   • [claimed-docs] “Instant rollback to a previous agent deployment”
   • [probe] “official CLI documented at https://github.com/livekit/livekit-cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): `lk --version` printed `lk version 2.18.6` after a plain `brew install livekit-cli` — the official CLI …”
   • [claimed-docs] “A programmatic participant is any code that joins a LiveKit room as a participant — this includes AI agents, media processors, or custom log…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round drawn
   Bland—none0/10

   No evidence of a data export feature or open-format export of account data (pathways, call logs, memories, etc.); the docs cover CLI, MCP, and SDK integrations but nothing about exporting user data for portability/exit.

   ⚑ flag

   LiveKit Agents—none0/10

   The evidence pack documents LiveKit Agents' open-source, self-hostable nature (Apache-2.0 server, pip/npm installable framework) but contains no explicit mention of data export tooling, open data formats for conversation/session logs, or a documented exit/migration path for user data. Self-hosting mitigates lock-in in principle, but that is not the same as a documented 'export all data' capability.

   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”
   • [claimed-docs] “The good news is LiveKit includes an embedded TURN server. It's a secure TURN implementation that has integrated authentication with the res…”

   proof ↗⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round to LiveKit Agents
   Bland—none0/10

   Bland is a closed, commercial SaaS voice AI platform; there is no evidence of any open-source license or public source code repository. A community comment explicitly asks for an open-source alternative, implying Bland itself is not open source. This is an applicable axis (a product could publish open-source components) but no evidence supports it.

   • [community] “I want to experiment with building my own phone agent. Currently experimented with bland.ai but it gets expensive. Any open source alternati…”

   proof ↗⚑ flag

   LiveKit Agents✓fullprobed8/10

   LiveKit Agents is hosted at github.com/livekit/agents and documented as an open-source framework, with hands-on verification that the packages install and run without any account or key (pip/npm) and that the underlying server is Apache-2.0 OSS. The llms.txt docs also explicitly describe LiveKit as 'an open-source platform.' Missing for 10: an explicit license file/badge citation specifically for the livekit/agents repo (only inferred via probe evidence and sibling repo license).

   • [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.livekit.io/llms.txt # LiveKit docs > LiveKit is an open-source platform for building voice, video,…”
   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”
   • [probe] “official CLI documented at https://github.com/livekit/livekit-cli”

   proof ↗⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round to LiveKit Agents
   Bland—none0/10

   Bland is documented as a hosted enterprise voice AI platform (call dispatch, pathways, SIP, MCP, CLI) with no mention of an on-prem/self-hosted deployment option; 'enterprise' release controls (docs-14, docs-29, docs-31) only govern version adoption timing on Bland's own infrastructure, not customer self-hosting. A community post explicitly looks for an open-source self-hostable alternative because Bland itself doesn't offer this.

   • [claimed-docs] “You choose when to adopt a new release. Your infrastructure stays on the version you've selected until you decide to move forward.”
   • [claimed-docs] “Bland supports canary deployments — a way to run a new release on a separate set of containers alongside your production infrastructure and …”
   • [claimed-docs] “You choose when to adopt a new release... Bland supports canary deployments — a way to run a new release on a separate set of containers alo…”
   • [community] “I want to experiment with building my own phone agent. Currently experimented with bland.ai but it gets expensive. Any open source alternati…”

   proof ↗⚑ flag

   LiveKit Agents✓fullprobed9/10

   LiveKit provides a documented, Apache-2.0 open-source self-hosting deployment guide including embedded TURN server and SSL setup, and this was hands-on verified: `livekit-server --dev` boots with no account or key, and the open-source livekit-agents framework installs and imports keylessly via pip/npm. This directly demonstrates the core product (server + agents framework) can be self-hosted. Missing for 10: independent third-party production self-hosting case study beyond the probe verification.

   • [claimed-docs] “The good news is LiveKit includes an embedded TURN server. It's a secure TURN implementation that has integrated authentication with the res…”
   • [claimed-docs] “In order to have a secure LiveKit deployment, you will need a domain as well as a SSL certificate for that domain.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”
   • [probe] “PROBE runtime (recorded 2026-09-05): `lk --version` printed `lk version 2.18.6` after a plain `brew install livekit-cli` — the official CLI …”

   proof ↗⚑ flag

Pricing

1. founderSee published per-minute or usage pricing and estimate cost per call before committing

   weight 2 · round drawn
   Bland—none0/10

   No evidence pack item shows published per-minute or usage pricing, a pricing page, or any cost calculator; the only pricing-adjacent mention is a community complaint that Bland 'gets expensive' with no figures. Missing for 10: published price list, per-minute rate documentation, cost calculator or estimator tool.

   • [community] “I want to experiment with building my own phone agent. Currently experimented with bland.ai but it gets expensive. Any open source alternati…”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   The pack references a pricing URL (livekit.io/pricing) only for an unrelated 'rollback' feature and never quotes actual per-minute or usage rates; a community comment even highlights that operators struggled with 'cost per call' visibility, but no evidence shows published pricing figures a founder could use to estimate costs before committing.

   • [claimed-docs] “Instant rollback to a previous agent deployment”
   • [community] “We've been running multiple voice AI agents on LiveKit and kept running into visibility issues — no way to measure TTFT, latency across STT …”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round to LiveKit Agents
   Bland—none0/10

   No evidence in the pack mentions data residency, regional hosting options, or geographic storage controls for Bland; enterprise/infra docs discuss release versioning and SIP/canary deployments but not region selection. Missing for 10: any mention of data residency options, region-specific hosting, or compliance-driven storage location controls.

   ⚑ flag

   LiveKit Agents~partialprobed5/10

   LiveKit Agents/server is open-source and self-hostable (embedded TURN server, deployment docs, hands-on verified self-host boot), which lets an operator choose their own hosting region/data location, but there is no explicit documented region-selection or data-residency feature for LiveKit Cloud itself. missing for 10: explicit LiveKit Cloud region-selection/data-residency controls, compliance certifications (e.g. GDPR/SOC2 region guarantees), and any customer-facing UI for choosing storage region.

   • [claimed-docs] “The good news is LiveKit includes an embedded TURN server. It's a secure TURN implementation that has integrated authentication with the res…”
   • [claimed-docs] “In order to have a secure LiveKit deployment, you will need a domain as well as a SSL certificate for that domain.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `livekit-server --dev` (brew-installed, Apache-2.0 OSS) booted on this machine with NO account or key —…”

   proof ↗⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   Bland—none0/10

   No evidence in the pack addresses data-training opt-out or AI-training privacy policies for Bland; nothing about training-data usage or opt-out controls is mentioned across any of the docs or probes.

   ⚑ flag

   LiveKit Agents—none0/10

   No evidence in the pack addresses data usage for AI model training, opt-out policies, or any privacy commitments regarding training data; the evidence pack is entirely about agent framework features (tools, telephony, testing, MCP support). This is a plausible axis for a platform handling user voice/media data, but absent any documentation on training-data usage or opt-out, it cannot be credited as delivered.

   ⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round drawn
   Bland—none0/10

   No evidence pack items address data retention policies, data deletion controls, or privacy/compliance settings for call recordings, transcripts, or memory data. This axis clearly applies to an enterprise voice AI platform handling call data, but nothing in the evidence documents retention periods, deletion APIs, or GDPR/CCPA-style data controls. Missing for 10: retention policy docs, data deletion API/endpoint, compliance certifications, memory/data purge mechanism.

   ⚑ flag

   LiveKit Agents—none0/10

   The evidence pack contains no mention of data retention policies, recording/session deletion controls, or configurable retention windows for LiveKit Agents. While self-hosting is documented (implying infrastructure control), no explicit retention/deletion feature or API is evidenced.

   ⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Bland—none0/10

   No evidence pack items mention telemetry, analytics opt-out, or usage-tracking controls for Bland's own platform; this is a fair question for an enterprise SaaS product but is simply unaddressed in the evidence.

   ⚑ flag

   LiveKit Agents—none0/10

   No evidence pack item discusses telemetry, usage tracking, analytics collection, or an opt-out mechanism for LiveKit Agents; the framework being open-source and self-hostable is not sufficient evidence of an explicit telemetry opt-out control.

   ⚑ flag

Call control

1. developerEscalate a live call to a human with warm or blind transfer, passing context along

   weight 2 · round drawn
   Bland—none0/10

   The evidence pack covers pathways, webhooks, SIP, MCP, CLI, and other Bland features, but contains no mention of call transfer (warm or blind) or handing off a live call to a human agent with context. This is a standard telephony capability that could plausibly be documented, so absence of evidence yields 'none' rather than 'na'.

   ⚑ flag

   LiveKit Agents—none0/10

   LiveKit Agents documents inbound/outbound telephony (SIP trunks, phone calls) but the evidence pack contains no mention of call transfer (warm or blind) or passing conversational context to a human agent during a live call — this specific escalation capability is never described.

   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [claimed-docs] “Enable your voice agent to make or take phone calls.”
   • [claimed-docs] “Outbound trunks are used to place outgoing calls.”
   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”

   proof ↗⚑ flag

2. developerMy agent can send DTMF keypresses, navigate IVR menus, and detect or leave voicemail

   weight 1 · round drawn
   Bland—none0/10

   The evidence pack covers pathways, SIP, webhooks, evals, and MCP/CLI tooling, but nowhere mentions DTMF keypress sending, IVR menu navigation, or voicemail detection/leaving — capabilities that are plausible for a voice telephony platform but are simply undocumented here.

   ⚑ flag

   LiveKit Agents—none0/10

   Evidence confirms LiveKit's telephony/SIP stack supports inbound/outbound calls (docs-26, docs-31, docs-32, gh-2), but nothing in the pack mentions DTMF keypress sending, IVR menu navigation, or voicemail detection/leaving capabilities specifically. Missing for 10: any documentation or example of DTMF tone generation, automated IVR menu traversal, or voicemail detection/answering-machine-detection logic.

   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [claimed-docs] “Enable your voice agent to make or take phone calls.”
   • [claimed-docs] “Outbound trunks are used to place outgoing calls.”
   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”

   proof ↗⚑ flag

Campaigns

1. founderRun batch outbound call campaigns with scheduling and throughput controls

   weight 2 · round to Bland
   Bland✓fullclaimed8/10

   Bland's docs explicitly describe batch calls for uploading CSV recipient lists to run high-volume outbound campaigns, backed by scheduling/throughput-related infrastructure like SIP integration, phone number management, and analytics/evals to monitor campaign performance. Missing for 10: explicit documentation of scheduling controls (e.g., call windows/timing) and rate-limiting/throughput knobs specifically, plus independent hands-on verification of batch campaign behavior at scale.

   • [claimed-docs] “Batch calls let you initiate high-volume call campaigns by uploading a list of recipients via CSV.”
   • [claimed-docs] “Dispatch AI phone calls to call customers, leads, and to streamline operations.”
   • [claimed-docs] “Bland supports inbound and outbound SIP. Point calls from your carrier or PBX at Bland to be answered by an agent, or have Bland place calls…”
   • [claimed-docs] “Number porting to bring existing numbers to Bland”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   Evidence shows LiveKit Agents can place individual outbound calls via SIP outbound trunks (docs-26, docs-32, docs-31), but there is no evidence of any batch campaign management, call scheduling, or throughput/rate-limiting controls for running outbound calling campaigns at scale — these features are simply not documented anywhere in the pack.

   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [claimed-docs] “Outbound trunks are used to place outgoing calls.”
   • [claimed-docs] “Enable your voice agent to make or take phone calls.”
   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”

   proof ↗⚑ flag

Numbers

1. developerProvision phone numbers and run both inbound and outbound calls through the platform's API

   weight 3 · round to Bland
   Bland✓fullprobed8/10

   Bland's docs explicitly document creating/managing inbound phone numbers, outbound call dispatch via API, bringing your own Twilio numbers, and SIP for both inbound/outbound, all programmatically accessible, plus a CLI/MCP that manage phone numbers and calls end-to-end. Missing for 10: no explicit REST API reference/OpenAPI spec confirmed (probe shows openapi.json 404s) and no independent hands-on developer report of a full provision+call round trip.

   • [claimed-docs] “Dispatch AI phone calls to call customers, leads, and to streamline operations.”
   • [claimed-docs] “Create inbound phone numbers for customer support, etc.”
   • [claimed-docs] “Users can connect their own Twilio account to Bland, easily bring over your existing phone numbers to use within the platform.”
   • [claimed-docs] “Number porting to bring existing numbers to Bland”
   • [claimed-docs] “Bland supports inbound and outbound SIP. Point calls from your carrier or PBX at Bland to be answered by an agent, or have Bland place calls…”
   • [claimed-docs] “Manage your entire Bland account from the terminal: make calls, build and test pathways, configure phone numbers, and more.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.bland.ai/openapi.json, https://docs.bland.ai/swagger.json, https://docs.bland.ai/api/op…”

   proof ↗⚑ flag

   LiveKit Agents~partialclaimed6/10

   LiveKit's SIP/telephony stack clearly supports inbound and outbound calling ('LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls', 'Outbound trunks are used to place outgoing calls', 'Enable your voice agent to make or take phone calls') and integrates with the agents runtime via SIP trunks. However, the evidence never shows LiveKit itself provisioning phone numbers through its own API — telephony typically requires configuring an external SIP trunk provider, and no docs here describe a native number-provisioning endpoint. Missing for 10: explicit documentation/evidence of a LiveKit API call that provisions/purchases phone numbers directly (rather than just configuring trunks against externally-acquired numbers), and independent/hands-on confirmation of the inbound+outbound call flow.

   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [claimed-docs] “Enable your voice agent to make or take phone calls.”
   • [claimed-docs] “Outbound trunks are used to place outgoing calls.”
   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”

   proof ↗⚑ flag

Sip

1. platform-engineerConnect my own carrier or PBX via SIP trunking (or import Twilio/Telnyx numbers) instead of being locked to bundled telephony

   weight 2 · round to Bland
   Bland✓fullclaimed8/10

   Bland's docs explicitly document inbound/outbound SIP trunking to connect a customer's own carrier or PBX, plus number porting, and separately support connecting an existing Twilio account/numbers. Missing for 10: no mention of Telnyx import specifically and no independent/hands-on validation of SIP setup success.

   • [claimed-docs] “Bland supports inbound and outbound SIP. Point calls from your carrier or PBX at Bland to be answered by an agent, or have Bland place calls…”
   • [claimed-docs] “Number porting to bring existing numbers to Bland”
   • [claimed-docs] “Users can connect their own Twilio account to Bland, easily bring over your existing phone numbers to use within the platform.”

   proof ↗⚑ flag

   LiveKit Agents✓fullclaimed7/10

   LiveKit's SIP-based telephony stack lets agents make/receive calls via inbound and outbound SIP trunks, which is the standard mechanism for bringing your own carrier or PBX (docs-26, docs-32, gh-2, docs-31). This is architecture-agnostic SIP, not a bundled/proprietary telephony lock-in, satisfying the platform-engineer's need to connect external trunks. Missing for 10: explicit documented walkthroughs or examples of importing Twilio/Telnyx numbers specifically, and no independent/hands-on confirmation of a real carrier trunk connection succeeding.

   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [claimed-docs] “Outbound trunks are used to place outgoing calls.”
   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”
   • [claimed-docs] “Enable your voice agent to make or take phone calls.”

   proof ↗⚑ flag

Analytics

1. ai-native userThe platform's AI reviews my calls for me — scoring quality, flagging failures, and analyzing resolution automatically

   weight 2 · round to Bland
   Bland~partialclaimed7/10

   Bland's Evals feature explicitly lets users define LLM-judge agents that grade calls on custom dimensions (quality, resolution, etc.), and the Testbed lets you replay and analyze specific call nodes to spot failures — directly matching automated call review/scoring. However, this requires the user to configure eval criteria rather than being a fully out-of-the-box automatic analysis, and there's no evidence of a pre-built default 'failure flagging' report. Missing for 10: evidence of fully automatic, no-setup call scoring/dashboard, and independent/hands-on validation of eval accuracy.

   • [claimed-docs] “Evals let you measure the quality of your calls with LLM judges.”
   • [claimed-docs] “Evals let you measure the quality of your calls with LLM judges. You define eval agents, each of which grades one dimension of a call”
   • [claimed-docs] “The testbed lets you take any call ... isolate a specific node interaction, edit the prompt, and run it multiple times to see how the output…”
   • [claimed-docs] “The testbed lets you take any call — whether it's a historical call from call logs or a test chat from the pathway editor — isolate a specif…”

   proof ↗⚑ flag

   LiveKit Agents!disputedcontradicted3/10

   LiveKit ships a dev-time testing framework with 'judges' and agent simulations to evaluate scripted interactions (docs-9/10/25, gh-4), but this is pre-deployment test tooling, not automatic scoring/flagging/resolution-analysis of live production calls. A hands-on community report explicitly says operators running real agents on LiveKit have no built-in way to measure latency or cost per call, let alone automated quality/resolution review, contradicting the idea that the platform reviews calls for you. Missing for 10: production call-level QA scoring, automatic failure flagging on real traffic, resolution/outcome analysis dashboards.

   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
   • [claimed-docs] “Agent simulations run end-to-end conversations between your agent and an LLM-driven user, then evaluate the results across the full interact…”
   • [github] “Builtin test framework: Write tests and use judges to ensure your agent is performing as expected.”
   • [community] “We've been running multiple voice AI agents on LiveKit and kept running into visibility issues — no way to measure TTFT, latency across STT …”

   proof ↗⚑ flag

2. founderSee call analytics — success rates, durations, outcomes, sentiment — in dashboards without building my own

   weight 2 · round to Bland
   Bland~partialprobed4/10

   Evidence confirms Bland has an analytics layer (MCP server can 'query analytics') and quality-grading tools like Evals (LLM judges scoring call dimensions) and a testbed for reviewing call interactions, implying some built-in metrics exist. However, there is no direct evidence of an actual dashboard UI showing success rates, call durations, outcomes, or sentiment trends over time — analytics access shown is via MCP/API query rather than a visual dashboard. Missing for 10: screenshots or docs of a native analytics dashboard, explicit mention of success-rate/duration/sentiment metrics, and independent confirmation the dashboard requires no custom building.

   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
   • [claimed-docs] “Evals let you measure the quality of your calls with LLM judges. You define eval agents, each of which grades one dimension of a call”
   • [claimed-docs] “The testbed lets you take any call — whether it's a historical call from call logs or a test chat from the pathway editor — isolate a specif…”
   • [probe] “official MCP server documented at https://docs.bland.ai/integrations/mcp/overview”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   The evidence pack shows testing/simulation tools (pytest/Vitest, LLM-driven simulations) but no built-in production analytics dashboard for success rates, durations, outcomes, or sentiment. A community report explicitly confirms this gap — users running LiveKit voice agents said they had 'no way to measure TTFT, latency across STT→LLM→TTS, or even cost per call' and found debugging painful, indicating no such dashboard exists out of the box.

   • [community] “We've been running multiple voice AI agents on LiveKit and kept running into visibility issues — no way to measure TTFT, latency across STT …”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
   • [claimed-docs] “Agent simulations run end-to-end conversations between your agent and an LLM-driven user, then evaluate the results across the full interact…”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes... Agent simulations run end-to-end conversations between your agent and…”

   proof ↗⚑ flag

Monitoring

1. platform-engineerMonitor live calls in production and get alerts when agents misbehave or error rates spike

   weight 1 · round to Bland
   Bland~partialclaimed4/10

   Bland provides post-call webhooks, evals with LLM judges, and an MCP integration that can 'query analytics' — giving some after-the-fact quality/analytics visibility — but there is no documented live-call monitoring dashboard, real-time alerting, or error-rate-spike notification system in the evidence pack. Missing for 10: live/real-time call monitoring UI, configurable alert thresholds, error-rate spike detection/paging, independent confirmation these exist in production.

   • [claimed-docs] “Post-call webhooks are HTTP notifications that Bland AI automatically sends to your server after a phone call completes.”
   • [claimed-docs] “Evals let you measure the quality of your calls with LLM judges.”
   • [claimed-docs] “Evals let you measure the quality of your calls with LLM judges. You define eval agents, each of which grades one dimension of a call”
   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
   • [claimed-docs] “The testbed lets you take any call — whether it's a historical call from call logs or a test chat from the pathway editor — isolate a specif…”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   No evidence pack items describe production monitoring dashboards, error-rate alerting, or call-quality observability tooling for LiveKit Agents; the only related evidence is a community report explicitly describing the lack of visibility into TTFT, per-call latency, and cost — a gap, not a delivered capability.

   • [community] “We've been running multiple voice AI agents on LiveKit and kept running into visibility issues — no way to measure TTFT, latency across STT …”

   proof ↗⚑ flag

Testing

1. developerTest agents with simulated conversations or evals before putting them on real phone calls

   weight 2 · round drawn
   Bland✓fullclaimed8/10

   Bland provides explicit pre-production testing tools: Evals (LLM-judge grading of call quality), the Testbed (replay/edit/re-run node interactions on historical or test chats), staging promotion and draft-vs-production pathway separation, and canary/A/B rollout testing against real calls before full deployment. Together these let a developer simulate conversations and grade agent behavior before real phone calls go live. missing for 10: no independent/hands-on evidence of eval accuracy or testbed usage from outside vendor docs, and no explicit description of a pure text-based conversation simulator separate from testbed/staging.

   • [claimed-docs] “Evals let you measure the quality of your calls with LLM judges.”
   • [claimed-docs] “The testbed lets you take any call ... isolate a specific node interaction, edit the prompt, and run it multiple times to see how the output…”
   • [claimed-docs] “Evals let you measure the quality of your calls with LLM judges. You define eval agents, each of which grades one dimension of a call”
   • [claimed-docs] “The testbed lets you take any call — whether it's a historical call from call logs or a test chat from the pathway editor — isolate a specif…”
   • [claimed-docs] “You can also promote a version to staging to test it before it goes live, or send an individual call against a specific”
   • [claimed-docs] “When you edit a pathway, you are working on a draft. Live calls are not affected. They keep using the published production version.”
   • [claimed-docs] “This is how you A/B test a new agent release against your live production version, with real calls, before committing to a full rollout.”
   • [claimed-docs] “Bland supports canary deployments — a way to run a new release on a separate set of containers alongside your production infrastructure and …”

   proof ↗⚑ flag

   LiveKit Agents✓fullclaimed8/10

   Docs explicitly describe both behavioral tests (pytest/Vitest) and agent simulations that run end-to-end conversations with an LLM-driven user, evaluating results before deployment, and GitHub notes a 'builtin test framework' with judges — directly matching the testing-before-real-calls story, complemented by separate telephony/SIP support for real calls. Missing for 10: independent/hands-on validation of the eval/simulation framework and explicit documentation tying test simulations to pre-phone-call validation workflows.

   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes. They integrate with your existing test suite using pytest (Python) or V…”
   • [claimed-docs] “Agent simulations run end-to-end conversations between your agent and an LLM-driven user, then evaluate the results across the full interact…”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes... Agent simulations run end-to-end conversations between your agent and…”
   • [claimed-docs] “Behavioral tests verify specific interactions and expected outcomes... using pytest (Python) or Vitest (Node.js)”
   • [github] “Builtin test framework: Write tests and use judges to ensure your agent is performing as expected.”
   • [github] “Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.”
   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”

   proof ↗⚑ flag

Post call

1. developerExtract structured data from every call — outcomes, entities, dispositions — delivered via API or webhook after the call

   weight 2 · round to Bland
   Bland~partialclaimed6/10

   Bland documents post-call webhooks that automatically deliver call data to a developer's server after each call, and evals let you grade/classify calls (dispositions) via LLM judges — both align with the API/webhook delivery and outcome-tagging parts of the story. However, there's no explicit documentation of structured entity extraction (e.g., named fields like names, dates, custom entities) as a distinct capability, nor a described webhook payload schema. Missing for 10: explicit entity-extraction feature docs, sample webhook payload showing structured outcome/entity/disposition fields, and independent confirmation of the data delivered.

   • [claimed-docs] “Post-call webhooks are HTTP notifications that Bland AI automatically sends to your server after a phone call completes.”
   • [claimed-docs] “Evals let you measure the quality of your calls with LLM judges. You define eval agents, each of which grades one dimension of a call”
   • [claimed-docs] “Give your agent instructions on how it should respond at specific points of the conversation. Choose between prompting or fixed sentences. E…”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   LiveKit Agents provides tool-calling, MCP integration, and telephony hooks, but nothing in the evidence describes a built-in mechanism for automatically extracting structured call outcomes/entities/dispositions and delivering them via API or webhook after a call ends — that would have to be custom-built by the developer using the tool-calling primitives. Since no such capability is documented, this applicable axis is unmet.

   • [claimed-docs] “Call external APIs or lookup data for RAG.”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use. This feature allows you to create a custom library of tools to extend your agent's context”
   • [claimed-docs] “LiveKit telephony lets you build AI-powered voice apps that handle inbound and outbound calls.”
   • [claimed-docs] “Outbound trunks are used to place outgoing calls.”

   proof ↗⚑ flag

Tools

1. ai-native userMy voice agent can plug in MCP servers as tool sources so one integration grants it whole toolsets mid-call

   weight 2 · round to LiveKit Agents
   Bland—none0/10

   Bland ships an MCP *server* so coding agents can control the Bland account (docs-11/20/27, probe-3/probe-rt-2), which is the opposite role from what the story asks — the voice agent itself acting as an MCP *client* that plugs in external MCP servers as tool sources mid-call. Tool/function calling is documented only via custom HTTP API integrations (bland-docs-4, bland-docs-25), with no mention of the agent consuming MCP servers as a toolset source during calls.

   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
   • [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
   • [probe] “official MCP server documented at https://docs.bland.ai/integrations/mcp/overview”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP endpoint https://api.bland.ai/v1/mcp returned HTTP 4…”
   • [claimed-docs] “The API integration lets you connect your AI agent to any HTTP endpoint.”
   • [claimed-docs] “Connect external APIs and take live actions during phone calls.”

   proof ↗⚑ flag

   LiveKit Agents✓fullclaimed9/10

   LiveKit Agents has first-class, documented MCP support ("first-class support for Model Context Protocol (MCP) servers", "Integrate tools provided by MCP servers with one line of code") via `MCPToolset`, which wraps an MCP server and passes it directly to the agent's tools parameter — enabling mid-call toolset access for voice agents. This is corroborated across both docs and GitHub README. Missing for 10: independent hands-on validation of actual mid-call MCP toolset usage beyond vendor docs.

   • [claimed-docs] “Wrap an MCP server in an \`MCPToolset\` and pass it to the agent's \`tools\` parameter”
   • [claimed-docs] “Wrap an MCP server in an `MCPToolset` and pass it to the agent's `tools` parameter”
   • [claimed-docs] “LiveKit Agents has first-class support for Model Context Protocol (MCP) servers.”
   • [github] “MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.”
   • [github] “Native support for MCP. Integrate tools provided by MCP servers with one line of code.”

   proof ↗⚑ flag

2. developerMy agent can call external APIs and custom functions mid-conversation and speak the result without awkward dead air

   weight 3 · round to LiveKit Agents
   Bland~partialclaimed6/10

   Bland's pathway/tools docs confirm agents can call external APIs and execute webhooks mid-conversation (bland-docs-4, bland-docs-25, bland-docs-30), which supports live function calling during a call. However, no evidence describes mechanisms for avoiding dead air (e.g., filler speech, streaming partial responses) while waiting on API results. missing for 10: explicit documentation of latency-masking/filler-speech behavior during API calls, and independent/hands-on confirmation of smooth conversational flow.

   • [claimed-docs] “The API integration lets you connect your AI agent to any HTTP endpoint.”
   • [claimed-docs] “Connect external APIs and take live actions during phone calls.”
   • [claimed-docs] “Give your agent instructions on how it should respond at specific points of the conversation. Choose between prompting or fixed sentences. E…”
   • [claimed-docs] “Conversational pathways are our new way of prompting Bland that has led to major breakthroughs in realism. Give your agent instructions on h…”

   proof ↗⚑ flag

   LiveKit Agents✓fullprobed9/10

   LiveKit Agents has full documented support for LLM tool/function calling (custom Python/Node functions calling external APIs mid-conversation) plus MCP server tool integration, combined with adaptive interruption handling and turn-detection to avoid awkward dead air while tools execute. Runtime probes confirm the framework is real and installable, corroborating the docs claims. missing for 10: independent/hands-on evidence specifically showing tool-call latency handled gracefully in a live conversation (only vendor docs cover this exact combination).

   • [claimed-docs] “Call external APIs or lookup data for RAG.”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use. This feature allows you to create a custom library of tools to extend your agent's context”
   • [claimed-docs] “LiveKit Agents has full support for LLM tool use.”
   • [claimed-docs] “LiveKit Agents has first-class support for Model Context Protocol (MCP) servers.”
   • [claimed-docs] “Wrap an MCP server in an \`MCPToolset\` and pass it to the agent's \`tools\` parameter”
   • [claimed-docs] “Adaptive interruption handling allows an agent to respond naturally when users speak mid-response... to identify intentional interruptions (…”
   • [claimed-docs] “the model analyzes the acoustic signals to identify intentional interruptions (barge-ins) from conversational backchanneling”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi livekit-agents 1.8.0 installs and `import livekit.agents` succeeds with no key (npm @livekit/agent…”

   proof ↗⚑ flag

Recording

1. platform-engineerRetrieve full call recordings and transcripts programmatically for every call

   weight 2 · round drawn
   Bland—none0/10

   No evidence pack item explicitly documents an API or endpoint for retrieving full call recordings or transcripts programmatically; references to call logs (testbed) and post-call webhooks hint at underlying data but never confirm a recording/transcript retrieval capability.

   ⚑ flag

   LiveKit Agents—none0/10

   The evidence pack covers agent building, tool use, MCP, telephony, and testing but contains no mention of recording APIs, transcript storage, or programmatic retrieval of call recordings/transcripts for platform engineers.

   ⚑ flag

Transcription

1. developerGet accurate real-time transcription with control over the STT provider, language models, or key terms

   weight 2 · round to LiveKit Agents
   Bland—none0/10

   Bland is a phone-call AI platform where transcription accuracy is clearly relevant, but the evidence pack contains no mention of STT provider selection, language model choice for transcription, or key-term/vocabulary boosting features. missing for 10: STT provider selection, transcription accuracy documentation, custom key terms/vocabulary support, language model configuration for transcription.

   ⚑ flag

   LiveKit Agents~partialclaimed6/10

   LiveKit Agents supports swapping STT/LLM providers via its open-source plugin ecosystem and lets you override API key/base URL for any provider (docs-12, docs-24, docs-35), giving real control over transcription and language-model choice. However there is no evidence of support for custom vocabulary/key-term or hotword boosting in STT, and no independent benchmark of transcription accuracy. Missing for 10: key-term/hotword/phrase-hint configuration support, accuracy benchmarks or independent corroboration of transcription quality.

   • [claimed-docs] “LiveKit Agents includes a large ecosystem of open source plugins for a variety of AI providers.”
   • [claimed-docs] “You can use LiveKit Inference to access many of these models directly through LiveKit Cloud, or you can use the open source plugins to conne…”
   • [claimed-docs] “For any provider not included, you can override the API key and base URL at initialization for the LLM, STT, and TTS interfaces in the plugi…”

   proof ↗⚑ flag

Voices

1. founderClone a custom brand voice and use it for my agents, with a documented consent process

   weight 2 · round to Bland
   Bland~partialclaimed4/10

   Bland documents voice cloning itself (a ~10-second clean sample sets the quality ceiling) which supports the 'clone a custom brand voice' half of the story, but no evidence describes a documented consent process, verification, or authorization requirement for cloning someone's voice. missing for 10: documented consent/verification workflow for voice cloning, legal/compliance guidance on brand-voice rights, independent confirmation of the cloning feature's fidelity.

   • [claimed-docs] “A clone needs one clean sample of about ten seconds. Quality of the sample sets the ceiling on quality of the voice”

   proof ↗⚑ flag

   LiveKit Agents—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

2. developerChoose from a broad voice library or plug in multiple TTS providers to get the voice I want

   weight 2 · round to LiveKit Agents
   Bland~partialclaimed3/10

   Docs show voice cloning support (a 10-second sample sets voice quality) but there is no evidence of a broad pre-built voice library to choose from, nor any mention of plugging in multiple third-party TTS providers. Missing for 10: documented voice library/catalog, multi-provider TTS integration, and any comparison of voice options.

   • [claimed-docs] “A clone needs one clean sample of about ten seconds. Quality of the sample sets the ceiling on quality of the voice”

   proof ↗⚑ flag

   LiveKit Agents~partialclaimed6/10

   Docs confirm a large open-source plugin ecosystem for TTS/STT/LLM providers, plus LiveKit Inference for provider access, and an override mechanism for API key/base URL for providers not natively included, supporting multi-provider TTS flexibility. However, there's no explicit mention of a 'voice library' (e.g., curated voice catalog/selection UI) or enumeration of specific TTS providers/voices, so the 'broad voice library' half of the story is unevidenced. Missing for 10: explicit voice catalog/list of supported TTS providers and voices, evidence of ease of switching between TTS voices, independent confirmation of provider breadth.

   • [claimed-docs] “LiveKit Agents includes a large ecosystem of open source plugins for a variety of AI providers.”
   • [claimed-docs] “You can use LiveKit Inference to access many of these models directly through LiveKit Cloud, or you can use the open source plugins to conne…”
   • [claimed-docs] “For any provider not included, you can override the API key and base URL at initialization for the LLM, STT, and TTS interfaces in the plugi…”

   proof ↗⚑ flag