ElevenLabs Agents vs Bolna
ElevenLabs Agents wins · 23–17 (21 drawn)
Agent building — building agents — abstractions, tool wiring, control flowAgent building
Building agents — abstractions, tool wiring, control flow
Agent ops
ai-native userMy coding agent can provision a complete voice agent end to end — create the agent, attach a number, and place a call — through the API, CLI, or MCP without touching the dashboard
weight 3 · round to BolnaDocs and runtime probes confirm agents can be created and managed via API, a real CLI (@elevenlabs/cli, verified live on npm), and a hosted MCP server (verified live with OAuth), all without the dashboard (docs-1,3,4,27,28,38; probe-rt-1,2). Phone-number attachment is also supported via SIP trunking to route calls to agents (docs-12,36), but there is no explicit documentation or evidence of actually placing/initiating an outbound call end-to-end through the API/CLI/MCP — only routing inbound infrastructure integration is described. Missing for 10: explicit API/CLI/MCP call to place an outbound call, and a concrete end-to-end walkthrough combining create+attach+call in one non-dashboard flow.
- [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
- [claimed-docs] “Use the ElevenLabs agents skill to build and manage voice agents from your AI coding assistant: npx skills add elevenlabs/skills --skill age…”
- [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
- [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”
- [claimed-docs] “Route calls to AI agents without changing your existing phone infrastructure”
- [claimed-docs] “connect the hosted MCP server to Claude or any other MCP client to create and manage agents through natural language”
- [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”
Bolna offers all three surfaces needed for agent-native provisioning: a documented API quickstart to create an agent, attach a number, and place a call (bolna-docs-14, bolna-docs-15), a native Go CLI to manage agents, calls, and phone numbers from the terminal (bolna-docs-16, bolna-probe-4), and an official MCP server to connect Claude/Cursor/Windsurf clients (bolna-docs-17, bolna-probe-3), all without requiring dashboard interaction. missing for 10: independent/hands-on confirmation that MCP or CLI can complete the full create-attach-call flow end-to-end (only vendor docs), and no OpenAPI spec was resolvable (bolna-probe-2) to verify API completeness.
- [claimed-docs] “Make your first Voice AI call with the Bolna API — verify your key, get an agent, place a call, and fetch the transcript in about five minut…”
- [claimed-docs] “Click the Create a new API Key button to generate your unique authentication credentials.”
- [claimed-docs] “The Bolna CLI (bolna) is a single Go binary that lets you manage agents, calls, phone numbers, and batches directly from your terminal”
- [claimed-docs] “Connect Claude, Cursor, Windsurf, or any MCP client to your Bolna account with the Bolna MCP server.”
- [probe] “official MCP server documented at https://www.bolna.ai/docs/build-with-ai/mcp”
- [probe] “official CLI documented at https://www.bolna.ai/docs/cli/introduction”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.bolna.ai/openapi.json, https://docs.bolna.ai/swagger.json, https://docs.bolna.ai/api/op…”
ai-native userThe platform's own AI helps me author agents — generating or improving prompts, flows, and test cases from a description
weight 1 · round to ElevenLabs AgentsElevenLabs documents that an external AI assistant (e.g., Claude) can create, configure, and manage agents through natural language via the hosted MCP server, and agent-testing docs describe turning real conversations into test cases for regression testing — both are AI-assisted authoring flows. However, this relies on a third-party AI client (Claude) rather than a built-in, platform-native 'describe your agent and we generate it' copilot for prompts/flows within the dashboard itself. Missing for 10: evidence of a first-party generative wizard that authors prompts/flows purely from a natural-language description inside the ElevenLabs UI, and evidence of AI-generated (not just conversation-derived) test-case authoring.
- [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
- [claimed-docs] “connect the hosted MCP server to Claude or any other MCP client to create and manage agents through natural language”
- [claimed-docs] “Transform real conversations into test cases when you find an interaction where the agent underperformed.”
- [claimed-docs] “Agent testing lets you verify conversational responses, tool usage, and full multi-turn outcomes before you deploy.”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”
Bolnanone0/10No evidence shows Bolna's own platform AI generating or improving prompts, flows, or test cases from a description; the closest feature (build-with-ai/mcp) is an MCP server that lets external AI clients like Claude or Cursor manage the account via API, not the platform itself authoring agents. Templates can be cloned, but that's static copying, not AI-generated authoring.
- [claimed-docs] “Connect Claude, Cursor, Windsurf, or any MCP client to your Bolna account with the Bolna MCP server.”
- [claimed-docs] “Clone any template and customize it for your business.”
Build
developerBuild a working phone voice agent — prompt, voice, and phone number — and take my first live call within an hour
weight 3 · round to BolnaDocs show a clear quickstart path — create an agent via dashboard/API, pick from 5k+ voices, and connect phone numbers via SIP trunking to route calls to the agent without changing existing phone infrastructure — which together cover the prompt+voice+phone number pieces of the story. However, there's no explicit end-to-end 'first call in under an hour' walkthrough or independent timing evidence, and phone-number provisioning specifics (vs. just SIP integration) are thin. Missing for 10: an explicit timed/complete quickstart-to-live-call walkthrough, first-party or independent evidence of the actual time-to-first-call, and clearer native phone-number purchase/setup docs beyond SIP trunking.
- [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
- [claimed-docs] “Select from 5k+ voices across 31 languages with customization options”
- [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”
- [claimed-docs] “Route calls to AI agents without changing your existing phone infrastructure”
- [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”
- [claimed-docs] “This assistant can be embedded into your website or app to provide real-time support to your customers”
Docs directly support the story: templates can be cloned and customized, build/deploy claimed under 5 minutes, voice/accent selection, phone number and telephony setup (BYOT/SIP), and a quickstart walking through key verification, agent creation, placing a call, and fetching a transcript in ~5 minutes. This covers prompt (agent config/templates), voice (voice/provider selection), and phone number (telephony/SIP) end-to-end for a first live call within an hour. Missing for 10: independent/hands-on confirmation of the 'under 5 minutes' or 'first call in an hour' claims beyond vendor docs, and no third-party success report.
- [claimed-docs] “Build and deploy in under 5 minutes”
- [claimed-docs] “Clone any template and customize it for your business.”
- [claimed-docs] “You can choose different voices, accents, and speaking styles to match your brand.”
- [claimed-docs] “Bring Your Own Telephony (BYOT) lets you connect any standards-compliant SIP trunk to the Bolna platform.”
- [claimed-docs] “Make your first Voice AI call with the Bolna API — verify your key, get an agent, place a call, and fetch the transcript in about five minut…”
- [claimed-docs] “Bolna supports Deepgram, Azure, ElevenLabs, and more.”
developerRun conversations in multiple languages, including detecting and switching language mid-call
weight 2 · round to BolnaDocs confirm broad multi-language support (70+ languages, language-specific voice settings) and voice-switching for multi-character use, but there is no explicit documentation of automatic language detection or switching languages mid-call. Missing for 10: explicit language-detection/mid-call switching feature docs, and any hands-on demonstration of a live language switch during a single conversation.
- [claimed-docs] “Configure, deploy and monitor natural, human-sounding agents in 70+ languages with leading accuracy and ultra-low latency across voice or ch…”
- [claimed-docs] “You can customize various aspects of your AI agent's voice to create a more natural and engaging conversation experience. This includes cont…”
- [claimed-docs] “Enable your agent to switch between different voices for multi-character conversations, storytelling, and language tutoring.”
- [claimed-docs] “Select from 5k+ voices across 31 languages with customization options”
Docs claim the agent "detects and switches mid-call on its own" and marketing mentions "multilingual intelligence" for thousands of calls, directly supporting the story, but there's no detail on supported languages, configuration options, or independent/hands-on verification of mid-call switching reliability. Missing for 10: language list/configuration docs, hands-on or third-party confirmation of switching accuracy, and edge-case handling details.
- [claimed-docs] “the agent detects and switches mid-call on its own”
- [claimed-docs] “Power thousands of inbound and outbound calls every minute with human-like, multilingual intelligence”
founderDesign multi-step conversation flows in a visual builder with branching, states, and handoffs without writing code
weight 2 · round to ElevenLabs AgentsDocs explicitly claim a 'visual workflow builder' for multi-step workflows, plus dashboard-based agent creation, conversation-flow settings (turn-taking, interruptions, timeouts), and tools/handoff-like actions (webhooks, client tools) — supporting the no-code, branching, multi-step story. However, evidence never details branching logic, explicit 'states' or agent-to-agent 'handoffs' within the builder, nor independent hands-on confirmation of the visual builder's depth. Missing for 10: concrete detail/screenshots of branching and state logic in the builder, explicit handoff-between-agents feature description, and independent/third-party corroboration of the builder's no-code sufficiency.
- [claimed-docs] “Build multi-step workflows with visual workflow builder”
- [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”
- [claimed-docs] “Configure turn-taking, interruptions, and timeout settings”
- [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”
- [claimed-docs] “Enable agents to call clients & APIs to perform actions”
Bolnanone0/10Evidence shows Bolna is built around APIs, CLI, and MCP server for defining agents and multi-step outreach sequences (bolna-docs-12, bolna-docs-16, bolna-docs-17), but there is no mention of a visual/no-code builder with branching, states, or handoffs; workflows are described as 'driven entirely through the Bolna API.' Missing for 10: any visual flow-builder UI, drag-and-drop branching/state design, no-code handoff configuration.
- [claimed-docs] “Build multi-step outreach sequences that combine voice agent calls, WhatsApp messages, API calls, waits and retries, driven entirely through…”
- [claimed-docs] “The Bolna CLI (bolna) is a single Go binary that lets you manage agents, calls, phone numbers, and batches directly from your terminal”
- [claimed-docs] “Connect Claude, Cursor, Windsurf, or any MCP client to your Bolna account with the Bolna MCP server.”
- [github] “Bolna is the end-to-end open source production ready framework for quickly building LLM based voice driven conversational applications.”
Personalization
developerInject dynamic variables and per-caller context at call time so each conversation is personalized
weight 2 · round to ElevenLabs AgentsDocs explicitly state 'Use dynamic variables and overrides for per-conversation customization' as a core build feature, directly matching the story of injecting per-caller context at call time. Missing for 10: no independent/hands-on example showing a real dynamic-variable injection call in code, and no detail on variable scoping/limits beyond the single doc mention.
- [claimed-docs] “Use dynamic variables and overrides for per-conversation customization”
developerGround the agent on my documents with a built-in knowledge base or RAG so it answers from my content
weight 2 · round to ElevenLabs AgentsDocs explicitly state agents support uploading documents and enabling RAG for grounded responses, directly matching the story's built-in knowledge base/RAG capability. Missing for 10: independent/hands-on corroboration of RAG quality and details on knowledge base management (chunking, size limits, sources).
- [claimed-docs] “Upload documents and enable RAG for grounded responses”
Bolnanone0/10Bolna is a voice AI agent platform (telephony, TTS/STT, LLM providers, workflows) but the evidence pack contains no mention of a knowledge base, document ingestion, or RAG capability for grounding agent responses in custom content. This axis clearly applies to conversational agent-building platforms, so absence of evidence yields 'none'.
Agenticness — how well agents can access and operate the productAgenticness
How well agents can access and operate the product
Agent access
ai-native userPoint an agent at llms.txt or agent-oriented docs
weight 2 · round drawnElevenLabs publishes a working llms.txt at the root (HTTP 200) and structures its docs as machine-readable .md pages with an explicit pointer to a full docs/llms.txt index, exactly matching the ai-native 'point an agent at llms.txt' pattern; this was independently verified via live probes rather than just claimed docs. Missing for 10: no evidence of a hands-on agent actually consuming the file end-to-end to complete a task (only that the endpoint/format is live and documented).
- [probe] “PROBE llms.txt: HTTP 200 at https://elevenlabs.io/llms.txt # ElevenLabs ElevenLabs is an AI research and product company transforming how w…”
- [probe] “PROBE docs-md: HTTP 200 at https://elevenlabs.io/docs/eleven-agents/overview.md > This is a page from the ElevenLabs documentation. For a co…”
A probe confirms llms.txt is live and returns 200 with structured agent-oriented doc links (bolna-probe-1), and Bolna also documents an official MCP server for connecting AI clients like Claude/Cursor to its docs/account (bolna-docs-17, bolna-probe-3), directly enabling agent-native consumption of its documentation. Missing for 10: no independent third-party confirmation that agents successfully use llms.txt in practice.
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.bolna.ai/llms.txt # Bolna Docs > Quickly build and deploy conversational Voice AI agents - [Home]…”
- [claimed-docs] “Connect Claude, Cursor, Windsurf, or any MCP client to your Bolna account with the Bolna MCP server.”
- [probe] “official MCP server documented at https://www.bolna.ai/docs/build-with-ai/mcp”
ai-native userRun the product headlessly / in CI for automation
weight 2 · round drawnElevenLabs Agents can be managed via a full REST API and an official CLI (verified runtime install/run), enabling scripted/headless creation and configuration of agents outside the dashboard, which supports CI-style automation. However, evidence is docs/probe-based with no CI pipeline examples, no first-class 'run in CI' guidance, and Agent testing is described as pre-deploy validation without confirmation it can be triggered non-interactively/headlessly. Missing for 10: explicit CI/CD integration examples, headless testing invocation docs, and independent hands-on confirmation of automated pipelines.
- [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
- [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”
- [claimed-docs] “Agent testing lets you verify conversational responses, tool usage, and full multi-turn outcomes before you deploy.”
- [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”
Bolna offers a CLI (single Go binary) for managing agents, calls, phone numbers, and batches from the terminal, plus a REST API for scripted call creation and transcript retrieval — both usable in CI/automation pipelines without a GUI. However, there's no explicit CI/pipeline example, headless-mode documentation, or exit-code/automation-specific guidance. missing for 10: explicit CI integration examples/guides, documented non-interactive/headless flags or exit codes, third-party confirmation of CI usage.
- [claimed-docs] “The Bolna CLI (bolna) is a single Go binary that lets you manage agents, calls, phone numbers, and batches directly from your terminal”
- [probe] “official CLI documented at https://www.bolna.ai/docs/cli/introduction”
- [claimed-docs] “Make your first Voice AI call with the Bolna API — verify your key, get an agent, place a call, and fetch the transcript in about five minut…”
- [claimed-docs] “Click the Create a new API Key button to generate your unique authentication credentials.”
ai-native userPlug MCP servers into this product so it can use their tools
weight 3 · round to ElevenLabs AgentsDocs explicitly describe connecting conversational agents to external MCP servers so agents can access data sources and tools via MCP, directly matching the story, and this is corroborated by dedicated documentation pages (mcp.md) rather than just passing mention. Missing for 10: independent/hands-on verification of an actual third-party MCP server being plugged in and tool-calls succeeding in production (only docs and a runtime probe on the separate hosted-server feature are present).
- [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers. This enables your agents to: Access and process informa…”
- [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers.”
- [claimed-docs] “Access and process information from various data sources via the MCP server”
Bolnanone0/10Evidence shows Bolna ships an MCP *server* so external clients like Claude, Cursor, or Windsurf can control a Bolna account (bolna-docs-17, bolna-probe-3) — this is the reverse direction from the story, which asks whether Bolna's own agents can plug in external MCP servers to gain their tools. No evidence describes Bolna voice agents consuming or calling external MCP servers.
- [claimed-docs] “Connect Claude, Cursor, Windsurf, or any MCP client to your Bolna account with the Bolna MCP server.”
- [probe] “official MCP server documented at https://www.bolna.ai/docs/build-with-ai/mcp”
ai-native userConnect an agent via an official MCP server
weight 3 · round to ElevenLabs AgentsElevenLabs explicitly documents and operates a hosted MCP server (api.elevenlabs.io/v1/mcp) that lets external MCP clients like Claude connect to create, configure, and manage agents through natural language, with a runtime probe confirming the live endpoint correctly responds with the MCP OAuth challenge. This is a first-party 'official MCP server' offering, not just client-side MCP consumption, matching the story precisely. Missing for 10: independent/community hands-on confirmation beyond ElevenLabs' own docs and the single runtime probe.
- [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
- [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
- [claimed-docs] “connect the hosted MCP server to Claude or any other MCP client to create and manage agents through natural language”
- [claimed-docs] “create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally”
- [probe] “official MCP server documented at https://elevenlabs.io/docs/eleven-agents/operate/hosted-mcp”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”
Bolna documents an official MCP server that lets Claude, Cursor, Windsurf, or any MCP client connect to a Bolna account, confirmed by both docs and an independent probe hitting the live page. Missing for 10: independent hands-on validation or community confirmation of the MCP server working in practice beyond the docs/probe.
- [claimed-docs] “Connect Claude, Cursor, Windsurf, or any MCP client to your Bolna account with the Bolna MCP server.”
- [probe] “official MCP server documented at https://www.bolna.ai/docs/build-with-ai/mcp”
ai-native userUse an official CLI
weight 2 · round to ElevenLabs AgentsElevenLabs documents an official Agents CLI for managing agents, and runtime probing confirms `npx @elevenlabs/cli` installs and runs keylessly from npm, corroborating the docs claim with hands-on evidence. Missing for 10: deeper documentation of full CLI command surface/subcommands beyond basic install confirmation.
- [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
- [claimed-docs] “Use the ElevenLabs agents skill to build and manage voice agents from your AI coding assistant: npx skills add elevenlabs/skills --skill age…”
- [claimed-docs] “Use the ElevenLabs agents skill to build and manage voice agents from your AI coding assistant”
- [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”
Bolna documents an official CLI (single Go binary) letting users manage agents, calls, phone numbers, and batches from the terminal, corroborated by a live docs probe confirming the page exists. missing for 10: no independent/hands-on community usage reports or version/release history confirming real-world adoption beyond first-party docs.
- [claimed-docs] “The Bolna CLI (bolna) is a single Go binary that lets you manage agents, calls, phone numbers, and batches directly from your terminal”
- [probe] “official CLI documented at https://www.bolna.ai/docs/cli/introduction”
ai-native userDrive the product through a documented public API
weight 3 · round to ElevenLabs AgentsElevenLabs Agents ships a documented public API alongside a dashboard, CLI, and hosted MCP server for programmatic agent creation/management, with quickstart docs explicitly noting 'creating an agent via the API' and full customization (tools, voice, analysis, testing) accessible programmatically. Runtime probes confirm the CLI installs keylessly and the hosted MCP endpoint is live and functional. Missing for 10: no independent third-party API reference/SDK usage examples or community corroboration beyond first-party docs.
- [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
- [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”
- [claimed-docs] “Enable agents to call clients & APIs to perform actions”
- [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”
Bolna provides a documented public API with quickstart, authentication, agent/call/extraction endpoints, and workflow automation entirely through the API, plus a CLI and MCP server built on top of it. Missing for 10: a discoverable OpenAPI/Swagger spec (probe found all standard OpenAPI paths 404) and independent third-party corroboration of API robustness.
- [claimed-docs] “Make your first Voice AI call with the Bolna API — verify your key, get an agent, place a call, and fetch the transcript in about five minut…”
- [claimed-docs] “Click the Create a new API Key button to generate your unique authentication credentials.”
- [claimed-docs] “Build multi-step outreach sequences that combine voice agent calls, WhatsApp messages, API calls, waits and retries, driven entirely through…”
- [claimed-docs] “The Bolna CLI (bolna) is a single Go binary that lets you manage agents, calls, phone numbers, and batches directly from your terminal”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.bolna.ai/openapi.json, https://docs.bolna.ai/swagger.json, https://docs.bolna.ai/api/op…”
- [probe] “official CLI documented at https://www.bolna.ai/docs/cli/introduction”
ai-native userIssue scoped/least-privilege API credentials for an agent
weight 2 · round drawnElevenLabs Agentsnone0/10Evidence covers agent-side authentication (protecting inbound access to a deployed agent) and OAuth for the hosted MCP server, but there is no documentation of issuing scoped or least-privilege API keys/tokens that limit what an agent itself can do via the ElevenLabs API. Missing for 10: any docs on API key permission scopes, workspace role-based tokens, or restricted-credential issuance for agents.
- [claimed-docs] “Secure agent access | Authentication | Implement custom authentication for protected agent access”
- [claimed-docs] “Implement custom authentication for protected agent access”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”
Bolnanone0/10Bolna documents API key creation but there is no evidence of scoped or least-privilege credential controls (e.g., role-based permissions, scoped tokens, restricted access levels) — only a single account-wide key generation flow is described.
- [claimed-docs] “Click the Create a new API Key button to generate your unique authentication credentials.”
ai-native userBuild against official SDKs
weight 2 · round to BolnaDocs confirm programmatic access via the ElevenLabs API and an official CLI (`@elevenlabs/cli`, verified runtime) for building agents, but no evidence pack item explicitly documents a first-party language SDK (Python/Node/etc.) for Agents beyond the generic API reference. Missing for 10: explicit official SDK docs/repos, SDK code examples, and independent confirmation of SDK feature parity with the API.
- [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
- [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”
- [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”
Bolna documents an official Web Call SDK (bolna-docs-18) under a 'developer-resources/sdks' path, plus a CLI and MCP server, giving AI-native builders some official tooling to build against, but there is no evidence of broader language SDKs (e.g., Python, Node, Go client libraries) beyond the browser-based Web Call SDK. missing for 10: multi-language server-side SDKs, independent/hands-on verification of SDK usability, breadth beyond the single web-call SDK.
- [claimed-docs] “The Web Call SDK connects a browser tab directly to a Bolna voice agent for a live, two-way conversation.”
- [claimed-docs] “The Bolna CLI (bolna) is a single Go binary that lets you manage agents, calls, phone numbers, and batches directly from your terminal”
- [probe] “official CLI documented at https://www.bolna.ai/docs/cli/introduction”
- [claimed-docs] “Connect Claude, Cursor, Windsurf, or any MCP client to your Bolna account with the Bolna MCP server.”
ai-native userSubscribe to events via webhooks
weight 2 · round drawnElevenLabs Agentsnone0/10The evidence describes 'webhook tools' that let an agent make outbound calls to external endpoints during a conversation (docs-14, docs-29, docs-31), which is the opposite of subscribing to platform-emitted events via webhooks. There is no documented mechanism for a developer to register a webhook URL to receive event notifications (e.g., conversation-ended, call-completed) from ElevenLabs Agents.
- [claimed-docs] “Tools give your assistant the ability to make external function calls to third party apps so you can get real-time information.”
- [claimed-docs] “Tools allow ElevenLabs agents to perform actions beyond generating text responses.”
- [claimed-docs] “allow an assistant to trigger authenticated actions based on the conversation, like scheduling meetings or initiating order returns”
Bolnanone0/10No evidence in the pack mentions webhooks or event subscription for Bolna; the docs cover agents, calls, CLI, MCP, SIP, pricing, and security but nothing about outbound event notifications. Missing for 10: any webhook registration/config API, event payload documentation, or third-party confirmation of webhook delivery.
Agentic features
ai-native userGet AI-generated insights and suggestions from my data inside the product
weight 2 · round drawnAgent-analysis features (sentiment analysis, success evaluation, structured data extraction, semantic conversation search) generate AI-derived insights from the user's own conversation data, which partially satisfies the story. However, these are evaluation/reporting metrics rather than proactive AI-generated suggestions or recommendations for next actions, and there is no independent/hands-on corroboration of this analytics experience. Missing for 10: evidence of proactive suggestion generation (e.g., recommended actions/optimizations), a dashboard summarization view, and independent verification beyond first-party docs.
- [claimed-docs] “Success evaluation: Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
- [claimed-docs] “Data collection: Extract specific data points from conversations such as contact information, issue details, or any structured information”
- [claimed-docs] “Sentiment analysis: Understand user sentiment across completed conversations”
- [claimed-docs] “Searching conversations: Find messages by keyword or meaning across your conversation history.”
- [claimed-docs] “Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
- [claimed-docs] “Extract structured information from conversations such as contact details and business data”
Bolna offers agent-level call_summary_enabled to auto-generate AI summaries and an extraction-categories API to pull structured insights out of call data, both of which are AI-generated insights derived from user data (calls). However, there's no evidence of broader cross-call analytics, trend insights, or proactive suggestions surfaced in a dashboard beyond per-call outputs. Missing for 10: aggregated/dashboard-level insights across multiple calls, proactive recommendation surfacing, and independent confirmation of quality of generated summaries/extractions.
- [claimed-docs] “POST /agent/{agent_id}/extraction-categories — create a category (name + model) and attach it to an agent”
- [claimed-docs] “Call summaries are an agent-level toggle: call_summary_enabled (boolean, off by default) on the agent create and update APIs”
ai-native userSet up automations that run autonomously in the background
weight 2 · round to BolnaElevenLabs Agents supports tool-calling, webhook actions, and multi-step workflows that let a voice agent perform actions (scheduling, API calls, MCP access) during a conversation, and agents can be created/managed via API for programmatic triggering, which give some automation flavor. But all documented action-taking is tied to a live conversational turn (voice/chat) rather than an autonomous background process that runs without a triggering conversation or human interaction, so the 'runs autonomously in the background' framing is only weakly supported. Missing for 10: evidence of scheduled/cron-triggered autonomous runs, evidence of multi-step tasks executing without a live conversation session, and independent confirmation of background automation outside the demoed conversational/tool-call flow.
- [claimed-docs] “Build multi-step workflows with visual workflow builder”
- [claimed-docs] “Enable agents to call clients & APIs to perform actions”
- [claimed-docs] “Tools give your assistant the ability to make external function calls to third party apps so you can get real-time information.”
- [claimed-docs] “allow an assistant to trigger authenticated actions based on the conversation, like scheduling meetings or initiating order returns”
- [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers. This enables your agents to: Access and process informa…”
- [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”
Bolna's workflows feature lets users build multi-step outreach sequences (voice calls, WhatsApp, API calls, waits, retries) driven entirely through the API, which supports background automation, and the CLI can manage batches of calls autonomously. However there's no documented evidence of scheduling/triggers, monitoring/alerting on autonomous runs, or a dashboard for managing long-running background jobs beyond the API-driven workflow primitives. missing for 10: scheduling/trigger mechanisms, autonomous-run monitoring/alerting, independent hands-on confirmation of unattended long-running workflows.
- [claimed-docs] “Build multi-step outreach sequences that combine voice agent calls, WhatsApp messages, API calls, waits and retries, driven entirely through…”
- [claimed-docs] “The Bolna CLI (bolna) is a single Go binary that lets you manage agents, calls, phone numbers, and batches directly from your terminal”
- [probe] “official CLI documented at https://www.bolna.ai/docs/cli/introduction”
ai-native userDelegate tasks to a built-in AI assistant inside the product
weight 3 · round to ElevenLabs AgentsElevenLabs documents a first-party hosted MCP server letting an assistant like Claude create, configure, and manage agents in the workspace via natural language, with nothing to install locally — a form of task delegation to an AI assistant for managing the product. However this is not a native in-product chat/assistant UI; it requires an external MCP client (e.g., Claude) to act as the assistant, and delegation is scoped to agent-management tasks rather than general product tasks. Missing for 10: a built-in assistant surface inside the ElevenLabs dashboard itself, evidence of broader task delegation beyond agent CRUD, and independent hands-on confirmation of the natural-language workflow.
- [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
- [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language”
- [claimed-docs] “connect the hosted MCP server to Claude or any other MCP client to create and manage agents through natural language”
- [claimed-docs] “create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”
ai-native userOperate the product with natural-language commands
weight 2 · round to ElevenLabs AgentsElevenLabs Agents ships a hosted MCP server explicitly designed so assistants like Claude can 'create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally,' and a runtime probe confirms the endpoint is live and speaks the MCP OAuth flow. This is a direct, first-party natural-language control surface, not just a client-side integration. Missing for 10: independent/community hands-on confirmation of a full natural-language agent-management session (only endpoint liveness was probed, not an actual successful NL task).
- [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
- [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language”
- [claimed-docs] “connect the hosted MCP server to Claude or any other MCP client to create and manage agents through natural language”
- [claimed-docs] “create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”
- [probe] “official MCP server documented at https://elevenlabs.io/docs/eleven-agents/operate/hosted-mcp”
Bolna ships an official MCP server that lets AI-native tools like Claude, Cursor, or Windsurf connect to a Bolna account and operate it (manage agents, calls, etc.) via natural-language commands, confirmed both in docs and via a live probe. Missing for 10: independent/hands-on verification that the MCP integration works as described, and no additional natural-language control surfaces (e.g., chat-based dashboard) beyond MCP.
- [claimed-docs] “Connect Claude, Cursor, Windsurf, or any MCP client to your Bolna account with the Bolna MCP server.”
- [probe] “official MCP server documented at https://www.bolna.ai/docs/build-with-ai/mcp”
Api quality
ai-native userExplore an interactive API reference with runnable examples
weight 2 · round drawnElevenLabs Agentsnone0/10The evidence pack contains extensive markdown documentation for ElevenLabs Agents (quickstart, customization, tools, etc.) but nothing describes an interactive API reference page with runnable/'try it' examples — no mention of a Swagger/OpenAPI explorer, live code sandbox, or embedded runnable snippets. Since ElevenLabs Agents exposes an API, this axis is applicable, but no evidence supports the specific capability.
Bolnanone0/10Bolna has an API reference doc listing basics like API key creation, but there is no evidence of an interactive reference with runnable/try-it examples, and a probe for a machine-readable OpenAPI spec (which typically powers such interactive docs) returned 404 on all checked paths.
- [claimed-docs] “Click the Create a new API Key button to generate your unique authentication credentials.”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.bolna.ai/openapi.json, https://docs.bolna.ai/swagger.json, https://docs.bolna.ai/api/op…”
ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)
weight 2 · round drawnElevenLabs Agentsnone0/10The evidence pack documents the API, CLI, dashboard, and hosted MCP server for ElevenLabs Agents, but nowhere mentions a downloadable OpenAPI/Swagger spec or machine-readable schema for the API. This axis clearly applies to an API-driven product, but no evidence shows it being delivered.
Bolnanone0/10A direct probe for OpenAPI/Swagger spec files at all standard locations returned 404s, and no evidence pack item shows a downloadable machine-readable API spec despite having an API reference page.
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.bolna.ai/openapi.json, https://docs.bolna.ai/swagger.json, https://docs.bolna.ai/api/op…”
- [claimed-docs] “Click the Create a new API Key button to generate your unique authentication credentials.”
ai-native userTest against a sandbox environment without touching production data
weight 1 · round to ElevenLabs AgentsDocs describe an 'agent testing' feature to verify conversational responses, tool usage, and multi-turn outcomes 'before you deploy', implying a pre-production test workflow, and tool execution can run in a 'sandboxed environment' on ElevenLabs' infrastructure. However there is no explicit documentation of a separate sandbox/staging environment or test data isolation from production conversation/analytics data. Missing for 10: explicit sandbox/staging environment separate from production data, documentation of test-vs-prod data isolation, independent confirmation that test runs don't affect production analytics or logs.
- [claimed-docs] “Agent testing lets you verify conversational responses, tool usage, and full multi-turn outcomes before you deploy.”
- [claimed-docs] “Agent testing lets you verify conversational responses, tool usage, and full multi-turn outcomes before you deploy”
- [claimed-docs] “Custom JavaScript executed in a sandboxed environment on ElevenLabs' infrastructure.”
- [claimed-docs] “Custom JavaScript executed in a sandboxed environment on ElevenLabs' infrastructure”
Bolnanone0/10No evidence pack item mentions a sandbox environment, test mode, staging account, or separation from production data for Bolna; the closest concepts (self-hosting, BYOK, data residency) are about deployment/privacy, not a distinct sandbox/test mode. missing for 10: any mention of a sandbox/test environment, test API keys, or safe testing workflow that avoids production data.
ai-native userRely on versioned APIs with a documented deprecation policy
weight 2 · round drawnElevenLabs Agentsnone0/10No evidence in the pack mentions API versioning scheme, version numbers, or any documented deprecation policy for ElevenLabs Agents' API/CLI/MCP interfaces. missing for 10: versioned API endpoints or version headers, changelog/release notes, explicit deprecation policy or sunset timelines, migration guides for breaking changes.
Bolnanone0/10There is a changelog page referenced and an API reference, but no evidence of API versioning scheme or a documented deprecation policy anywhere in the pack; OpenAPI spec probes even returned 404s. Missing for 10: versioned API endpoints/headers, explicit deprecation policy documentation, migration/versioning changelog entries.
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.bolna.ai/openapi.json, https://docs.bolna.ai/swagger.json, https://docs.bolna.ai/api/op…”
- [claimed-docs] “Click the Create a new API Key button to generate your unique authentication credentials.”
Automation depth — how much of the product can run unattendedAutomation depth
How much of the product can run unattended
ai-native userPerform bulk operations across many items at once
weight 2 · round to BolnaElevenLabs Agentsnone0/10The evidence documents API, CLI, and MCP-based management of agents, but nothing describes bulk/batch operations (e.g., batch-create, batch-update, or batch-delete across many agents or conversations at once). This axis is fair to ask given the product exposes an API and CLI, but no evidence confirms bulk-operation support.
The CLI explicitly manages 'batches' of calls alongside agents and phone numbers (bolna-docs-16), and the platform claims to power 'thousands of inbound and outbound calls every minute' (bolna-docs-22), implying built-in bulk calling capability. However, there is no documentation of a specific bulk API endpoint, CSV/list upload, or bulk agent/category management workflow. Missing for 10: explicit bulk-operation API docs (e.g., batch call creation, bulk agent updates), a documented interface for uploading many items at once, and independent confirmation of bulk throughput.
- [claimed-docs] “The Bolna CLI (bolna) is a single Go binary that lets you manage agents, calls, phone numbers, and batches directly from your terminal”
- [claimed-docs] “Power thousands of inbound and outbound calls every minute with human-like, multilingual intelligence”
- [claimed-docs] “Build multi-step outreach sequences that combine voice agent calls, WhatsApp messages, API calls, waits and retries, driven entirely through…”
ai-native userDefine rules that trigger actions automatically on events
weight 3 · round to ElevenLabs AgentsElevenLabs Agents supports tool-calling and webhook actions that are triggered automatically based on conversation events (e.g., 'trigger authenticated actions based on the conversation, like scheduling meetings or initiating order returns'), plus multi-step visual workflows and MCP-based external actions. This covers event-triggered automation within a conversation but the evidence doesn't show general rule-based triggers on arbitrary system events (e.g., scheduled/external triggers outside a live conversation) — missing for 10: documentation of standalone event/rule engine independent of a live conversation, examples of conditional branching logic beyond tool calls, and independent verification of automation reliability.
- [claimed-docs] “Enable agents to call clients & APIs to perform actions”
- [claimed-docs] “Tools give your assistant the ability to make external function calls to third party apps so you can get real-time information.”
- [claimed-docs] “allow an assistant to trigger authenticated actions based on the conversation, like scheduling meetings or initiating order returns”
- [claimed-docs] “Build multi-step workflows with visual workflow builder”
- [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers. This enables your agents to: Access and process informa…”
Bolna's workflow feature lets users chain voice calls, WhatsApp messages, API calls, waits and retries via the API, which is a form of automation, but the docs never describe defining conditional 'if event X happens then do Y' rules or an event/trigger system — it's presented as sequential outreach flows rather than rule-based event triggers. Missing for 10: explicit event-trigger/rule definition syntax, conditional branching logic, and any webhook-or-event-driven automation examples beyond the sequential workflow description.
- [claimed-docs] “Build multi-step outreach sequences that combine voice agent calls, WhatsApp messages, API calls, waits and retries, driven entirely through…”
ai-native userSchedule recurring jobs or workflows
weight 2 · round drawnElevenLabs Agentsnone0/10The evidence pack covers multi-step workflow building, tool calling, MCP integration, and analysis, but no documentation mentions scheduling recurring jobs, cron-like triggers, or automated recurring workflow execution. This is a fair capability to expect from an automation-focused agent platform, but no evidence supports it.
Bolnanone0/10Bolna's workflow docs describe multi-step outreach sequences with waits and retries (bolna-docs-12), but there is no evidence of a scheduler for recurring/cron-based jobs or workflows that re-trigger automatically over time. missing for 10: recurring/scheduled trigger mechanism, cron-like job definitions, evidence of periodic re-execution.
- [claimed-docs] “Build multi-step outreach sequences that combine voice agent calls, WhatsApp messages, API calls, waits and retries, driven entirely through…”
ai-native userVersion, review, and roll back my automations
weight 1 · round drawnElevenLabs Agentsnone0/10The docs describe building, testing, and analyzing agents (agent-testing, agent-analysis) but there is no mention of version history, change review workflows, or rollback/revert capability for agent configurations. Missing for 10: version history UI, rollback/restore of prior agent configs, change-review/approval workflow, audit trail of edits.
Compliance trust — stories about compliance trust in this arenaCompliance trust
Stories about compliance trust in this arena
Compliance
founderMeet call-recording consent and disclosure obligations with per-call recording controls and configurable data retention
weight 2 · round to ElevenLabs AgentsDocs confirm configurable data-retention settings for conversation transcripts and audio recordings, which addresses the retention half of the story, plus HIPAA/BAA support suggesting compliance posture. However, there is no evidence of per-call recording enable/disable controls or built-in consent/disclosure messaging mechanisms for calls. Missing for 10: per-call recording toggle, consent/disclosure prompt configuration, and any independent verification of retention enforcement.
- [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings.”
- [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings”
- [claimed-docs] “ElevenLabs allows such customers to develop AI-powered voice agents for the handling Protected Health Information (PHI)”
- [claimed-docs] “ElevenLabs Agents is one of ElevenLabs' HIPAA-eligible services, and we offer Business Associate Agreements (BAAs) to eligible customers”
Bolnanone0/10The evidence pack shows security/data-residency features and call-summary toggles, but nothing about call-recording consent, per-call recording enable/disable controls, disclosure/announcement features, or configurable retention periods for recordings/transcripts. No mention of recording consent mechanisms or retention settings anywhere in the docs pack.
platform-engineerRun regulated workloads with HIPAA/BAA support, SOC 2, and data-residency options
weight 2 · round to ElevenLabs AgentsElevenLabs Agents documents HIPAA eligibility and BAA availability for eligible customers, directly supporting regulated PHI workloads. However, there is no evidence in the pack of SOC 2 certification or data-residency options, both explicitly named in the story. Missing for 10: SOC 2 attestation/report evidence, data-residency region selection or documentation, independent audit corroboration of HIPAA/BAA claims.
- [claimed-docs] “ElevenLabs allows such customers to develop AI-powered voice agents for the handling Protected Health Information (PHI)”
- [claimed-docs] “ElevenLabs allows such customers to develop AI-powered voice agents for the handling Protected Health Information (PHI).”
- [claimed-docs] “ElevenLabs Agents is one of ElevenLabs' HIPAA-eligible services, and we offer Business Associate Agreements (BAAs) to eligible customers”
Bolna documents Indian data residency and self-hosting via Docker/Kubernetes for private enterprise deployments, but there is no mention of HIPAA/BAA support or SOC 2 certification anywhere in the evidence pack. missing for 10: HIPAA/BAA attestation, SOC 2 report or certification, broader multi-region data-residency options beyond India.
- [claimed-docs] “Indian data residency is available for deployments where data must remain in India.”
- [claimed-docs] “Learn how to self-host Bolna Voice AI on your own servers using Docker or Kubernetes for secure, private, and scalable enterprise deployment…”
Deployment scale — stories about deployment scale in this arenaDeployment scale
Stories about deployment scale in this arena
Scale
platform-engineerSee documented concurrency limits and scale to many simultaneous calls without manual capacity begging
weight 2 · round to BolnaElevenLabs Agentsnone0/10No evidence pack item documents concurrency limits, per-account call caps, autoscaling behavior, or a process for scaling to many simultaneous calls without contacting sales/support; docs cover features like tools, voice customization, SIP trunking, HIPAA, and MCP but nothing on capacity/scale numbers or self-service scaling guarantees.
Bolna's marketing page claims it can 'power thousands of inbound and outbound calls every minute' and docs mention self-hosting via Docker/Kubernetes for 'scalable enterprise deployments,' but there are no documented concurrency limits, rate-limit tables, or self-serve capacity-scaling mechanisms (e.g., no API rate-limit docs, no autoscaling guide, no numeric SLA). missing for 10: documented numeric concurrency/rate limits, self-serve scale-up process, autoscaling architecture details, independent verification of claimed call volume.
- [claimed-docs] “Power thousands of inbound and outbound calls every minute with human-like, multilingual intelligence”
- [claimed-docs] “Learn how to self-host Bolna Voice AI on your own servers using Docker or Kubernetes for secure, private, and scalable enterprise deployment…”
Self host
platform-engineerSelf-host the voice agent runtime from open-source code on my own infrastructure
weight 3 · round to BolnaElevenLabs Agentsnone0/10ElevenLabs Agents is entirely a managed/hosted service — the CLI and MCP server are clients/interfaces to ElevenLabs' cloud infrastructure, not open-source runtime code that can be deployed on a platform-engineer's own servers. Nothing in the evidence pack mentions open-sourcing the runtime, self-hosting instructions, or on-prem deployment options; all references (dashboard, API, CLI, hosted MCP, SIP trunking) point to ElevenLabs-operated infrastructure.
- [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
- [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
- [claimed-docs] “create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally”
- [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.elevenlabs.io/v1/mcp returned HTT…”
Bolna is explicitly an open-source framework on GitHub, and docs include dedicated enterprise instructions for self-hosting on Docker or Kubernetes for private/enterprise deployments. Missing for 10: independent hands-on reports confirming successful self-host deployment and more detail on infrastructure requirements/limitations of the self-hosted version vs SaaS.
- [github] “Bolna is the end-to-end open source production ready framework for quickly building LLM based voice driven conversational applications.”
- [claimed-docs] “Learn how to self-host Bolna Voice AI on your own servers using Docker or Kubernetes for secure, private, and scalable enterprise deployment…”
Latency turntaking — stories about latency turntaking in this arenaLatency turntaking
Stories about latency turntaking in this arena
Latency
platform-engineerSee documented end-to-end voice latency numbers or tuning guidance backing the platform's speed claims
weight 3 · round to ElevenLabs AgentsElevenLabs markets 'ultra-low latency' (docs-37) and does document conversation-flow tuning knobs like turn-taking, interruption handling and timeouts (docs-33, docs-42, docs-19/49/54), which counts as tuning guidance for latency/turn-taking behavior. However there are no documented end-to-end latency benchmark numbers (e.g., ms figures, percentile breakdowns) anywhere in the evidence pack. Missing for 10: concrete latency benchmark numbers/metrics, methodology for measuring end-to-end latency, and any independent corroboration of the 'ultra-low latency' marketing claim.
- [claimed-docs] “Configure, deploy and monitor natural, human-sounding agents in 70+ languages with leading accuracy and ultra-low latency across voice or ch…”
- [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”
- [claimed-docs] “Configure turn-taking, interruptions, and timeout settings”
- [claimed-docs] “The Max conversation duration setting limits the total time a conversation can remain active. ... The default is 600 seconds (10 minutes).”
Bolnanone0/10No evidence pack item provides documented latency numbers, benchmarks, or tuning guidance for end-to-end voice latency; only vague marketing claims like 'build in under 5 minutes' or 'power thousands of calls' appear, with no actual latency figures or turn-taking tuning documentation.
- [claimed-docs] “Build and deploy in under 5 minutes”
- [claimed-docs] “Power thousands of inbound and outbound calls every minute with human-like, multilingual intelligence”
Turn taking
developerRely on the agent to handle interruptions (barge-in) gracefully — stopping speech, updating context, and recovering the turn
weight 3 · round to ElevenLabs AgentsDocs explicitly document configuring turn-taking, interruptions, and timeout settings, and conversation-flow settings that 'determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavior,' confirming barge-in handling is a first-class configurable feature. However, evidence lacks detail on how context is updated after an interruption or how turn recovery specifically behaves, and there's no hands-on/independent validation of graceful recovery in practice. Missing for 10: technical detail on context truncation/update after interruption, independent or hands-on testing confirming graceful recovery, and latency benchmarks for barge-in specifically.
- [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”
- [claimed-docs] “Configure turn-taking, interruptions, and timeout settings”
- [claimed-docs] “The Max conversation duration setting limits the total time a conversation can remain active. ... The default is 600 seconds (10 minutes).”
Bolnanone0/10No evidence pack item mentions barge-in, interruption handling, speech-stop, or turn recovery mechanics; the docs cover extraction categories, summaries, providers, CLI, MCP, pricing, etc., but nothing about turn-taking or interruption behavior. missing for 10: any documentation of barge-in/interruption detection, mid-speech stop mechanics, context update on interruption, turn recovery logic.
developerEnable noise suppression or audio filtering so the agent stays coherent on noisy real-world calls
weight 1 · round drawnElevenLabs Agentsnone0/10The evidence pack covers voice customization, turn-taking/interruption settings, tools, testing, and telephony (SIP) integration, but nowhere mentions noise suppression, background noise handling, or audio filtering for noisy real-world calls. This is a fair capability to expect for a voice-agent platform handling phone calls, but no documentation or feature reference confirms it exists.
- [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”
- [claimed-docs] “Configure turn-taking, interruptions, and timeout settings”
- [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”
Bolnanone0/10No evidence pack item mentions noise suppression, background noise handling, or audio filtering settings for calls; the docs cover extraction categories, summaries, telephony, SDKs, and pricing but not audio quality/noise robustness features. Missing for 10: any mention of noise suppression/audio filtering configuration, VAD tuning for noisy environments, or documented handling of real-world call noise.
developerUse model-based end-of-turn detection beyond simple VAD silence timeouts so the agent doesn't talk over slow speakers
weight 2 · round drawnElevenLabs Agentsnone0/10Docs mention configurable turn-taking, interruption handling, and timeout settings (docs-33, docs-42), but nothing in the evidence describes a model-based end-of-turn detector that goes beyond a VAD/silence-timeout mechanism — no mention of semantic or ML-based turn-completion detection to avoid interrupting slow speakers. Missing for 10: any documentation of a dedicated end-of-turn prediction model, benchmarks or comparisons versus simple VAD timeouts, or hands-on evidence that the agent waits appropriately for slow speakers.
- [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”
- [claimed-docs] “Configure turn-taking, interruptions, and timeout settings”
- [claimed-docs] “The Max conversation duration setting limits the total time a conversation can remain active. ... The default is 600 seconds (10 minutes).”
Bolnanone0/10No evidence describes model-based end-of-turn/turn-taking detection distinct from VAD silence timeouts; evidence only covers mid-call model switching, extraction categories, and unrelated features. Missing for 10: any mention of endpointing model, semantic turn detection, or configurable end-of-turn logic beyond silence thresholds.
Openness — open source, data portability, and self-hosting storiesOpenness
Open source, data portability, and self-hosting stories
ai-native userDo everything through the API that I can do in the UI
weight 2 · round drawnDocs show agents can be created, configured, and managed via the API, CLI, dashboard, or hosted MCP server, and quickstart explicitly says agents can be created 'via the API or the web dashboard,' implying broad parity across surfaces (elevenlabs-agents-docs-1, docs-39, docs-44). However, there's no explicit feature-by-feature confirmation that every dashboard capability (e.g., visual workflow builder, analytics dashboards, testing UI) has a full API equivalent, nor independent verification of parity. Missing for 10: an explicit parity statement or changelog confirming all dashboard features (workflow builder, analytics, testing suite) are fully scriptable via API, and independent/hands-on confirmation of that parity.
- [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
- [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”
- [claimed-docs] “create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally”
- [claimed-docs] “Build multi-step workflows with visual workflow builder”
- [probe] “PROBE runtime (recorded 2026-09-05): `npx -y @elevenlabs/cli --version` printed `elevenlabs 1.1.0` keylessly — the official ElevenLabs CLI (…”
Docs show the platform is API-first — agent creation, extraction categories, call summaries, workflows, and telephony are all described as configurable via the API, and a CLI and MCP server both wrap that same API, implying broad parity. However, there is no explicit documentation stating full UI/API feature parity, and probes found no discoverable OpenAPI/swagger spec (404s), leaving some doubt about completeness of coverage for every UI action. Missing for 10: explicit UI-API parity statement, published OpenAPI/swagger spec, independent confirmation that all UI actions have API equivalents.
- [claimed-docs] “POST /agent/{agent_id}/extraction-categories — create a category (name + model) and attach it to an agent”
- [claimed-docs] “Call summaries are an agent-level toggle: call_summary_enabled (boolean, off by default) on the agent create and update APIs”
- [claimed-docs] “Build multi-step outreach sequences that combine voice agent calls, WhatsApp messages, API calls, waits and retries, driven entirely through…”
- [claimed-docs] “The Bolna CLI (bolna) is a single Go binary that lets you manage agents, calls, phone numbers, and batches directly from your terminal”
- [claimed-docs] “Connect Claude, Cursor, Windsurf, or any MCP client to your Bolna account with the Bolna MCP server.”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.bolna.ai/openapi.json, https://docs.bolna.ai/swagger.json, https://docs.bolna.ai/api/op…”
- [probe] “official CLI documented at https://www.bolna.ai/docs/cli/introduction”
ai-native userExport all of my data in open formats and leave
weight 3 · round to BolnaElevenLabs Agentsnone0/10No evidence describes a data export feature, open-format data dumps, or account deletion/portability tooling; only conversation retention settings are documented, not export mechanisms. Missing for 10: any documented export API/CLI command, supported open export formats (e.g., JSON/CSV), and confirmation of full data portability/account closure process.
Bolna's API lets you fetch call transcripts (bolna-docs-14) and the platform can be self-hosted via Docker/Kubernetes (bolna-docs-9), with an open-source core framework (bolna-gh-1), which together provide some data portability, but there is no documented bulk 'export all data' feature or guarantee of open, standard export formats covering call recordings, agent configs, analytics, etc. Missing for 10: an explicit full-data export tool/endpoint, documented open format specification for exported data, and independent confirmation that a user can fully migrate away with all their data.
- [claimed-docs] “Make your first Voice AI call with the Bolna API — verify your key, get an agent, place a call, and fetch the transcript in about five minut…”
- [claimed-docs] “Learn how to self-host Bolna Voice AI on your own servers using Docker or Kubernetes for secure, private, and scalable enterprise deployment…”
- [github] “Bolna is the end-to-end open source production ready framework for quickly building LLM based voice driven conversational applications.”
ai-native userRead the product's source under an open license
weight 2 · round to BolnaElevenLabs Agentsnone0/10ElevenLabs Agents is a closed, proprietary SaaS product; the CLI is open on npm but no evidence indicates the core Agents platform/backend source is available under an open license. No license file, GitHub repo, or open-source statement is present in the evidence pack.
The GitHub repo description explicitly calls Bolna 'the end-to-end open source production ready framework,' supporting that its source is publicly available under an open-license framing, but no specific license (e.g., MIT/Apache) is named or verified, and no independent corroboration exists. Missing for 10: explicit license identification (e.g., LICENSE file contents), confirmation the entire product (not just the framework) is open-sourced, and independent verification.
- [github] “Bolna is the end-to-end open source production ready framework for quickly building LLM based voice driven conversational applications.”
ai-native userSelf-host the core product
weight 3 · round to BolnaElevenLabs Agentsnone0/10ElevenLabs Agents is entirely a hosted SaaS product—managed via dashboard, API, CLI, and hosted MCP server, all pointing to ElevenLabs' own infrastructure; there is no evidence of an open-source release, downloadable server binary, or self-hosting option for the core voice-agent runtime. missing for 10: any open-source repo or self-hosted deployment package, docs describing running the core service on one's own infrastructure, independent confirmation of self-hosting.
- [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
- [claimed-docs] “an assistant such as Claude can create, configure, and manage the agents in your workspace through natural language, with nothing to install…”
- [claimed-docs] “create, configure, and manage the agents in your workspace through natural language, with nothing to install or run locally”
Bolna is explicitly open source (GitHub repo) and documentation provides dedicated on-premise instructions for self-hosting via Docker or Kubernetes for enterprise deployments, directly satisfying the self-host story. Missing for 10: independent/hands-on confirmation of a successful self-hosted deployment and more detail on feature parity between self-hosted and cloud versions.
- [github] “Bolna is the end-to-end open source production ready framework for quickly building LLM based voice driven conversational applications.”
- [claimed-docs] “Learn how to self-host Bolna Voice AI on your own servers using Docker or Kubernetes for secure, private, and scalable enterprise deployment…”
Pricing plans — plan structure and value — what each tier costs and what it unlocksPricing plans
Plan structure and value — what each tier costs and what it unlocks
Pricing
founderSee published per-minute or usage pricing and estimate cost per call before committing
weight 2 · round to BolnaElevenLabs Agentsnone0/10No evidence item in the pack references pricing, per-minute rates, usage-based billing, or cost calculators for ElevenLabs Agents; all citations are product/docs feature descriptions. Since founders evaluating a voice-agent SaaS could reasonably expect published pricing, the axis applies but is entirely unaddressed in this evidence pack.
There is a dedicated call-pricing doc page and mentions that BYOK/connected providers aren't charged (bolna-docs-19, bolna-docs-10), indicating some usage-based pricing structure exists, but no evidence pack item actually quotes a per-minute rate, tiered pricing table, or a cost calculator a founder could use to estimate cost per call. Missing for 10: explicit per-minute/usage rate figures, a pricing calculator or worked cost example, and independent confirmation that published rates are accurate.
- [claimed-docs] “When you bring your own keys (BYOK), Bolna does not charge for those components.”
- [claimed-docs] “We don't charge for any usage for providers that you have connected to Bolna.”
Privacy posture — data-handling and privacy storiesPrivacy posture
Data-handling and privacy stories
ai-native userChoose where my data is stored (region/residency)
weight 2 · round to BolnaElevenLabs Agentsnone0/10Evidence covers data retention duration settings, HIPAA/BAA eligibility, and infrastructure details, but nothing addresses selectable data region/residency options for storage location. missing for 10: any mention of region selection, data residency controls, or geographic storage options.
- [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings.”
- [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings”
- [claimed-docs] “ElevenLabs Agents is one of ElevenLabs' HIPAA-eligible services, and we offer Business Associate Agreements (BAAs) to eligible customers”
Docs explicitly mention Indian data residency as an available option, and self-hosting via Docker/Kubernetes lets enterprises control where data lives entirely. However, there's no broader multi-region selection (e.g., EU, US) documented beyond India, and no detail on how residency is configured or enforced. Missing for 10: documented multi-region choice beyond India, configuration/UI for selecting region, and independent confirmation of residency enforcement.
- [claimed-docs] “Indian data residency is available for deployments where data must remain in India.”
- [claimed-docs] “Learn how to self-host Bolna Voice AI on your own servers using Docker or Kubernetes for secure, private, and scalable enterprise deployment…”
ai-native userPrevent my data from being used to train AI models
weight 3 · round drawnElevenLabs Agentsnone0/10The evidence pack covers data retention settings (how long transcripts/audio are stored) and HIPAA/BAA compliance, but nowhere mentions an explicit opt-out or policy regarding use of customer data/conversations for AI model training. missing for 10: explicit training-data opt-out policy or setting, terms-of-service language on model training use, any statement distinguishing enterprise vs free-tier data usage for training.
- [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings.”
- [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings”
- [claimed-docs] “ElevenLabs Agents is one of ElevenLabs' HIPAA-eligible services, and we offer Business Associate Agreements (BAAs) to eligible customers”
Bolnanone0/10No evidence pack item addresses a training-data opt-out or policy on AI model training use; security docs cover data residency and IP whitelisting only, not training-data usage controls. missing for 10: any explicit statement on AI training data usage/opt-out policy, data processing agreement terms, or contractual guarantee against training use.
ai-native userControl data retention and deletion
weight 2 · round to ElevenLabs AgentsDocs explicitly document configurable retention settings for conversation transcripts and audio recordings, giving users control over how long data is stored. However, evidence does not detail a deletion mechanism (e.g., data subject deletion requests, API/CLI commands to purge stored data) beyond retention duration settings. Missing for 10: explicit deletion/export controls, granular per-conversation deletion API, and independent verification of retention enforcement.
- [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings.”
- [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings”
Bolnanone0/10No evidence of data retention controls, deletion APIs, or user-facing privacy/data-lifecycle settings; only tangential mentions of security (IP whitelisting, India data residency) which do not address retention/deletion. Self-hosting option exists but that's deployment control, not a retention/deletion feature.
- [claimed-docs] “Indian data residency is available for deployments where data must remain in India.”
- [claimed-docs] “Whitelist all three IPs on your server or firewall”
- [claimed-docs] “Learn how to self-host Bolna Voice AI on your own servers using Docker or Kubernetes for secure, private, and scalable enterprise deployment…”
ai-native userOpt out of telemetry and usage tracking
weight 2 · round drawnElevenLabs Agentsnone0/10The evidence pack covers conversation transcript/audio retention settings (docs-26, docs-52) but nothing about opting out of product telemetry or usage tracking (e.g., CLI analytics, dashboard usage metrics) for the AI-native user. Retention configuration is about customer conversation data, not the vendor's own telemetry collection, so this axis has no supporting evidence.
Bolnanone0/10No evidence pack item addresses telemetry, analytics opt-out, or usage-tracking controls for the platform itself; existing security/data-residency docs don't cover this. Missing for 10: any documentation of telemetry collection or an opt-out/disable mechanism, privacy policy statement on usage tracking, self-host confirmation of no phone-home behavior.
Telephony — stories about telephony in this arenaTelephony
Stories about telephony in this arena
Call control
developerEscalate a live call to a human with warm or blind transfer, passing context along
weight 2 · round drawnElevenLabs Agentsnone0/10The evidence pack covers telephony features like SIP trunking, tools, webhooks, and MCP integrations, but nowhere documents call transfer (warm or blind) to a human agent or passing conversation context during handoff. Missing for 10: any mention of transfer_to_number/transfer_to_agent tool types, warm/blind transfer configuration, or context-passing during escalation.
developerMy agent can send DTMF keypresses, navigate IVR menus, and detect or leave voicemail
weight 1 · round drawnElevenLabs Agentsnone0/10The evidence pack confirms ElevenLabs Agents supports telephony integration via SIP trunking and phone numbers, so the telephony axis clearly applies, but nothing in the docs mentions DTMF tone sending, IVR menu navigation, or voicemail detection/leaving capabilities.
- [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”
- [claimed-docs] “Route calls to AI agents without changing your existing phone infrastructure”
Campaigns
founderRun batch outbound call campaigns with scheduling and throughput controls
weight 2 · round to BolnaElevenLabs Agentsnone0/10The evidence pack documents phone-number/SIP trunking integration for routing calls to agents, but contains no mention of batch outbound calling, campaign scheduling, or throughput/rate controls for outbound call volume. Since telephony campaign management is a fair capability to expect from a voice-agent telephony product, absence of evidence here is 'none' rather than 'na'.
- [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”
- [claimed-docs] “Route calls to AI agents without changing your existing phone infrastructure”
The CLI explicitly manages 'batches' of calls alongside agents/phone numbers, and marketing copy claims support for 'thousands of inbound and outbound calls every minute,' plus a workflows API for multi-step outreach with waits/retries — together implying batch campaign and throughput capability. However, there's no explicit documentation of scheduling controls (e.g., time windows, campaign start/stop) or throughput/rate-limiting configuration for outbound campaigns specifically. Missing for 10: explicit scheduling/calendar controls for campaigns, documented throughput/rate-limit configuration knobs, and any hands-on/independent evidence of batch campaign execution.
- [claimed-docs] “The Bolna CLI (bolna) is a single Go binary that lets you manage agents, calls, phone numbers, and batches directly from your terminal”
- [claimed-docs] “Power thousands of inbound and outbound calls every minute with human-like, multilingual intelligence”
- [claimed-docs] “Build multi-step outreach sequences that combine voice agent calls, WhatsApp messages, API calls, waits and retries, driven entirely through…”
- [probe] “official CLI documented at https://www.bolna.ai/docs/cli/introduction”
Numbers
developerProvision phone numbers and run both inbound and outbound calls through the platform's API
weight 3 · round drawnDocs confirm phone/telephony integration via SIP trunking, letting agents route inbound/outbound calls over existing phone infrastructure, and agents can be created/configured via the API. However, evidence does not explicitly show a documented API endpoint for provisioning new phone numbers or explicitly triggering outbound calls programmatically vs. via SIP trunk setup. Missing for 10: explicit phone-number provisioning API endpoint docs, explicit outbound-call-initiation API example, and independent/hands-on confirmation of inbound+outbound call flows.
- [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”
- [claimed-docs] “Route calls to AI agents without changing your existing phone infrastructure”
- [claimed-docs] “we'll start by creating an agent via the API or the web dashboard”
Bolna's API/CLI clearly support placing outbound calls and receiving inbound calls (quickstart shows placing a call and fetching transcript via API, CLI manages calls and phone numbers, marketing claims thousands of inbound/outbound calls per minute), and BYOT lets you attach SIP trunks/numbers. However there's no explicit documented endpoint for provisioning/purchasing new phone numbers through the API itself — only CLI mention of managing 'phone numbers' and bringing your own trunk. Missing for 10: explicit phone-number provisioning API endpoint/documentation, independent confirmation of end-to-end inbound+outbound flow.
- [claimed-docs] “Make your first Voice AI call with the Bolna API — verify your key, get an agent, place a call, and fetch the transcript in about five minut…”
- [claimed-docs] “The Bolna CLI (bolna) is a single Go binary that lets you manage agents, calls, phone numbers, and batches directly from your terminal”
- [claimed-docs] “Bring Your Own Telephony (BYOT) lets you connect any standards-compliant SIP trunk to the Bolna platform.”
- [claimed-docs] “Power thousands of inbound and outbound calls every minute with human-like, multilingual intelligence”
- [probe] “official CLI documented at https://www.bolna.ai/docs/cli/introduction”
Sip
platform-engineerConnect my own carrier or PBX via SIP trunking (or import Twilio/Telnyx numbers) instead of being locked to bundled telephony
weight 2 · round to BolnaDocs explicitly describe a SIP trunking integration that lets customers connect their existing phone systems/carrier infrastructure to ElevenLabs agents without changing carriers, directly addressing the core ask of bringing your own PBX/carrier. However, the pack has no explicit mention of importing existing Twilio/Telnyx numbers or detailed carrier onboarding steps, and no independent/hands-on confirmation of a live SIP trunk setup working end-to-end. Missing for 10: explicit Twilio/Telnyx number import instructions, and third-party/hands-on validation of the SIP trunking flow.
- [claimed-docs] “This integration enables all customers to use their existing phone systems while leveraging ElevenLabs' advanced voice AI capabilities.”
- [claimed-docs] “Route calls to AI agents without changing your existing phone infrastructure”
Bolna explicitly documents 'Bring Your Own Telephony (BYOT)' allowing any standards-compliant SIP trunk to connect to the platform, plus IP whitelisting requirements for the connection, directly addressing carrier/PBX independence from bundled telephony. missing for 10: explicit step-by-step Twilio/Telnyx number import walkthrough and independent/hands-on verification of SIP trunk setup working end-to-end.
- [claimed-docs] “Bring Your Own Telephony (BYOT) lets you connect any standards-compliant SIP trunk to the Bolna platform.”
- [claimed-docs] “Whitelist all three IPs on your server or firewall”
Testing analytics — stories about testing analytics in this arenaTesting analytics
Stories about testing analytics in this arena
Analytics
ai-native userThe platform's AI reviews my calls for me — scoring quality, flagging failures, and analyzing resolution automatically
weight 2 · round to ElevenLabs AgentsDocs explicitly describe automated post-call analysis: 'Success evaluation' scores conversation quality/goal achievement, 'Sentiment analysis' flags user sentiment, and 'Data collection' extracts structured resolution details — directly matching the story's scoring/flagging/resolution-analysis ask. Missing for 10: no independent/hands-on validation of analysis accuracy, and no explicit 'failure flagging' terminology beyond success-criteria evaluation.
- [claimed-docs] “Success evaluation: Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
- [claimed-docs] “Data collection: Extract specific data points from conversations such as contact information, issue details, or any structured information”
- [claimed-docs] “Sentiment analysis: Understand user sentiment across completed conversations”
- [claimed-docs] “Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
- [claimed-docs] “Extract structured information from conversations such as contact details and business data.”
Bolna offers post-call analysis primitives — configurable extraction-categories (bolna-docs-1) and toggleable call summaries (bolna-docs-2) — which could be used to build custom scoring/flagging, but there's no documented automated QA/quality-scoring, failure-flagging, or resolution-analysis feature out of the box. Missing for 10: dedicated call-scoring/QA dashboard, automatic failure detection, resolution-analysis reporting, and independent evidence these extraction fields are used for review purposes.
- [claimed-docs] “POST /agent/{agent_id}/extraction-categories — create a category (name + model) and attach it to an agent”
- [claimed-docs] “Call summaries are an agent-level toggle: call_summary_enabled (boolean, off by default) on the agent create and update APIs”
founderSee call analytics — success rates, durations, outcomes, sentiment — in dashboards without building my own
weight 2 · round to ElevenLabs AgentsDocs confirm built-in agent-analysis features—success evaluation, sentiment analysis, data extraction, and conversation search—managed via the ElevenAgents dashboard, which covers most of the story's asked-for metrics without custom-building analytics. However, there's no explicit evidence of a dashboard showing aggregated success-rate percentages or call-duration analytics/rollups across calls, only per-conversation criteria and settings. Missing for 10: dashboard screenshots/evidence of aggregate success-rate metrics, call-duration reporting/rollups, and independent corroboration that these appear in a unified analytics view.
- [claimed-docs] “Success evaluation: Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
- [claimed-docs] “Data collection: Extract specific data points from conversations such as contact information, issue details, or any structured information”
- [claimed-docs] “Sentiment analysis: Understand user sentiment across completed conversations”
- [claimed-docs] “Searching conversations: Find messages by keyword or meaning across your conversation history.”
- [claimed-docs] “ElevenLabs Agents are managed either through the ElevenAgents dashboard, the ElevenLabs API, the Agents CLI or the hosted MCP server.”
- [claimed-docs] “Extract structured information from conversations such as contact details and business data.”
Bolnanone0/10Evidence only shows API-level features like call summaries (call_summary_enabled) and extraction categories, but there is no mention of a dashboard UI presenting success rates, durrations, outcomes, or sentiment analytics — missing for 10: any dashboard/UI screenshot or docs describing aggregated call analytics views.
- [claimed-docs] “POST /agent/{agent_id}/extraction-categories — create a category (name + model) and attach it to an agent”
- [claimed-docs] “Call summaries are an agent-level toggle: call_summary_enabled (boolean, off by default) on the agent create and update APIs”
Monitoring
platform-engineerMonitor live calls in production and get alerts when agents misbehave or error rates spike
weight 1 · round to ElevenLabs AgentsDocs show post-call analytics (sentiment analysis, success evaluation, data extraction, conversation search) and a marketing claim to 'monitor' agents, but there is no evidence of live/real-time call monitoring dashboards, error-rate metrics, or alerting/notification mechanisms for misbehaving agents. Missing for 10: real-time monitoring dashboard, error-rate/latency metrics, alerting/notification system, on-call integration (e.g., PagerDuty/Slack alerts).
- [claimed-docs] “Success evaluation: Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
- [claimed-docs] “Sentiment analysis: Understand user sentiment across completed conversations”
- [claimed-docs] “Searching conversations: Find messages by keyword or meaning across your conversation history.”
- [claimed-docs] “Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
- [claimed-docs] “Configure, deploy and monitor natural, human-sounding agents in 70+ languages with leading accuracy and ultra-low latency across voice or ch…”
Bolnanone0/10Evidence covers call summaries, extraction categories, transcripts, CLI, MCP, and SIP/telephony features, but nothing describes live-call monitoring dashboards, real-time observability, or alerting on agent misbehavior or error-rate spikes. No mention of metrics, alert thresholds, webhooks for anomaly detection, or a monitoring console for platform engineers. Missing for 10: real-time monitoring dashboard, alerting/notification system for errors or misbehavior, error-rate tracking, on-call/integration with observability tools.
Testing
developerTest agents with simulated conversations or evals before putting them on real phone calls
weight 2 · round to ElevenLabs AgentsElevenLabs documents an explicit Agent Testing feature that verifies conversational responses, tool usage, and full multi-turn outcomes before deployment, including converting real conversations into simulated test cases, plus success-evaluation criteria for assessing conversation quality/goal achievement — directly matching the pre-deployment simulated testing story. Missing for 10: no independent/hands-on report validating test accuracy or CI-integration details, and no explicit mention of automated eval scoring pipelines beyond docs description.
- [claimed-docs] “Agent testing lets you verify conversational responses, tool usage, and full multi-turn outcomes before you deploy.”
- [claimed-docs] “Transform real conversations into test cases when you find an interaction where the agent underperformed.”
- [claimed-docs] “Agent testing lets you verify conversational responses, tool usage, and full multi-turn outcomes before you deploy”
- [claimed-docs] “Success evaluation: Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
- [claimed-docs] “Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
Bolnanone0/10No evidence of any simulated conversation testing, evals, or pre-production testing framework for agents; the evidence pack only covers deployment, telephony, providers, pricing, and SDKs. Missing for 10: any mention of test/eval tooling, simulated call testing, or conversation quality scoring before live calls.
Tools function calling — stories about tools function calling in this arenaTools function calling
Stories about tools function calling in this arena
Post call
developerExtract structured data from every call — outcomes, entities, dispositions — delivered via API or webhook after the call
weight 2 · round drawnDocs show structured post-call outputs (data collection with contact/business data extraction, success-evaluation criteria, sentiment analysis) available via 'agent-analysis' features, and the platform is manageable via API. However, the pack never explicitly documents a webhook payload/callback delivering this structured analysis after a call, only webhook *tools* for in-call actions — missing for 10: explicit post-call webhook delivery mechanism/schema for outcomes and dispositions, and confirmation these analysis fields are retrievable via a documented API endpoint.
- [claimed-docs] “Success evaluation: Define custom criteria to assess conversation quality, goal achievement, and customer satisfaction”
- [claimed-docs] “Data collection: Extract specific data points from conversations such as contact information, issue details, or any structured information”
- [claimed-docs] “Sentiment analysis: Understand user sentiment across completed conversations”
- [claimed-docs] “Extract structured information from conversations such as contact details and business data”
- [claimed-docs] “Extract structured information from conversations such as contact details and business data.”
- [claimed-docs] “Tools give your assistant the ability to make external function calls to third party apps so you can get real-time information.”
- [claimed-docs] “allow an assistant to trigger authenticated actions based on the conversation, like scheduling meetings or initiating order returns”
Bolna provides extraction-categories API to attach custom entity/data extraction to agents and a call_summary_enabled toggle for summaries, plus transcript fetch via API — covering entities, summaries, and outcomes. However, there's no explicit documentation of a unified structured 'disposition' field, no clear webhook delivery mechanism for post-call structured payloads (only API fetch is shown), and no example of the combined extraction+summary+disposition JSON schema. missing for 10: webhook-based delivery of extraction results, documented disposition/outcome field, end-to-end example combining all three data types in one payload.
- [claimed-docs] “POST /agent/{agent_id}/extraction-categories — create a category (name + model) and attach it to an agent”
- [claimed-docs] “Call summaries are an agent-level toggle: call_summary_enabled (boolean, off by default) on the agent create and update APIs”
- [claimed-docs] “Make your first Voice AI call with the Bolna API — verify your key, get an agent, place a call, and fetch the transcript in about five minut…”
Tools
ai-native userMy voice agent can plug in MCP servers as tool sources so one integration grants it whole toolsets mid-call
weight 2 · round to ElevenLabs AgentsDocs explicitly describe connecting conversational agents to external MCP servers as a tool source, enabling access to data/processing capabilities via that server mid-call, directly matching the story. Missing for 10: independent hands-on confirmation of mid-call multi-tool retrieval from a connected MCP server (only vendor docs, no third-party validation) and detail on how many tools/toolsets a single MCP connection exposes.
- [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers. This enables your agents to: Access and process informa…”
- [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers.”
- [claimed-docs] “Access and process information from various data sources via the MCP server”
Bolnanone0/10Evidence only shows Bolna exposing its OWN MCP server so external clients (Claude, Cursor, Windsurf) can manage a Bolna account (bolna-docs-17, bolna-probe-3) — the reverse direction of the story, which asks whether a Bolna voice agent can consume external MCP servers as tool sources mid-call. No documentation shows Bolna agents acting as an MCP client or importing toolsets from third-party MCP servers during a call. missing for 10: any evidence of Bolna voice agents connecting to external MCP servers as tool sources, mid-call toolset loading via MCP.
- [claimed-docs] “Connect Claude, Cursor, Windsurf, or any MCP client to your Bolna account with the Bolna MCP server.”
- [probe] “official MCP server documented at https://www.bolna.ai/docs/build-with-ai/mcp”
developerMy agent can call external APIs and custom functions mid-conversation and speak the result without awkward dead air
weight 3 · round to ElevenLabs AgentsDocs clearly describe multiple tool-calling mechanisms — webhook tools for external API calls, client-side tools, sandboxed custom JS, and MCP server integration — explicitly for making external function calls mid-conversation to fetch real-time info and trigger authenticated actions. Conversation-flow settings (turn-taking, interruption handling, ultra-low latency) address smooth conversational pacing, though not explicitly framed as avoiding dead air during tool execution. Missing for 10: explicit documentation of filler/hold-speech behavior while awaiting API results, and independent/hands-on verification that tool calls don't introduce audible pauses.
- [claimed-docs] “Enable agents to call clients & APIs to perform actions”
- [claimed-docs] “Tools executed directly on the client-side application (e.g., web browser, mobile app).”
- [claimed-docs] “Tools give your assistant the ability to make external function calls to third party apps so you can get real-time information.”
- [claimed-docs] “Custom JavaScript executed in a sandboxed environment on ElevenLabs' infrastructure.”
- [claimed-docs] “ElevenLabs allows you to connect your conversational agents to external MCP servers. This enables your agents to: Access and process informa…”
- [claimed-docs] “Tools allow ElevenLabs agents to perform actions beyond generating text responses.”
- [claimed-docs] “allow an assistant to trigger authenticated actions based on the conversation, like scheduling meetings or initiating order returns”
- [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”
- [claimed-docs] “Configure, deploy and monitor natural, human-sounding agents in 70+ languages with leading accuracy and ultra-low latency across voice or ch…”
Bolnanone0/10The evidence pack covers extraction categories, call summaries, workflows, and custom LLM providers, but nothing describes an agent invoking external APIs or custom functions mid-call and voicing the result seamlessly. Missing for 10: documentation of a function-calling/tool-use API, examples of mid-call API invocation, and any mention of latency-masking/filler techniques to avoid dead air.
Transcription recording — stories about transcription recording in this arenaTranscription recording
Stories about transcription recording in this arena
Recording
platform-engineerRetrieve full call recordings and transcripts programmatically for every call
weight 2 · round drawnDocs confirm that conversation transcripts and audio recordings are stored with configurable retention, and that conversations can be searched, analyzed, and have structured data extracted (docs-26/52, docs-21/23/56), implying underlying storage of call recordings/transcripts. However, there is no explicit documentation of a specific API endpoint or method for programmatically retrieving full call recordings and transcripts on a per-call basis. Missing for 10: an explicit 'get conversation' or 'get recording/transcript' API reference, example request/response showing programmatic export of audio + transcript per call.
- [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings.”
- [claimed-docs] “Retention settings allow you to configure how long your conversational agent stores conversation transcripts and audio recordings”
- [claimed-docs] “Data collection: Extract specific data points from conversations such as contact information, issue details, or any structured information”
- [claimed-docs] “Searching conversations: Find messages by keyword or meaning across your conversation history.”
- [claimed-docs] “Extract structured information from conversations such as contact details and business data.”
Docs confirm fetching call transcripts via API (quickstart mentions 'fetch the transcript') and call summaries can be toggled, but there is no explicit documentation of a dedicated recordings-retrieval endpoint or bulk/programmatic export for all calls. missing for 10: explicit call-recording download/API endpoint, confirmation that recordings are retained and accessible for every call, and independent verification of transcript completeness/format.
- [claimed-docs] “Make your first Voice AI call with the Bolna API — verify your key, get an agent, place a call, and fetch the transcript in about five minut…”
- [claimed-docs] “Call summaries are an agent-level toggle: call_summary_enabled (boolean, off by default) on the agent create and update APIs”
- [claimed-docs] “POST /agent/{agent_id}/extraction-categories — create a category (name + model) and attach it to an agent”
Transcription
developerGet accurate real-time transcription with control over the STT provider, language models, or key terms
weight 2 · round to BolnaDocs confirm agents let developers choose or bring custom LLMs (docs-41) and mention real-time voice interaction (docs-37), but there is no evidence of selectable STT providers, transcription accuracy tuning, or 'key terms'/vocabulary boosting controls anywhere in the pack. Missing for 10: STT provider selection, transcription-specific accuracy settings, key-term/glossary support, and any independent verification of transcription accuracy claims.
- [claimed-docs] “Choose from supported LLMs or bring your own custom model”
- [claimed-docs] “Configure, deploy and monitor natural, human-sounding agents in 70+ languages with leading accuracy and ultra-low latency across voice or ch…”
- [claimed-docs] “Conversation flow settings determine how your assistant handles periods of user silence, interruptions during speech, and turn-taking behavi…”
Docs confirm STT provider choice (Deepgram, Azure, ElevenLabs) and full LLM control including custom OpenAI-compatible endpoints, plus transcript retrieval via API, giving developers real provider/model flexibility. However there is no evidence of key-term/vocabulary boosting or accuracy tuning controls, nor any real-time transcription accuracy benchmarks. Missing for 10: key-term/keyword-boost configuration, accuracy/latency benchmarks for real-time transcription, and independent validation of transcription quality.
- [claimed-docs] “Bolna supports Deepgram, Azure, ElevenLabs, and more.”
- [claimed-docs] “For custom llm simply keep provider in the llm_agent key as custom and add a openai compatible base_url”
- [claimed-docs] “Make your first Voice AI call with the Bolna API — verify your key, get an agent, place a call, and fetch the transcript in about five minut…”
Voices tts — stories about voices tts in this arenaVoices tts
Stories about voices tts in this arena
Voices
founderClone a custom brand voice and use it for my agents, with a documented consent process
weight 2 · round drawnElevenLabs Agentsnone0/10The evidence pack describes selecting from 5k+ pre-built voices and voice customization (speed, multi-voice switching) but contains no mention of a voice-cloning feature for creating a custom brand voice, nor any documented consent process for cloning (e.g., voice verification, spoken consent statement) required by ElevenLabs' broader platform. Missing for 10: any docs on voice cloning/Instant or Professional Voice Cloning for Agents, and any consent-verification workflow.
- [claimed-docs] “Select from 5k+ voices across 31 languages with customization options”
- [claimed-docs] “Enable your agent to switch between different voices for multi-character conversations, storytelling, and language tutoring.”
- [claimed-docs] “You can customize various aspects of your AI agent's voice to create a more natural and engaging conversation experience. This includes cont…”
Bolnanone0/10Evidence shows Bolna lets founders choose among third-party voices/accents/styles (bolna-docs-8) and integrates providers like ElevenLabs (bolna-docs-6), but there is no mention of a voice-cloning feature for a custom brand voice, nor any documented consent process for cloning. Missing for 10: voice cloning capability, consent/verification workflow documentation, any policy on brand voice usage.
- [claimed-docs] “You can choose different voices, accents, and speaking styles to match your brand.”
- [claimed-docs] “Bolna supports Deepgram, Azure, ElevenLabs, and more.”
developerChoose from a broad voice library or plug in multiple TTS providers to get the voice I want
weight 2 · round to BolnaDocs confirm a broad in-house voice library (5k+ voices across 31 languages, docs-6) with customization like speed, pronunciation, and multi-voice switching (docs-17/32/48), satisfying the first half of the story. However, there is no evidence of plugging in third-party/alternative TTS providers — the docs only mention bring-your-own LLM (docs-41), not bring-your-own TTS engine. Missing for 10: any documented support for external/third-party TTS provider integration, evidence of provider-agnostic voice pipeline, independent confirmation of using non-ElevenLabs voices.
- [claimed-docs] “Select from 5k+ voices across 31 languages with customization options”
- [claimed-docs] “Enable your agent to switch between different voices for multi-character conversations, storytelling, and language tutoring.”
- [claimed-docs] “You can customize various aspects of your AI agent's voice to create a more natural and engaging conversation experience. This includes cont…”
- [claimed-docs] “Enable your agent to switch between different voices for multi-character conversations, storytelling, and language tutoring”
- [claimed-docs] “Choose from supported LLMs or bring your own custom model”
Docs confirm multiple TTS providers (Deepgram, Azure, ElevenLabs, and more) plus voice/accent/style customization, satisfying the 'plug in providers' part of the story. However, there's no evidence of a broad pre-built voice library or catalog to browse/select from, only provider integration. Missing for 10: evidence of a large built-in voice library/catalog, provider-count breadth beyond three named examples, independent/hands-on confirmation of voice quality or selection UX.
- [claimed-docs] “Bolna supports Deepgram, Azure, ElevenLabs, and more.”
- [claimed-docs] “You can choose different voices, accents, and speaking styles to match your brand.”