Vapi vs Bland
Bland wins · 11–20 (30 drawn)
Agent building — building agents — abstractions, tool wiring, control flowAgent building
Building agents — abstractions, tool wiring, control flow
Agent ops
ai-native userMy coding agent can provision a complete voice agent end to end — create the agent, attach a number, and place a call — through the API, CLI, or MCP without touching the dashboard
weight 3 · round drawnVapi documents API/SDK-based assistant creation, phone number attachment, and call placement (vapi-docs-1, vapi-docs-18, vapi-docs-19), a CLI for managing assistants, phone numbers, and calls entirely from the terminal (vapi-docs-2, vapi-docs-31, vapi-probe-rt-1 confirming keyless install/version), and an official MCP server exposing these same operations to any MCP-compatible agent, verified live and auth-gated in a runtime probe (vapi-docs-33, vapi-probe-rt-2). Together these three surfaces (API, CLI, MCP) cover the full agent-provision-number-call workflow without dashboard use. missing for 10: no single end-to-end hands-on trace showing one agent chaining create→attach→call purely via CLI/MCP/API in one session, and no independent (non-vendor) confirmation of the full workflow succeeding.
- [claimed-docs] “const assistant = await vapi.assistant”
- [claimed-docs] “Manage assistants, phone numbers, and calls from your terminal”
- [claimed-docs] “Create a voice assistant, connect it to a phone number, and make your first calls.”
- [claimed-docs] “In under 5 minutes, you'll create a voice assistant and start talking to it over the phone.”
- [claimed-docs] “Build, test, and deploy voice AI applications without leaving your development environment.”
- [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
- [probe] “official MCP server documented at https://docs.vapi.ai/sdk/mcp-server”
- [probe] “official CLI documented at https://docs.vapi.ai/cli”
- [probe] “PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bas…”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”
Bland's docs show all three provisioning steps available programmatically: pathway/agent creation via API (bland-docs-1, bland-docs-18), phone number acquisition/porting/Twilio and SIP attachment (bland-docs-3, bland-docs-16, bland-docs-17), and call placement via API or batch calls (bland-docs-6, bland-docs-23). Both the CLI ('make calls, build and test pathways, configure phone numbers' — bland-docs-12) and the MCP server ('place and inspect calls, build and validate pathways, manage agents' — bland-docs-11/20/27) explicitly cover the full create-agent/attach-number/place-call lifecycle without the dashboard, and runtime probes confirm both the CLI and hosted MCP endpoint are live and functional (bland-probe-rt-1, bland-probe-rt-2). Missing for 10: a single consolidated end-to-end tutorial/example walking through create→attach→call in one flow, and independent (non-vendor) confirmation of the full pipeline working end-to-end.
- [claimed-docs] “See the Pathways API reference to create and manage pathways programmatically.”
- [claimed-docs] “Users can connect their own Twilio account to Bland, easily bring over your existing phone numbers to use within the platform.”
- [claimed-docs] “Batch calls let you initiate high-volume call campaigns by uploading a list of recipients via CSV.”
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
- [claimed-docs] “Manage your entire Bland account from the terminal: make calls, build and test pathways, configure phone numbers, and more.”
- [claimed-docs] “Number porting to bring existing numbers to Bland”
- [claimed-docs] “Bland supports inbound and outbound SIP. Point calls from your carrier or PBX at Bland to be answered by an agent, or have Bland place calls…”
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
- [probe] “PROBE runtime (recorded 2026-09-05): `npx -y bland-cli --version` printed `0.6.2` keylessly — the official npm CLI (bland-cli, with dev term…”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP endpoint https://api.bland.ai/v1/mcp returned HTTP 4…”
ai-native userThe platform's own AI helps me author agents — generating or improving prompts, flows, and test cases from a description
weight 1 · round to BlandVapinone0/10Evidence shows Vapi provides CLI, MCP server, tool/webhook infrastructure, and testing frameworks (Evals, Voice Test Suites), plus a Customer Support template, but nothing indicates the platform itself uses AI to generate or improve prompts, conversation flows, or test cases from a natural-language description — testing tools require manually defined mock conversations/scripts rather than AI-authored ones.
- [claimed-docs] “Voice Test Suites enable you to test your AI voice agents through simulated phone conversations.”
- [claimed-docs] “Evals is Vapi's AI agent testing framework that enables you to systematically test assistants and squads using mock conversations with autom…”
- [claimed-docs] “Our AI tester calls your voice agent and follows a script that simulates real customer behavior.”
- [claimed-docs] “you'll create mock conversations, define expected behaviors, and validate your agents work correctly before production”
- [claimed-docs] “Select the down arrow next to Create Assistant, then choose Customer Support.”
Bland's MCP server lets external AI coding agents 'build and validate pathways ... through natural language' (bland-docs-11/20/27), and the testbed/evals tools support iterating on and grading prompts, which loosely supports AI-assisted authoring. However there is no evidence of a built-in, first-party generative feature where Bland's own platform AI drafts a full pathway/flow/test-cases from a plain-language description inside the product itself — the closest capability requires an external AI agent connecting via MCP. Missing for 10: a native 'describe your agent, we generate the pathway/prompts/tests' feature, in-product prompt-improvement AI, and independent/hands-on confirmation of AI-generated flows.
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
- [claimed-docs] “The testbed lets you take any call ... isolate a specific node interaction, edit the prompt, and run it multiple times to see how the output…”
- [claimed-docs] “The testbed lets you take any call — whether it's a historical call from call logs or a test chat from the pathway editor — isolate a specif…”
- [claimed-docs] “Evals let you measure the quality of your calls with LLM judges.”
- [claimed-docs] “Evals let you measure the quality of your calls with LLM judges. You define eval agents, each of which grades one dimension of a call”
Build
developerBuild a working phone voice agent — prompt, voice, and phone number — and take my first live call within an hour
weight 3 · round to VapiDocs explicitly walk through creating a voice assistant, connecting a phone number, and making a first call in under 5 minutes (vapi-docs-18, vapi-docs-19), with voice selection via the Voice Library (vapi-docs-8, vapi-docs-24) and dashboard-driven assistant creation with templates (vapi-docs-16); community evidence corroborates a working live-call demo (vapi-comm-1). Missing for 10: independent third-party benchmarking of the full 'within an hour' timing claim beyond vendor docs and a single anecdotal community comment.
- [claimed-docs] “Create a voice assistant, connect it to a phone number, and make your first calls.”
- [claimed-docs] “In under 5 minutes, you'll create a voice assistant and start talking to it over the phone.”
- [claimed-docs] “Select the down arrow next to Create Assistant, then choose Customer Support.”
- [claimed-docs] “The Voice Library in the Vapi Dashboard lists every voice available to your organization. Browse and preview voices there”
- [claimed-docs] “Browse and preview voices there, then copy a voice's ID to use on an assistant.”
- [community] “Called the demo number, sounds smooth! Good luck.”
Docs cover the whole first-call workflow: creating a pathway/prompt (bland-docs-18, bland-docs-26, bland-docs-30), voice cloning/selection (bland-docs-10), phone number setup via own Twilio or new inbound numbers (bland-docs-3, bland-docs-24), and dispatching outbound calls (bland-docs-23), plus a CLI and MCP server confirmed live at runtime (bland-probe-rt-1, bland-probe-rt-2) that let a developer configure and place calls quickly. Missing for 10: no independent hands-on account of a developer actually completing a first call within an hour, and one community comment notes cost concerns rather than time-to-first-call, so onboarding speed is only documented, not externally verified.
- [claimed-docs] “Conversational pathways are our new way of prompting Bland that has led to major breakthroughs in realism. Give your agent instructions on h…”
- [claimed-docs] “Conversational pathways are our new way of prompting Bland that has led to major breakthroughs in realism.”
- [claimed-docs] “Give your agent instructions on how it should respond at specific points of the conversation. Choose between prompting or fixed sentences. E…”
- [claimed-docs] “A clone needs one clean sample of about ten seconds. Quality of the sample sets the ceiling on quality of the voice”
- [claimed-docs] “Users can connect their own Twilio account to Bland, easily bring over your existing phone numbers to use within the platform.”
- [claimed-docs] “Create inbound phone numbers for customer support, etc.”
- [claimed-docs] “Dispatch AI phone calls to call customers, leads, and to streamline operations.”
- [claimed-docs] “The API integration lets you connect your AI agent to any HTTP endpoint.”
- [probe] “PROBE runtime (recorded 2026-09-05): `npx -y bland-cli --version` printed `0.6.2` keylessly — the official npm CLI (bland-cli, with dev term…”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP endpoint https://api.bland.ai/v1/mcp returned HTTP 4…”
developerRun conversations in multiple languages, including detecting and switching language mid-call
weight 2 · round drawnVapinone0/10The evidence pack covers CLI, MCP server, tools, voices, compliance, and testing, but contains no mention of multi-language support, language detection, or mid-call language switching capabilities. Absence of evidence for this applicable axis (multilingual voice agents are a fair capability to expect from a voice AI platform) means it must be judged 'none'. missing for 10: any mention of multi-language conversation support, language auto-detection, or mid-call language switching features.
Blandnone0/10The evidence pack contains no mention of multi-language support, language detection, or mid-call language switching anywhere in Bland's docs; only pathways, TTS voice cloning, and infrastructure features are documented. Missing for 10: any documentation of multilingual conversation support, automatic language detection, or dynamic language switching mid-call.
founderDesign multi-step conversation flows in a visual builder with branching, states, and handoffs without writing code
weight 2 · round to BlandVapinone0/10Evidence covers Vapi's assistant configuration, tools, CLI, MCP server, voices, and testing, but there is no mention of a visual flow builder with branching, states, or handoffs — Vapi appears configuration/code-driven (assistant config, function tools, webhooks) rather than a no-code visual conversation designer.
Bland's 'Conversational Pathways' feature is explicitly a node-based flow builder where founders give instructions at specific conversation points, test individual node interactions in a pathway editor/testbed, and publish drafts separately from production—matching branching/states/handoffs without code (bland-docs-18, 26, 30, 28, 19). Missing for 10: explicit confirmation of a drag-and-drop visual canvas UI (docs describe 'nodes' and 'pathway editor' but no screenshot/UI walkthrough) and independent hands-on corroboration beyond vendor docs.
- [claimed-docs] “Conversational pathways are our new way of prompting Bland that has led to major breakthroughs in realism. Give your agent instructions on h…”
- [claimed-docs] “Conversational pathways are our new way of prompting Bland that has led to major breakthroughs in realism.”
- [claimed-docs] “Give your agent instructions on how it should respond at specific points of the conversation. Choose between prompting or fixed sentences. E…”
- [claimed-docs] “The testbed lets you take any call — whether it's a historical call from call logs or a test chat from the pathway editor — isolate a specif…”
- [claimed-docs] “When you edit a pathway, you are working on a draft. Live calls are not affected. They keep using the published production version.”
- [claimed-docs] “See the Pathways API reference to create and manage pathways programmatically.”
Personalization
developerInject dynamic variables and per-caller context at call time so each conversation is personalized
weight 2 · round to BlandVapinone0/10The evidence pack covers assistant creation, tools/webhooks, CLI, MCP server, voices, and testing, but contains no documentation of variable injection, assistantOverrides, or per-caller context personalization at call time. Missing for 10: docs on dynamic variable substitution (e.g., {{variableName}} templating), call-time overrides/metadata injection, and any example showing per-caller personalization.
Evidence shows personalization mechanisms exist—Memory for per-caller context (bland-docs-9), batch calls that likely carry per-recipient data (bland-docs-6), and pathway webhooks/API integrations that could fetch live data (bland-docs-4, bland-docs-30)—but there is no explicit documentation of a 'dynamic variables' injection API or call-time variable substitution mechanism. missing for 10: explicit dynamic-variable/request_data injection documentation, examples of per-call variable interpolation into prompts, independent confirmation of personalization working in practice.
- [claimed-docs] “Memory lets Bland agents remember people across conversations, including calls, SMS, and other channels, so each interaction feels continuou…”
- [claimed-docs] “Batch calls let you initiate high-volume call campaigns by uploading a list of recipients via CSV.”
- [claimed-docs] “The API integration lets you connect your AI agent to any HTTP endpoint.”
- [claimed-docs] “Give your agent instructions on how it should respond at specific points of the conversation. Choose between prompting or fixed sentences. E…”
developerGround the agent on my documents with a built-in knowledge base or RAG so it answers from my content
weight 2 · round drawnVapinone0/10Vapi is a voice-agent platform focused on assistants, phone numbers, custom webhook tools, and telephony/testing; the evidence pack shows no built-in knowledge base or RAG feature for grounding assistants on uploaded documents. Custom tools/webhooks could be used to build a workaround, but no document ingestion or retrieval capability is documented.
- [claimed-docs] “Create your own webhook-based tools to extend assistant capabilities”
- [claimed-docs] “This guide shows you how to create custom tools, including Function Tools, for your Vapi assistants.”
- [claimed-docs] “Custom tools that you create... interact with your systems via webhooks”
Blandnone0/10Bland is a voice AI/phone agent platform; evidence covers pathways, memory, tools, MCP, CLI, etc., but nowhere mentions a document knowledge base or RAG capability for grounding agent responses in uploaded content. Since a voice/conversational agent platform could plausibly ship this (many competitors do), absence of evidence makes this 'none' rather than 'na'.
Agenticness — how well agents can access and operate the productAgenticness
How well agents can access and operate the product
Agent access
ai-native userPoint an agent at llms.txt or agent-oriented docs
weight 2 · round drawnVapi confirms an llms.txt file exists and is served at docs.vapi.ai/llms.txt (HTTP 200), explicitly instructing agents on how to fetch clean Markdown per page, which directly satisfies the story. missing for 10: no independent/community corroboration of agents actually consuming llms.txt in practice, only first-party probe evidence.
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.vapi.ai/llms.txt # Vapi ## Instructions for AI Agents - For clean Markdown of any page, append `.…”
Bland has a confirmed live llms.txt at docs.bland.ai/llms.txt (HTTP 200) providing agent-oriented documentation, plus an official MCP server and CLI explicitly designed for AI coding agents to interact with the platform via natural language. missing for 10: independent third-party confirmation that agents actually consume llms.txt successfully in practice, and broader agent-oriented docs beyond the single llms.txt file.
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.bland.ai/llms.txt # Bland Documentation Bland is an enterprise voice AI platform for high-volume, …”
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
- [probe] “official MCP server documented at https://docs.bland.ai/integrations/mcp/overview”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP endpoint https://api.bland.ai/v1/mcp returned HTTP 4…”
ai-native userRun the product headlessly / in CI for automation
weight 2 · round to BlandVapi ships an official CLI (vapi-docs-2/31/32, confirmed working keylessly in vapi-probe-rt-1) that can manage assistants, calls, and forward webhooks, which supports scripted/CI-style usage, and SDKs/REST APIs imply headless programmatic calls. However there is no explicit CI/automation documentation, no examples of running in a pipeline, and the MCP server requires bearer-key auth (vapi-probe-rt-2) which is unaddressed for CI contexts. missing for 10: explicit CI/automation examples or docs, non-interactive auth/service-account flow for CI, evidence of headless voice-testing/evals running in a pipeline.
- [claimed-docs] “Manage assistants, phone numbers, and calls from your terminal”
- [claimed-docs] “Build, test, and deploy voice AI applications without leaving your development environment.”
- [claimed-docs] “The CLI auto-detects your tech stack and sets up everything you need.”
- [probe] “PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bas…”
- [claimed-docs] “Evals is Vapi's AI agent testing framework that enables you to systematically test assistants and squads using mock conversations with autom…”
- [claimed-docs] “you'll create mock conversations, define expected behaviors, and validate your agents work correctly before production”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”
Bland offers a full programmatic/REST API for pathways, calls, batch calls, webhooks, and evals, plus an official CLI (verified runtime installable keylessly via npx) for terminal-based automation, enabling headless/CI usage. missing for 10: no explicit CI/CD pipeline example (e.g., GitHub Actions) or independent case study confirming CI usage beyond docs and CLI probe.
- [claimed-docs] “See the Pathways API reference to create and manage pathways programmatically.”
- [claimed-docs] “Batch calls let you initiate high-volume call campaigns by uploading a list of recipients via CSV.”
- [claimed-docs] “Manage your entire Bland account from the terminal: make calls, build and test pathways, configure phone numbers, and more.”
- [probe] “official CLI documented at https://docs.bland.ai/sdks/cli”
- [probe] “PROBE runtime (recorded 2026-09-05): `npx -y bland-cli --version` printed `0.6.2` keylessly — the official npm CLI (bland-cli, with dev term…”
- [claimed-docs] “Evals let you measure the quality of your calls with LLM judges.”
ai-native userPlug MCP servers into this product so it can use their tools
weight 3 · round drawnVapinone0/10The evidence shows Vapi exposes its own APIs as an MCP server (so external MCP clients like Claude Desktop can control Vapi), but this is the opposite direction from the story — there's no evidence that Vapi assistants can consume/plug-in external MCP servers to use their tools within a conversation. Vapi's tool docs describe webhook-based custom tools and OpenAI-style function calling only, with no mention of MCP-server ingestion.
- [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
- [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
- [claimed-docs] “Create your own webhook-based tools to extend assistant capabilities”
- [claimed-docs] “Custom tools that you create... interact with your systems via webhooks”
- [probe] “official MCP server documented at https://docs.vapi.ai/sdk/mcp-server”
Blandnone0/10Bland's evidence only documents it exposing an outbound MCP server so that external AI coding agents can call Bland's own tools (docs-11, docs-20, docs-27, probe-rt-2) — the reverse direction of this story. There is no evidence that Bland itself can consume/plug in third-party MCP servers as a client; its tool integration story is limited to custom HTTP API endpoints and webhooks (bland-docs-4, bland-docs-25).
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
- [claimed-docs] “The API integration lets you connect your AI agent to any HTTP endpoint.”
- [claimed-docs] “Connect external APIs and take live actions during phone calls.”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP endpoint https://api.bland.ai/v1/mcp returned HTTP 4…”
ai-native userConnect an agent via an official MCP server
weight 3 · round to VapiVapi ships an official hosted MCP server (mcp.vapi.ai) exposing its assistant/phone/call APIs as MCP tools for any MCP-compatible agent (Claude Desktop, agent frameworks), documented in first-party docs and confirmed live via a runtime probe (401 bearer-key gate exactly as documented). Missing for 10: independent/community hands-on testimonials specifically about using the MCP server (community evidence only covers general demo/founder trivia, not MCP usage).
- [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
- [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
- [probe] “official MCP server documented at https://docs.vapi.ai/sdk/mcp-server”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”
Bland ships an official MCP server (docs and runtime probe confirm it's live and gated by API key) that lets AI coding agents place/inspect calls, build pathways, manage agents, query analytics, run evals, and search docs — directly fulfilling the story. Missing for 10: independent third-party hands-on review of the MCP server beyond vendor docs/probe.
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
- [probe] “official MCP server documented at https://docs.bland.ai/integrations/mcp/overview”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP endpoint https://api.bland.ai/v1/mcp returned HTTP 4…”
ai-native userUse an official CLI
weight 2 · round drawnVapi ships a documented official CLI for managing assistants, phone numbers, and calls from the terminal, with webhook forwarding, org/environment switching, and tech-stack auto-detection, and this was independently verified in a runtime probe (successful install and `--version` output). missing for 10: no independent third-party review or community discussion of the CLI's day-to-day usage beyond the vendor docs and single install probe.
- [claimed-docs] “Manage assistants, phone numbers, and calls from your terminal”
- [claimed-docs] “vapi listen --forward-to localhost:3000/webhook”
- [claimed-docs] “Switch between organizations and environments seamlessly”
- [claimed-docs] “Forward webhooks to your local server for debugging”
- [claimed-docs] “Build, test, and deploy voice AI applications without leaving your development environment.”
- [claimed-docs] “The CLI auto-detects your tech stack and sets up everything you need.”
- [probe] “official CLI documented at https://docs.vapi.ai/cli”
- [probe] “PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bas…”
Bland ships an official CLI (bland-cli) documented to manage the entire account from the terminal, and a runtime probe confirms it installs and runs via npx keylessly. missing for 10: independent third-party review/usage reports of the CLI beyond the official docs and one probe run.
- [claimed-docs] “Manage your entire Bland account from the terminal: make calls, build and test pathways, configure phone numbers, and more.”
- [probe] “official CLI documented at https://docs.bland.ai/sdks/cli”
- [probe] “PROBE runtime (recorded 2026-09-05): `npx -y bland-cli --version` printed `0.6.2` keylessly — the official npm CLI (bland-cli, with dev term…”
ai-native userDrive the product through a documented public API
weight 3 · round drawnVapi ships a documented public API/SDK (assistant management, calls, tools), a CLI for terminal-driven workflows, and a hosted MCP server exposing the API as tools, all confirmed by runtime probes (CLI installs and runs, MCP endpoint live and auth-gated as documented). This directly satisfies programmatic/AI-native control via a documented public interface. Missing for 10: no independent third-party developer report deeply exercising the API beyond docs/probes.
- [claimed-docs] “const assistant = await vapi.assistant”
- [claimed-docs] “Manage assistants, phone numbers, and calls from your terminal”
- [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
- [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.vapi.ai/llms.txt # Vapi ## Instructions for AI Agents - For clean Markdown of any page, append `.…”
- [probe] “official MCP server documented at https://docs.vapi.ai/sdk/mcp-server”
- [probe] “official CLI documented at https://docs.vapi.ai/cli”
- [probe] “PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bas…”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”
Bland documents a comprehensive public API (pathways, calls, tools, webhooks, evals, batch calls) plus SDKs, CLI, and an official MCP server, and runtime probes confirm the CLI installs and the hosted MCP endpoint is live and gated as documented, showing agentic programmatic control. Missing for 10: a formally published OpenAPI/Swagger spec (probe found 404s at standard OpenAPI paths), so machine-readable spec discoverability is unconfirmed.
- [claimed-docs] “See the Pathways API reference to create and manage pathways programmatically.”
- [claimed-docs] “The API integration lets you connect your AI agent to any HTTP endpoint.”
- [claimed-docs] “Batch calls let you initiate high-volume call campaigns by uploading a list of recipients via CSV.”
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
- [claimed-docs] “Manage your entire Bland account from the terminal: make calls, build and test pathways, configure phone numbers, and more.”
- [probe] “official MCP server documented at https://docs.bland.ai/integrations/mcp/overview”
- [probe] “official CLI documented at https://docs.bland.ai/sdks/cli”
- [probe] “PROBE runtime (recorded 2026-09-05): `npx -y bland-cli --version` printed `0.6.2` keylessly — the official npm CLI (bland-cli, with dev term…”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP endpoint https://api.bland.ai/v1/mcp returned HTTP 4…”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.bland.ai/openapi.json, https://docs.bland.ai/swagger.json, https://docs.bland.ai/api/op…”
ai-native userIssue scoped/least-privilege API credentials for an agent
weight 2 · round to VapiVapi's enterprise docs mention Role Based Access Control (RBAC) and SSO, implying some access-scoping capability at the org level, and its MCP server/API require bearer API keys — but there is no documented mechanism for issuing per-agent, least-privilege scoped API keys or tokens tailored to a specific agent's permissions. missing for 10: explicit scoped API key creation/management UI or API, documentation of key-level permission granularity, and independent confirmation that RBAC restricts agent credentials rather than just human dashboard users.
- [claimed-docs] “Single Sign On (SSO) supported for Okta, Azure AD, SAML, and OIDC * Role Based Access Control (RBAC)”
- [claimed-docs] “Single Sign On (SSO) supported for Okta, Azure AD, SAML, and OIDC”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”
Blandnone0/10Evidence shows generic API-key auth and JWT-based webhook verification (bland-docs-15), but nothing about issuing scoped or least-privilege credentials specific to an agent's permissions (e.g., role-based API keys, scoped tokens limiting call/pathway/account access). Missing for 10: documentation of scoped API key creation, permission levels, or per-agent credential restriction.
- [claimed-docs] “JWT signatures eliminate these risks through asymmetric cryptography - you verify requests using our public JWKS endpoint without storing an…”
ai-native userBuild against official SDKs
weight 2 · round to VapiVapi documents an official JS/TS SDK usage pattern, official CLI (probe-confirmed installed and runnable), and official MCP server (probe-confirmed live and auth-gated), all clearly aimed at AI-native/agentic developer workflows including IDE assistant integration (Cursor/Windsurf/VSCode). missing for 10: independent third-party corroboration of SDK code quality and coverage across multiple languages beyond the docs snippets.
- [claimed-docs] “const assistant = await vapi.assistant”
- [claimed-docs] “Manage assistants, phone numbers, and calls from your terminal”
- [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
- [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
- [claimed-docs] “Your IDE's AI assistant (Cursor, Windsurf, VSCode) gains complete, accurate knowledge of Vapi's APIs and best practices. No more hallucinate…”
- [probe] “official MCP server documented at https://docs.vapi.ai/sdk/mcp-server”
- [probe] “official CLI documented at https://docs.vapi.ai/cli”
- [probe] “PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bas…”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”
Bland documents a Web Agent SDK for embedding voice agents (React/Vanilla JS/Node), a CLI, and a REST API used throughout tutorials, giving AI-native developers concrete building blocks; runtime probes confirm the CLI installs and runs. However, no dedicated 'official SDK' page for server-side languages (Python/Node backend SDK) is evidenced, and openapi/swagger spec endpoints all 404, suggesting the API reference isn't machine-consumable in a standard SDK-generation format. Missing for 10: a clearly documented multi-language backend SDK (Python/Node) beyond the browser widget SDK, and a working OpenAPI spec for auto-generating clients.
- [claimed-docs] “Embed a Bland voice agent into any web application (React, Vanilla JS, or Node). The SDK handles secure authentication between your server a…”
- [claimed-docs] “Manage your entire Bland account from the terminal: make calls, build and test pathways, configure phone numbers, and more.”
- [probe] “official CLI documented at https://docs.bland.ai/sdks/cli”
- [probe] “PROBE runtime (recorded 2026-09-05): `npx -y bland-cli --version` printed `0.6.2` keylessly — the official npm CLI (bland-cli, with dev term…”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.bland.ai/openapi.json, https://docs.bland.ai/swagger.json, https://docs.bland.ai/api/op…”
- [claimed-docs] “See the Pathways API reference to create and manage pathways programmatically.”
ai-native userSubscribe to events via webhooks
weight 2 · round to VapiVapi supports webhook-based events via Server URL (tool-calls messages), custom webhook tools, and CLI webhook forwarding for local debugging, giving AI-native users a documented event subscription mechanism. missing for 10: independent/hands-on confirmation of webhook delivery reliability, a full event-type catalog/schema, and signature/verification documentation.
- [claimed-docs] “Create your own webhook-based tools to extend assistant capabilities”
- [claimed-docs] “When tools are triggered, your Server URL receives a `tool-calls` message”
- [claimed-docs] “Vapi supports OpenAI-style tool/function calling. Assistants can ping your server to perform actions.”
- [claimed-docs] “Forward webhooks to your local server for debugging”
- [claimed-docs] “Custom tools that you create... interact with your systems via webhooks”
Bland documents post-call webhooks (automatic HTTP notifications sent after a call completes) and pathway-level webhook execution at specific conversation points, showing genuine event-driven webhook support tied to call lifecycle. However, evidence only covers call-related events (completion, in-call triggers) — missing for 10: a general-purpose event subscription/webhook management API covering other account events (e.g., evals, batch campaign status, pathway publishes), and any independent confirmation of webhook reliability/configuration options.
- [claimed-docs] “Post-call webhooks are HTTP notifications that Bland AI automatically sends to your server after a phone call completes.”
- [claimed-docs] “Give your agent instructions on how it should respond at specific points of the conversation. Choose between prompting or fixed sentences. E…”
Agentic features
ai-native userGet AI-generated insights and suggestions from my data inside the product
weight 2 · round to BlandVapinone0/10Evidence covers building/testing assistants (Evals, Voice Test Suites), CLI/MCP management tools, and tool-calling, but nothing describes the product itself surfacing AI-generated insights or suggestions derived from a user's own call/usage data (e.g., analytics dashboards with AI-generated recommendations).
Bland offers LLM-judge Evals to grade call quality and an MCP integration that can 'query analytics' on your account data, which are AI-generated evaluative outputs derived from your call data, but there's no dedicated insights/suggestions dashboard or proactive recommendation feature described. Missing for 10: a native analytics/insights UI, evidence of proactive suggestions surfaced to users, and independent confirmation of these AI-generated insights in practice.
- [claimed-docs] “Evals let you measure the quality of your calls with LLM judges.”
- [claimed-docs] “Evals let you measure the quality of your calls with LLM judges. You define eval agents, each of which grades one dimension of a call”
- [claimed-docs] “Memory lets Bland agents remember people across conversations, including calls, SMS, and other channels, so each interaction feels continuou…”
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
ai-native userSet up automations that run autonomously in the background
weight 2 · round to BlandVapi assistants operate autonomously once a call starts—handling conversation flow, invoking custom tools via webhooks, and triggering server-side automations without further human input (vapi-docs-6, vapi-docs-17, vapi-docs-23, vapi-docs-34). However, this autonomy is scoped to an active call session; there is no evidence of scheduled/cron-style background jobs or agent loops that run independently of a triggered call or user interaction. Missing for 10: evidence of scheduled/background triggers outside live calls, persistent autonomous task queues, or proactive (non-call-triggered) automation runs.
- [claimed-docs] “Create your own webhook-based tools to extend assistant capabilities”
- [claimed-docs] “When tools are triggered, your Server URL receives a `tool-calls` message”
- [claimed-docs] “Vapi supports OpenAI-style tool/function calling. Assistants can ping your server to perform actions.”
- [claimed-docs] “Custom tools that you create... interact with your systems via webhooks”
- [claimed-docs] “This guide shows you how to create custom tools, including Function Tools, for your Vapi assistants.”
Bland supports batch calls, webhooks, pathways, and scheduled/triggered call campaigns that run without manual intervention, which constitute a form of autonomous background automation for voice workflows. However, this is scoped to phone-call automation only, not general-purpose background task/agent scheduling. missing for 10: evidence of a generic scheduler/cron-like trigger system, independent hands-on validation of unattended background runs, and confirmation of failure handling/monitoring for long-running autonomous automations.
- [claimed-docs] “Batch calls let you initiate high-volume call campaigns by uploading a list of recipients via CSV.”
- [claimed-docs] “Post-call webhooks are HTTP notifications that Bland AI automatically sends to your server after a phone call completes.”
- [claimed-docs] “Conversational pathways are our new way of prompting Bland that has led to major breakthroughs in realism. Give your agent instructions on h…”
- [claimed-docs] “Give your agent instructions on how it should respond at specific points of the conversation. Choose between prompting or fixed sentences. E…”
- [claimed-docs] “Dispatch AI phone calls to call customers, leads, and to streamline operations.”
- [claimed-docs] “Connect external APIs and take live actions during phone calls.”
ai-native userDelegate tasks to a built-in AI assistant inside the product
weight 3 · round to BlandVapinone0/10Vapi's docs describe building and configuring voice AI assistants for end-users (phone/web calls, tools, testing) and even an MCP server/CLI for managing those assistants, but there is no evidence of a built-in AI assistant/copilot inside the Vapi product itself that the AI-native user can delegate platform tasks to (e.g., an in-dashboard copilot that configures assistants or writes tools for you). The CLI mentions external IDE assistants (Cursor/Windsurf) gaining API knowledge, but that's a third-party tool, not a built-in in-product assistant.
- [claimed-docs] “Your IDE's AI assistant (Cursor, Windsurf, VSCode) gains complete, accurate knowledge of Vapi's APIs and best practices. No more hallucinate…”
- [claimed-docs] “Manage assistants, phone numbers, and calls from your terminal”
- [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
Bland's core product is a built-in AI voice agent that users delegate tasks to (placing/answering calls, executing pathways, calling APIs, remembering context) rather than a separate feature bolted on — e.g., 'Dispatch AI phone calls to call customers, leads, and to streamline operations' and pathway/tool/memory docs show rich task delegation to the built-in agent. Missing for 10: independent/hands-on evidence of real-world task delegation outcomes beyond vendor docs.
- [claimed-docs] “Dispatch AI phone calls to call customers, leads, and to streamline operations.”
- [claimed-docs] “Conversational pathways are our new way of prompting Bland that has led to major breakthroughs in realism. Give your agent instructions on h…”
- [claimed-docs] “Connect external APIs and take live actions during phone calls.”
- [claimed-docs] “Memory lets Bland agents remember people across conversations, including calls, SMS, and other channels, so each interaction feels continuou…”
- [claimed-docs] “Give your agent instructions on how it should respond at specific points of the conversation. Choose between prompting or fixed sentences. E…”
ai-native userOperate the product with natural-language commands
weight 2 · round to BlandVapi exposes an official MCP server that lets any MCP-compatible AI assistant (e.g. Claude Desktop) manage assistants, phone numbers, and calls via natural-language tool calls, and this endpoint is confirmed live and auth-gated in a runtime probe. It also ships a CLI and llms.txt docs optimized for AI agents to operate it. Missing for 10: no first-party evidence of a built-in chat/NL command console inside the Vapi product itself (beyond MCP/CLI proxies), and no independent hands-on report of an agent successfully performing multi-step tasks via MCP.
- [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
- [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
- [claimed-docs] “Your IDE's AI assistant (Cursor, Windsurf, VSCode) gains complete, accurate knowledge of Vapi's APIs and best practices. No more hallucinate…”
- [probe] “official MCP server documented at https://docs.vapi.ai/sdk/mcp-server”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”
- [probe] “PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bas…”
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.vapi.ai/llms.txt # Vapi ## Instructions for AI Agents - For clean Markdown of any page, append `.…”
Bland offers an official MCP server that lets AI agents operate the entire account (calls, pathways, agents, analytics, evals) via natural language, plus an official CLI, both confirmed live via runtime probes. missing for 10: independent/hands-on third-party review of the MCP/CLI natural-language experience beyond vendor docs and probes.
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
- [probe] “official MCP server documented at https://docs.bland.ai/integrations/mcp/overview”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP endpoint https://api.bland.ai/v1/mcp returned HTTP 4…”
- [claimed-docs] “Manage your entire Bland account from the terminal: make calls, build and test pathways, configure phone numbers, and more.”
- [probe] “official CLI documented at https://docs.bland.ai/sdks/cli”
- [probe] “PROBE runtime (recorded 2026-09-05): `npx -y bland-cli --version` printed `0.6.2` keylessly — the official npm CLI (bland-cli, with dev term…”
Api quality
ai-native userExplore an interactive API reference with runnable examples
weight 2 · round drawnVapinone0/10The evidence pack shows static markdown documentation, code snippets, a CLI, and an MCP server, but no interactive API reference or runnable/try-it-out examples are described anywhere in the docs pages cited.
Blandnone0/10Docs reference an API reference for pathways but no evidence of an interactive playground with runnable examples; a probe for OpenAPI/Swagger specs at standard paths returned 404s, suggesting no such interactive reference exists.
- [claimed-docs] “See the Pathways API reference to create and manage pathways programmatically.”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.bland.ai/openapi.json, https://docs.bland.ai/swagger.json, https://docs.bland.ai/api/op…”
ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)
weight 2 · round drawnVapinone0/10The evidence pack covers Vapi's CLI, MCP server, docs, and SDKs, but no citation mentions an OpenAPI spec, API reference schema, or any machine-readable spec file available for download. This is a fair axis for an API-first product, but no supporting evidence exists in the pack.
Blandnone0/10A direct probe found no OpenAPI/Swagger spec at any expected location (all 404s), and no evidence pack item shows a downloadable machine-readable API spec despite extensive API documentation.
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.bland.ai/openapi.json, https://docs.bland.ai/swagger.json, https://docs.bland.ai/api/op…”
ai-native userTest against a sandbox environment without touching production data
weight 1 · round to VapiVapi offers dedicated Voice Test Suites (AI tester simulating conversations) and an Evals framework explicitly for creating mock conversations to 'validate your agents work correctly before production,' plus CLI webhook forwarding to localhost for local development/debugging — all separate from live production calls. Missing for 10: explicit vendor use of the term 'sandbox environment' and independent/hands-on verification that test runs are fully isolated from production data.
- [claimed-docs] “Voice Test Suites enable you to test your AI voice agents through simulated phone conversations.”
- [claimed-docs] “Evals is Vapi's AI agent testing framework that enables you to systematically test assistants and squads using mock conversations with autom…”
- [claimed-docs] “Our AI tester calls your voice agent and follows a script that simulates real customer behavior.”
- [claimed-docs] “you'll create mock conversations, define expected behaviors, and validate your agents work correctly before production”
- [claimed-docs] “Forward webhooks to your local server for debugging”
- [claimed-docs] “Build, test, and deploy voice AI applications without leaving your development environment.”
Bland provides draft/staging pathway editing where live production calls are unaffected, plus a testbed for isolated node testing, canary deployments, and staged version adoption — all functioning as sandbox-like mechanisms distinct from production. However, there's no explicit 'sandbox environment' or dedicated test account/data isolation concept described, and testing still appears to involve real calls/production infrastructure rather than a fully isolated non-production environment. missing for 10: a dedicated sandbox/test-mode account distinct from production billing and phone infrastructure, explicit documentation of synthetic/non-production test data, and independent confirmation that testbed/draft testing never touches real production call data or costs.
- [claimed-docs] “You can also promote a version to staging to test it before it goes live, or send an individual call against a specific”
- [claimed-docs] “When you edit a pathway, you are working on a draft. Live calls are not affected. They keep using the published production version.”
- [claimed-docs] “The testbed lets you take any call ... isolate a specific node interaction, edit the prompt, and run it multiple times to see how the output…”
- [claimed-docs] “The testbed lets you take any call — whether it's a historical call from call logs or a test chat from the pathway editor — isolate a specif…”
- [claimed-docs] “Bland supports canary deployments — a way to run a new release on a separate set of containers alongside your production infrastructure and …”
- [claimed-docs] “This is how you A/B test a new agent release against your live production version, with real calls, before committing to a full rollout.”
ai-native userRely on versioned APIs with a documented deprecation policy
weight 2 · round to BlandVapinone0/10No evidence pack item mentions API versioning scheme, version numbers in endpoints, or any documented deprecation/sunset policy for Vapi's APIs; the docs cover features (CLI, MCP, tools, voices) but not API lifecycle governance.
Bland documents infrastructure versioning concepts (staged/canary rollouts, choosing when to adopt a new release, draft vs. production pathway versions) but there is no evidence of a documented API versioning scheme (e.g., v1/v2 endpoints) or an explicit deprecation policy for its APIs, and the OpenAPI spec probe returned 404s. missing for 10: documented API version numbering/endpoints, an explicit deprecation/sunset policy for APIs, published OpenAPI spec, and independent confirmation of versioning practices.
- [claimed-docs] “You choose when to adopt a new release. Your infrastructure stays on the version you've selected until you decide to move forward.”
- [claimed-docs] “Bland supports canary deployments — a way to run a new release on a separate set of containers alongside your production infrastructure and …”
- [claimed-docs] “You choose when to adopt a new release... Bland supports canary deployments — a way to run a new release on a separate set of containers alo…”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.bland.ai/openapi.json, https://docs.bland.ai/swagger.json, https://docs.bland.ai/api/op…”
Automation depth — how much of the product can run unattendedAutomation depth
How much of the product can run unattended
ai-native userPerform bulk operations across many items at once
weight 2 · round to BlandVapinone0/10The evidence pack documents CLI/SDK/MCP management of individual assistants, phone numbers, and calls, but nowhere shows bulk/batch endpoints or commands (e.g., batch-create, bulk-update, mass-delete) for operating across many items at once.
Batch calls let users upload a CSV of recipients to initiate high-volume call campaigns, directly enabling bulk operations across many items, and the CLI/MCP server extend programmatic/bulk management of pathways, agents, and calls. Missing for 10: independent/hands-on verification of batch call performance at scale, and documentation of bulk operations beyond calls (e.g., bulk pathway or number management).
- [claimed-docs] “Batch calls let you initiate high-volume call campaigns by uploading a list of recipients via CSV.”
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
- [claimed-docs] “Manage your entire Bland account from the terminal: make calls, build and test pathways, configure phone numbers, and more.”
- [probe] “PROBE runtime (recorded 2026-09-05): `npx -y bland-cli --version` printed `0.6.2` keylessly — the official npm CLI (bland-cli, with dev term…”
ai-native userDefine rules that trigger actions automatically on events
weight 3 · round to BlandVapi supports event-driven server-side tools/webhooks that fire on call events (tool-calls messages, server-url events) which act as a rule-trigger mechanism, but this is scoped to voice-call events only rather than a general-purpose automation/rules engine for arbitrary triggers. missing for 10: a documented general condition/trigger-action rules engine spanning non-call events, cross-system automation, or independent hands-on evidence of complex conditional automation chains.
- [claimed-docs] “When tools are triggered, your Server URL receives a `tool-calls` message”
- [claimed-docs] “Vapi supports OpenAI-style tool/function calling. Assistants can ping your server to perform actions.”
- [claimed-docs] “Custom tools that you create... interact with your systems via webhooks”
- [claimed-docs] “Create your own webhook-based tools to extend assistant capabilities”
Bland's pathways let users define conditional logic that executes webhooks/API calls at specific conversation nodes, and post-call webhooks automatically fire HTTP notifications when a call-completion event occurs — this is a documented rules-trigger-action-on-event mechanism. missing for 10: independent/hands-on verification of the webhook triggering in production, and evidence of event types beyond call-based ones (e.g., generic account-level event automation).
- [claimed-docs] “Post-call webhooks are HTTP notifications that Bland AI automatically sends to your server after a phone call completes.”
- [claimed-docs] “Give your agent instructions on how it should respond at specific points of the conversation. Choose between prompting or fixed sentences. E…”
- [claimed-docs] “Connect external APIs and take live actions during phone calls.”
- [claimed-docs] “Conversational pathways are our new way of prompting Bland that has led to major breakthroughs in realism. Give your agent instructions on h…”
ai-native userSchedule recurring jobs or workflows
weight 2 · round drawnVapinone0/10No evidence in the pack shows Vapi supports scheduling recurring jobs, workflows, or automated recurring calls/tasks — the docs cover assistants, tools, CLI, MCP server, testing, and voice customization but nothing about cron-like scheduling or recurring automation triggers.
Blandnone0/10Bland's docs cover batch calls, pathways, webhooks, and API integrations, but nothing describes scheduling recurring jobs or workflows (e.g., cron-like triggers for calls or campaigns) — batch calls are one-off CSV uploads, not recurring schedules.
- [claimed-docs] “Batch calls let you initiate high-volume call campaigns by uploading a list of recipients via CSV.”
- [claimed-docs] “Dispatch AI phone calls to call customers, leads, and to streamline operations.”
ai-native userVersion, review, and roll back my automations
weight 1 · round to BlandVapinone0/10Vapi's docs cover assistant creation, tools, CLI, MCP server, and testing/evals, but there is no mention of versioning assistant configurations, review/approval workflows, or rollback to prior versions of automations. No evidence pack item addresses version history or rollback capability.
Bland pathways support draft/staging/production versioning, A/B testing new releases against live production, and canary/staged rollout with adoption control (bland-docs-2, bland-docs-14, bland-docs-19, bland-docs-21, bland-docs-29, bland-docs-31), plus a testbed and evals for reviewing behavior before shipping (bland-docs-7, bland-docs-8, bland-docs-22, bland-docs-28). However, there is no explicit documentation of an automated 'rollback' mechanism to revert a live pathway/release to a prior version — only forward-adoption and canary controls are described. Missing for 10: explicit rollback/revert-to-previous-version capability, version history/diff view, and independent confirmation of rollback in practice.
- [claimed-docs] “You can also promote a version to staging to test it before it goes live, or send an individual call against a specific”
- [claimed-docs] “You choose when to adopt a new release. Your infrastructure stays on the version you've selected until you decide to move forward.”
- [claimed-docs] “When you edit a pathway, you are working on a draft. Live calls are not affected. They keep using the published production version.”
- [claimed-docs] “This is how you A/B test a new agent release against your live production version, with real calls, before committing to a full rollout.”
- [claimed-docs] “Bland supports canary deployments — a way to run a new release on a separate set of containers alongside your production infrastructure and …”
- [claimed-docs] “You choose when to adopt a new release... Bland supports canary deployments — a way to run a new release on a separate set of containers alo…”
- [claimed-docs] “Evals let you measure the quality of your calls with LLM judges.”
- [claimed-docs] “The testbed lets you take any call ... isolate a specific node interaction, edit the prompt, and run it multiple times to see how the output…”
- [claimed-docs] “Evals let you measure the quality of your calls with LLM judges. You define eval agents, each of which grades one dimension of a call”
- [claimed-docs] “The testbed lets you take any call — whether it's a historical call from call logs or a test chat from the pathway editor — isolate a specif…”
Compliance trust — stories about compliance trust in this arenaCompliance trust
Stories about compliance trust in this arena
Compliance
founderMeet call-recording consent and disclosure obligations with per-call recording controls and configurable data retention
weight 2 · round drawnVapinone0/10The evidence pack covers HIPAA toggle, SSO/RBAC, testing/evals, and CLI/MCP tooling, but contains no mention of call-recording enable/disable controls, consent disclosure features, or configurable data retention policies for call recordings — a fair and applicable axis for a voice-calling platform.
platform-engineerRun regulated workloads with HIPAA/BAA support, SOC 2, and data-residency options
weight 2 · round to VapiVapi documents HIPAA compliance toggle in the dashboard and enterprise SSO/RBAC, indicating some regulated-workload support, but evidence lacks explicit SOC 2 certification details, BAA execution process, or data-residency region options. missing for 10: SOC 2 report/certification evidence, explicit BAA process details, data-residency/region selection options, independent audit confirmation.
- [claimed-docs] “HIPAA compliance can be ensured by turning on HIPAA compliance in the Dashboard.”
- [claimed-docs] “Single Sign On (SSO) supported for Okta, Azure AD, SAML, and OIDC * Role Based Access Control (RBAC)”
- [claimed-docs] “Single Sign On (SSO) supported for Okta, Azure AD, SAML, and OIDC”
Blandnone0/10The evidence pack contains no mention of HIPAA, BAA, SOC 2 certification, or data-residency options anywhere in the docs or probes; only a generic tagline calling Bland an 'enterprise' platform for 'regulated' workflows without specifics. Missing for 10: HIPAA/BAA documentation, SOC 2 report or certification evidence, data-residency/region controls, and any compliance attestations.
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.bland.ai/llms.txt # Bland Documentation Bland is an enterprise voice AI platform for high-volume, …”
Deployment scale — stories about deployment scale in this arenaDeployment scale
Stories about deployment scale in this arena
Scale
platform-engineerSee documented concurrency limits and scale to many simultaneous calls without manual capacity begging
weight 2 · round to BlandVapinone0/10No evidence pack items document concurrency limits, per-org call caps, autoscaling behavior, or capacity-request processes; the docs cover CLI, MCP server, tools, voices, and enterprise SSO but nothing about simultaneous-call scaling limits or how to raise them.
Docs mention batch calling for high-volume campaigns and enterprise infrastructure controls (canary releases, staged rollout), implying some capacity for scale, but there is no documented concurrency limit, rate ceiling, or auto-scaling guarantee, and no evidence that capacity increases don't require contacting sales/support. Missing for 10: explicit concurrent-call limits, auto-scaling documentation, and evidence that scaling doesn't require manual requests to Bland.
- [claimed-docs] “Batch calls let you initiate high-volume call campaigns by uploading a list of recipients via CSV.”
- [claimed-docs] “Bland supports canary deployments — a way to run a new release on a separate set of containers alongside your production infrastructure and …”
- [claimed-docs] “You choose when to adopt a new release... Bland supports canary deployments — a way to run a new release on a separate set of containers alo…”
Self host
platform-engineerSelf-host the voice agent runtime from open-source code on my own infrastructure
weight 3 · round drawnVapinone0/10Vapi is presented entirely as a hosted SaaS platform with a CLI, MCP server, and dashboard for managing cloud-hosted assistants; there is no evidence of open-source runtime code or any self-hosting option. One community note even references that a different product (pipecat by daily.co) is the open-source alternative, implicitly contrasting with Vapi's closed hosted model.
- [community] “Vapi is also built on media framework by daily.co. They have an open source version of voice ai called pipecat.”
- [claimed-docs] “Build, test, and deploy voice AI applications without leaving your development environment.”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”
Blandnone0/10Bland is presented throughout as a hosted enterprise SaaS platform (managed infrastructure, release adoption controls, canary deployments on Bland's own containers) with no mention of open-source code or a self-hostable runtime; a community post even shows a user asking for an open-source alternative because Bland itself isn't one.
- [claimed-docs] “You choose when to adopt a new release. Your infrastructure stays on the version you've selected until you decide to move forward.”
- [claimed-docs] “Bland supports canary deployments — a way to run a new release on a separate set of containers alongside your production infrastructure and …”
- [community] “I want to experiment with building my own phone agent. Currently experimented with bland.ai but it gets expensive. Any open source alternati…”
Latency turntaking — stories about latency turntaking in this arenaLatency turntaking
Stories about latency turntaking in this arena
Latency
platform-engineerSee documented end-to-end voice latency numbers or tuning guidance backing the platform's speed claims
weight 3 · round to VapiVapi's docs mention that 'Ultra Fast' vs 'High Intelligence' model settings trade off latency, giving conceptual tuning guidance, but there are no documented end-to-end latency numbers (e.g., ms benchmarks) or detailed tuning playbooks in the evidence pack. Missing for 10: concrete latency benchmark figures, component-level latency breakdowns (STT/LLM/TTS), and independent verification of speed claims.
- [claimed-docs] “Ultra Fast uses speed-optimized models for lower latency. High Intelligence accepts higher latency for greater capability.”
Turn taking
developerRely on the agent to handle interruptions (barge-in) gracefully — stopping speech, updating context, and recovering the turn
weight 3 · round drawnVapinone0/10The evidence pack covers CLI, MCP server, tools, voices, compliance, and testing, but contains no documentation or evidence about interruption handling, barge-in behavior, context updates during interruptions, or turn recovery mechanics. This is a core latency/turn-taking capability for voice agents, but nothing in the pack addresses it directly.
Blandnone0/10The evidence pack covers pathways, webhooks, batch calls, evals, MCP/CLI integrations, and infrastructure features, but contains no mention of barge-in, interruption handling, stopping TTS mid-utterance, or turn recovery logic — a core voice-agent capability that would be a fair and expected axis for this product type. missing for 10: any documentation or claim about detecting user interruptions, halting agent speech, updating context after a barge-in, and resuming/recovering the conversational turn.
developerEnable noise suppression or audio filtering so the agent stays coherent on noisy real-world calls
weight 1 · round drawnVapinone0/10No evidence pack items mention noise suppression, background noise filtering, or audio denoising features for handling noisy real-world calls; the pack covers voices, tools, CLI, MCP, latency modes, and testing but nothing about audio filtering.
developerUse model-based end-of-turn detection beyond simple VAD silence timeouts so the agent doesn't talk over slow speakers
weight 2 · round drawnVapinone0/10The evidence pack lacks any mention of model-based end-of-turn detection, turn-taking models, or semantic VAD alternatives; only latency tuning between 'Ultra Fast' and 'High Intelligence' model modes is documented, which does not address end-of-turn detection beyond silence-based VAD. Missing for 10: any documentation of a model-based or semantic end-of-turn detector, comparison to VAD-only timeout behavior, or configuration options for turn-taking sensitivity.
- [claimed-docs] “Ultra Fast uses speed-optimized models for lower latency. High Intelligence accepts higher latency for greater capability.”
Blandnone0/10The evidence pack contains no mention of turn-taking, end-of-turn detection, VAD, or interruption handling mechanisms of any kind — nothing addresses how Bland decides when a speaker has finished talking. This is a fair and applicable axis for a voice AI platform, but no capability is documented; missing for 10: any mention of end-of-turn/turn-taking model, VAD configuration, or handling of slow speakers/pauses.
Openness — open source, data portability, and self-hosting storiesOpenness
Open source, data portability, and self-hosting stories
ai-native userDo everything through the API that I can do in the UI
weight 2 · round to VapiVapi's API/SDK covers assistant, phone number, and call management, backed by a full CLI (vapi-docs-2 through 4, 27, 31-32, verified live in vapi-probe-rt-1) and an official MCP server exposing the same operations for external agents (vapi-docs-5,33, confirmed live and auth-gated in vapi-probe-rt-2). Dashboard-only conveniences like Voice Library browsing/preview remain UI-centric, though voice IDs and custom voices can be set via API (vapi-docs-9,24,26,35). Missing for 10: explicit confirmation that every dashboard-only feature (e.g., visual Voice Library browsing, HIPAA toggle) has a full API equivalent, and independent (non-vendor) verification of full UI/API parity.
- [claimed-docs] “Manage assistants, phone numbers, and calls from your terminal”
- [claimed-docs] “vapi listen --forward-to localhost:3000/webhook”
- [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
- [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
- [claimed-docs] “Build, test, and deploy voice AI applications without leaving your development environment.”
- [claimed-docs] “The CLI auto-detects your tech stack and sets up everything you need.”
- [probe] “PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bas…”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”
- [claimed-docs] “You can use your own custom voice with any supported provider by setting the `voice` property in your assistant configuration”
- [claimed-docs] “Browse and preview voices there, then copy a voice's ID to use on an assistant.”
Bland's docs show extensive programmatic control mirroring UI features: pathway creation/versioning, testbed, evals, batch calls, webhooks, SIP/phone number management, and a CLI/MCP server that explicitly lets users 'manage your entire Bland account from the terminal' and perform the same actions (calls, pathways, agents, analytics) as the UI. This breadth strongly supports API parity, though there's no explicit first-party statement guaranteeing 100% feature parity and no discoverable OpenAPI spec (all candidate URLs 404), so full parity isn't independently confirmed. Missing for 10: an explicit parity guarantee/documentation and a machine-readable OpenAPI spec, plus independent hands-on confirmation that every UI action has an API equivalent.
- [claimed-docs] “See the Pathways API reference to create and manage pathways programmatically.”
- [claimed-docs] “Manage your entire Bland account from the terminal: make calls, build and test pathways, configure phone numbers, and more.”
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
- [claimed-docs] “The testbed lets you take any call ... isolate a specific node interaction, edit the prompt, and run it multiple times to see how the output…”
- [claimed-docs] “Batch calls let you initiate high-volume call campaigns by uploading a list of recipients via CSV.”
- [claimed-docs] “Evals let you measure the quality of your calls with LLM judges.”
- [probe] “official CLI documented at https://docs.bland.ai/sdks/cli”
- [probe] “PROBE runtime (recorded 2026-09-05): `npx -y bland-cli --version` printed `0.6.2` keylessly — the official npm CLI (bland-cli, with dev term…”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.bland.ai/openapi.json, https://docs.bland.ai/swagger.json, https://docs.bland.ai/api/op…”
ai-native userExport all of my data in open formats and leave
weight 3 · round drawnVapinone0/10No evidence of a data export feature, open-format data dumps, or account portability/deletion tooling; the docs cover CLI, MCP server, tools, and voice features but nothing about exporting assistants/call data/transcripts in open formats for user exit. Missing for 10: documented export functionality, open data formats supported, and any account/data portability guarantees.
ai-native userRead the product's source under an open license
weight 2 · round drawnVapinone0/10Vapi is a closed, commercial SaaS platform; nothing in the evidence indicates its core source (server, dashboard, or model runtime) is published under an open license—only SDKs/CLI tooling and docs are mentioned, and the one open-source reference (pipecat) is a third-party project, not Vapi itself.
- [community] “Vapi is also built on media framework by daily.co. They have an open source version of voice ai called pipecat.”
Blandnone0/10Bland is a closed, commercial SaaS voice AI platform; there is no evidence of any open-source license or public source code repository. A community comment explicitly asks for an open-source alternative, implying Bland itself is not open source. This is an applicable axis (a product could publish open-source components) but no evidence supports it.
- [community] “I want to experiment with building my own phone agent. Currently experimented with bland.ai but it gets expensive. Any open source alternati…”
ai-native userSelf-host the core product
weight 3 · round drawnVapinone0/10Vapi is presented exclusively as a hosted cloud platform (dashboard, hosted MCP server, hosted API/CLI against cloud endpoints); nothing in the evidence pack mentions a self-hostable core engine, open-source repo for the core product, or on-prem deployment option (the mentioned pipecat is a different open-source project, not Vapi itself). This is a fair axis for a voice-AI platform, but no evidence supports self-hosting.
- [community] “Vapi is also built on media framework by daily.co. They have an open source version of voice ai called pipecat.”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”
- [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
Blandnone0/10Bland is documented as a hosted enterprise voice AI platform (call dispatch, pathways, SIP, MCP, CLI) with no mention of an on-prem/self-hosted deployment option; 'enterprise' release controls (docs-14, docs-29, docs-31) only govern version adoption timing on Bland's own infrastructure, not customer self-hosting. A community post explicitly looks for an open-source self-hostable alternative because Bland itself doesn't offer this.
- [claimed-docs] “You choose when to adopt a new release. Your infrastructure stays on the version you've selected until you decide to move forward.”
- [claimed-docs] “Bland supports canary deployments — a way to run a new release on a separate set of containers alongside your production infrastructure and …”
- [claimed-docs] “You choose when to adopt a new release... Bland supports canary deployments — a way to run a new release on a separate set of containers alo…”
- [community] “I want to experiment with building my own phone agent. Currently experimented with bland.ai but it gets expensive. Any open source alternati…”
Pricing plans — plan structure and value — what each tier costs and what it unlocksPricing plans
Plan structure and value — what each tier costs and what it unlocks
Pricing
founderSee published per-minute or usage pricing and estimate cost per call before committing
weight 2 · round drawnVapinone0/10The evidence pack contains no mention of pricing pages, per-minute rates, usage-based cost breakdowns, or any pricing calculator/estimator; all evidence is about docs, CLI, MCP server, tools, and testing features. Absence of evidence for this applicable axis yields none.
Blandnone0/10No evidence pack item shows published per-minute or usage pricing, a pricing page, or any cost calculator; the only pricing-adjacent mention is a community complaint that Bland 'gets expensive' with no figures. Missing for 10: published price list, per-minute rate documentation, cost calculator or estimator tool.
- [community] “I want to experiment with building my own phone agent. Currently experimented with bland.ai but it gets expensive. Any open source alternati…”
Privacy posture — data-handling and privacy storiesPrivacy posture
Data-handling and privacy stories
ai-native userChoose where my data is stored (region/residency)
weight 2 · round drawnVapinone0/10The evidence pack contains no mention of data residency, regional storage options, or geographic data controls anywhere in Vapi's docs (only HIPAA toggle and SSO/RBAC are mentioned as compliance features). Missing for 10: any documentation of region selection, data residency guarantees, or storage location controls.
Blandnone0/10No evidence in the pack mentions data residency, regional hosting options, or geographic storage controls for Bland; enterprise/infra docs discuss release versioning and SIP/canary deployments but not region selection. Missing for 10: any mention of data residency options, region-specific hosting, or compliance-driven storage location controls.
ai-native userPrevent my data from being used to train AI models
weight 3 · round drawnVapinone0/10No evidence in the pack addresses data-training opt-out, model-training data policies, or privacy controls preventing AI training use; only HIPAA and SSO/RBAC enterprise features are mentioned, which are unrelated to this specific claim. missing for 10: explicit data-training opt-out policy, DPA/ToS language on training use, dashboard/API setting to exclude data from model training.
ai-native userControl data retention and deletion
weight 2 · round drawnVapinone0/10The evidence pack covers HIPAA, SSO/RBAC, tooling, CLI, and MCP server features but contains no documentation about data retention policies, call/recording deletion controls, or user-initiated data export/erasure mechanisms. Missing for 10: retention period settings, deletion/erasure APIs or dashboard controls, data export tools, and any policy documentation on how long call data/transcripts are stored.
Blandnone0/10No evidence pack items address data retention policies, data deletion controls, or privacy/compliance settings for call recordings, transcripts, or memory data. This axis clearly applies to an enterprise voice AI platform handling call data, but nothing in the evidence documents retention periods, deletion APIs, or GDPR/CCPA-style data controls. Missing for 10: retention policy docs, data deletion API/endpoint, compliance certifications, memory/data purge mechanism.
ai-native userOpt out of telemetry and usage tracking
weight 2 · round drawnVapinone0/10No evidence pack item addresses telemetry opt-out or usage tracking controls; Vapi's docs cover HIPAA/SSO/RBAC compliance but not a telemetry toggle.
Telephony — stories about telephony in this arenaTelephony
Stories about telephony in this arena
Call control
developerEscalate a live call to a human with warm or blind transfer, passing context along
weight 2 · round drawnVapinone0/10No evidence in the pack mentions call transfer, warm/blind transfer, or passing context to a human agent; the evidence covers assistants, CLI, MCP server, tools/webhooks, voices, and compliance, but nothing about live-call escalation/transfer capabilities.
Blandnone0/10The evidence pack covers pathways, webhooks, SIP, MCP, CLI, and other Bland features, but contains no mention of call transfer (warm or blind) or handing off a live call to a human agent with context. This is a standard telephony capability that could plausibly be documented, so absence of evidence yields 'none' rather than 'na'.
developerMy agent can send DTMF keypresses, navigate IVR menus, and detect or leave voicemail
weight 1 · round drawnVapinone0/10The evidence pack contains no mention of DTMF keypress sending, IVR menu navigation, or voicemail detection/leaving functionality anywhere in the docs or community items — these telephony-specific capabilities are entirely unevidenced despite being a plausible axis for a voice AI telephony product.
Campaigns
founderRun batch outbound call campaigns with scheduling and throughput controls
weight 2 · round to BlandVapinone0/10Evidence covers assistant creation, tools, CLI, MCP server, testing, and voice customization, but nothing addresses batch/outbound campaign management, call scheduling, or throughput/concurrency controls for bulk dialing. No mention of a campaigns API, CSV/list upload, dialer pacing, or rate-limiting controls for outbound calling at scale.
Bland's docs explicitly describe batch calls for uploading CSV recipient lists to run high-volume outbound campaigns, backed by scheduling/throughput-related infrastructure like SIP integration, phone number management, and analytics/evals to monitor campaign performance. Missing for 10: explicit documentation of scheduling controls (e.g., call windows/timing) and rate-limiting/throughput knobs specifically, plus independent hands-on verification of batch campaign behavior at scale.
- [claimed-docs] “Batch calls let you initiate high-volume call campaigns by uploading a list of recipients via CSV.”
- [claimed-docs] “Dispatch AI phone calls to call customers, leads, and to streamline operations.”
- [claimed-docs] “Bland supports inbound and outbound SIP. Point calls from your carrier or PBX at Bland to be answered by an agent, or have Bland place calls…”
- [claimed-docs] “Number porting to bring existing numbers to Bland”
Numbers
developerProvision phone numbers and run both inbound and outbound calls through the platform's API
weight 3 · round drawnDocs explicitly cover creating an assistant, connecting it to a phone number, and making inbound/outbound calls via the quickstart, plus CLI/MCP support for managing phone numbers and calls, and SIP for advanced telephony integration, with runtime probes confirming the CLI and MCP endpoints work as documented. Missing for 10: independent hands-on confirmation of actual outbound call placement via raw API (only demo/inbound anecdote in community evidence) and explicit multi-number provisioning workflow details.
- [claimed-docs] “const assistant = await vapi.assistant”
- [claimed-docs] “Create a voice assistant, connect it to a phone number, and make your first calls.”
- [claimed-docs] “In under 5 minutes, you'll create a voice assistant and start talking to it over the phone.”
- [claimed-docs] “Manage assistants, phone numbers, and calls from your terminal”
- [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
- [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
- [claimed-docs] “Use any SIP softphone (e.g., Zoiper, Linphone) to dial your SIP URI”
- [claimed-docs] “This guide shows you how to set up and test SIP calls to your Vapi assistant using any SIP client or softphone.”
- [probe] “PROBE runtime (recorded 2026-09-05): the official Vapi CLI installed via the vendor's one-liner (`curl -sSL https://vapi.ai/install.sh | bas…”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”
- [community] “Called the demo number, sounds smooth! Good luck.”
Bland's docs explicitly document creating/managing inbound phone numbers, outbound call dispatch via API, bringing your own Twilio numbers, and SIP for both inbound/outbound, all programmatically accessible, plus a CLI/MCP that manage phone numbers and calls end-to-end. Missing for 10: no explicit REST API reference/OpenAPI spec confirmed (probe shows openapi.json 404s) and no independent hands-on developer report of a full provision+call round trip.
- [claimed-docs] “Dispatch AI phone calls to call customers, leads, and to streamline operations.”
- [claimed-docs] “Create inbound phone numbers for customer support, etc.”
- [claimed-docs] “Users can connect their own Twilio account to Bland, easily bring over your existing phone numbers to use within the platform.”
- [claimed-docs] “Number porting to bring existing numbers to Bland”
- [claimed-docs] “Bland supports inbound and outbound SIP. Point calls from your carrier or PBX at Bland to be answered by an agent, or have Bland place calls…”
- [claimed-docs] “Manage your entire Bland account from the terminal: make calls, build and test pathways, configure phone numbers, and more.”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.bland.ai/openapi.json, https://docs.bland.ai/swagger.json, https://docs.bland.ai/api/op…”
Sip
platform-engineerConnect my own carrier or PBX via SIP trunking (or import Twilio/Telnyx numbers) instead of being locked to bundled telephony
weight 2 · round to BlandDocs confirm SIP connectivity: platform engineers can point any SIP client/softphone at a Vapi SIP URI to make/receive calls (vapi-docs-14, vapi-docs-22), which supports self-hosted/carrier SIP trunking use cases. However, the evidence never explicitly documents PBX/carrier trunk configuration options or a number-import workflow for Twilio/Telnyx accounts. missing for 10: explicit documentation of BYO-carrier/PBX trunk setup steps, and evidence of importing existing Twilio/Telnyx phone numbers into Vapi.
- [claimed-docs] “Use any SIP softphone (e.g., Zoiper, Linphone) to dial your SIP URI”
- [claimed-docs] “This guide shows you how to set up and test SIP calls to your Vapi assistant using any SIP client or softphone.”
Bland's docs explicitly document inbound/outbound SIP trunking to connect a customer's own carrier or PBX, plus number porting, and separately support connecting an existing Twilio account/numbers. Missing for 10: no mention of Telnyx import specifically and no independent/hands-on validation of SIP setup success.
- [claimed-docs] “Bland supports inbound and outbound SIP. Point calls from your carrier or PBX at Bland to be answered by an agent, or have Bland place calls…”
- [claimed-docs] “Number porting to bring existing numbers to Bland”
- [claimed-docs] “Users can connect their own Twilio account to Bland, easily bring over your existing phone numbers to use within the platform.”
Testing analytics — stories about testing analytics in this arenaTesting analytics
Stories about testing analytics in this arena
Analytics
ai-native userThe platform's AI reviews my calls for me — scoring quality, flagging failures, and analyzing resolution automatically
weight 2 · round to BlandVapi's Evals framework provides automated validation of assistant behavior via mock conversations and its Voice Test Suites use an AI tester to simulate calls and check outcomes, which is adjacent to automated call review, but neither is documented as automatically scoring, flagging failures in, or analyzing resolution of real production calls after the fact. Missing for 10: evidence of automatic post-call quality scoring on live/production transcripts, explicit failure-flagging analytics, and resolution-outcome analysis dashboards distinct from pre-production test suites.
- [claimed-docs] “Evals is Vapi's AI agent testing framework that enables you to systematically test assistants and squads using mock conversations with autom…”
- [claimed-docs] “you'll create mock conversations, define expected behaviors, and validate your agents work correctly before production”
- [claimed-docs] “Voice Test Suites enable you to test your AI voice agents through simulated phone conversations.”
- [claimed-docs] “Our AI tester calls your voice agent and follows a script that simulates real customer behavior.”
Bland's Evals feature explicitly lets users define LLM-judge agents that grade calls on custom dimensions (quality, resolution, etc.), and the Testbed lets you replay and analyze specific call nodes to spot failures — directly matching automated call review/scoring. However, this requires the user to configure eval criteria rather than being a fully out-of-the-box automatic analysis, and there's no evidence of a pre-built default 'failure flagging' report. Missing for 10: evidence of fully automatic, no-setup call scoring/dashboard, and independent/hands-on validation of eval accuracy.
- [claimed-docs] “Evals let you measure the quality of your calls with LLM judges.”
- [claimed-docs] “Evals let you measure the quality of your calls with LLM judges. You define eval agents, each of which grades one dimension of a call”
- [claimed-docs] “The testbed lets you take any call ... isolate a specific node interaction, edit the prompt, and run it multiple times to see how the output…”
- [claimed-docs] “The testbed lets you take any call — whether it's a historical call from call logs or a test chat from the pathway editor — isolate a specif…”
founderSee call analytics — success rates, durations, outcomes, sentiment — in dashboards without building my own
weight 2 · round to BlandVapinone0/10The evidence pack documents assistant creation, tools, testing (Evals, Voice Test Suites), CLI/MCP management, and voice library features, but contains no mention of a dashboard showing call success rates, durations, outcomes, or sentiment analytics — the core of this story.
Evidence confirms Bland has an analytics layer (MCP server can 'query analytics') and quality-grading tools like Evals (LLM judges scoring call dimensions) and a testbed for reviewing call interactions, implying some built-in metrics exist. However, there is no direct evidence of an actual dashboard UI showing success rates, call durations, outcomes, or sentiment trends over time — analytics access shown is via MCP/API query rather than a visual dashboard. Missing for 10: screenshots or docs of a native analytics dashboard, explicit mention of success-rate/duration/sentiment metrics, and independent confirmation the dashboard requires no custom building.
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
- [claimed-docs] “Evals let you measure the quality of your calls with LLM judges. You define eval agents, each of which grades one dimension of a call”
- [claimed-docs] “The testbed lets you take any call — whether it's a historical call from call logs or a test chat from the pathway editor — isolate a specif…”
- [probe] “official MCP server documented at https://docs.bland.ai/integrations/mcp/overview”
Monitoring
platform-engineerMonitor live calls in production and get alerts when agents misbehave or error rates spike
weight 1 · round to BlandVapinone0/10The evidence pack covers pre-production testing (Voice Test Suites, Evals) and webhook debugging via the CLI, but contains no mention of live call monitoring dashboards, real-time alerting, or error-rate-spike detection for production traffic. This is a fair axis for a voice AI platform, but nothing in the pack demonstrates it.
Bland provides post-call webhooks, evals with LLM judges, and an MCP integration that can 'query analytics' — giving some after-the-fact quality/analytics visibility — but there is no documented live-call monitoring dashboard, real-time alerting, or error-rate-spike notification system in the evidence pack. Missing for 10: live/real-time call monitoring UI, configurable alert thresholds, error-rate spike detection/paging, independent confirmation these exist in production.
- [claimed-docs] “Post-call webhooks are HTTP notifications that Bland AI automatically sends to your server after a phone call completes.”
- [claimed-docs] “Evals let you measure the quality of your calls with LLM judges.”
- [claimed-docs] “Evals let you measure the quality of your calls with LLM judges. You define eval agents, each of which grades one dimension of a call”
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
- [claimed-docs] “The testbed lets you take any call — whether it's a historical call from call logs or a test chat from the pathway editor — isolate a specif…”
Testing
developerTest agents with simulated conversations or evals before putting them on real phone calls
weight 2 · round to VapiVapi documents two dedicated testing features directly matching the story: Voice Test Suites for simulated phone conversations via an AI tester following scripted customer behavior, and Evals, a testing framework for mock conversations with automated validation before production. Missing for 10: independent/hands-on developer corroboration of these specific testing features beyond vendor docs.
- [claimed-docs] “Voice Test Suites enable you to test your AI voice agents through simulated phone conversations.”
- [claimed-docs] “Our AI tester calls your voice agent and follows a script that simulates real customer behavior.”
- [claimed-docs] “Evals is Vapi's AI agent testing framework that enables you to systematically test assistants and squads using mock conversations with autom…”
- [claimed-docs] “you'll create mock conversations, define expected behaviors, and validate your agents work correctly before production”
Bland provides explicit pre-production testing tools: Evals (LLM-judge grading of call quality), the Testbed (replay/edit/re-run node interactions on historical or test chats), staging promotion and draft-vs-production pathway separation, and canary/A/B rollout testing against real calls before full deployment. Together these let a developer simulate conversations and grade agent behavior before real phone calls go live. missing for 10: no independent/hands-on evidence of eval accuracy or testbed usage from outside vendor docs, and no explicit description of a pure text-based conversation simulator separate from testbed/staging.
- [claimed-docs] “Evals let you measure the quality of your calls with LLM judges.”
- [claimed-docs] “The testbed lets you take any call ... isolate a specific node interaction, edit the prompt, and run it multiple times to see how the output…”
- [claimed-docs] “Evals let you measure the quality of your calls with LLM judges. You define eval agents, each of which grades one dimension of a call”
- [claimed-docs] “The testbed lets you take any call — whether it's a historical call from call logs or a test chat from the pathway editor — isolate a specif…”
- [claimed-docs] “You can also promote a version to staging to test it before it goes live, or send an individual call against a specific”
- [claimed-docs] “When you edit a pathway, you are working on a draft. Live calls are not affected. They keep using the published production version.”
- [claimed-docs] “This is how you A/B test a new agent release against your live production version, with real calls, before committing to a full rollout.”
- [claimed-docs] “Bland supports canary deployments — a way to run a new release on a separate set of containers alongside your production infrastructure and …”
Tools function calling — stories about tools function calling in this arenaTools function calling
Stories about tools function calling in this arena
Post call
developerExtract structured data from every call — outcomes, entities, dispositions — delivered via API or webhook after the call
weight 2 · round to BlandVapinone0/10The evidence pack shows Vapi's webhook/tool-calling system for live in-call actions (tool-calls messages, custom webhook tools, function calling) but contains no mention of a post-call structured-data/analysis feature (outcomes, entities, dispositions) delivered via API or webhook after the call ends. Missing for 10: any docs on end-of-call reports, structured data extraction schemas, call analysis/summary webhooks, or an API endpoint returning call outcome/entity data.
- [claimed-docs] “When tools are triggered, your Server URL receives a `tool-calls` message”
- [claimed-docs] “Vapi supports OpenAI-style tool/function calling. Assistants can ping your server to perform actions.”
- [claimed-docs] “Custom tools that you create... interact with your systems via webhooks”
- [claimed-docs] “Create your own webhook-based tools to extend assistant capabilities”
Bland documents post-call webhooks that automatically deliver call data to a developer's server after each call, and evals let you grade/classify calls (dispositions) via LLM judges — both align with the API/webhook delivery and outcome-tagging parts of the story. However, there's no explicit documentation of structured entity extraction (e.g., named fields like names, dates, custom entities) as a distinct capability, nor a described webhook payload schema. Missing for 10: explicit entity-extraction feature docs, sample webhook payload showing structured outcome/entity/disposition fields, and independent confirmation of the data delivered.
- [claimed-docs] “Post-call webhooks are HTTP notifications that Bland AI automatically sends to your server after a phone call completes.”
- [claimed-docs] “Evals let you measure the quality of your calls with LLM judges. You define eval agents, each of which grades one dimension of a call”
- [claimed-docs] “Give your agent instructions on how it should respond at specific points of the conversation. Choose between prompting or fixed sentences. E…”
Tools
ai-native userMy voice agent can plug in MCP servers as tool sources so one integration grants it whole toolsets mid-call
weight 2 · round drawnVapinone0/10Evidence only shows Vapi exposing its own APIs as an MCP *server* for external AI assistants (e.g., Claude Desktop) to manage calls/assistants — the reverse direction of the story. There is no documentation or probe showing a Vapi voice assistant can itself act as an MCP *client*, plugging in external MCP servers as tool sources mid-call; tool integration is instead described only via webhook-based custom tools and OpenAI-style function calling.
- [claimed-docs] “so you can manage assistants, phone numbers, and calls from any MCP-compatible AI assistant (like Claude Desktop) or agent framework”
- [claimed-docs] “The Vapi MCP Server exposes Vapi APIs as tools via the Model Context Protocol (MCP), so you can manage assistants, phone numbers, and calls …”
- [claimed-docs] “Create your own webhook-based tools to extend assistant capabilities”
- [claimed-docs] “Vapi supports OpenAI-style tool/function calling. Assistants can ping your server to perform actions.”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://mcp.vapi.ai/mcp returned HTTP 401 — t…”
Blandnone0/10Bland ships an MCP *server* so coding agents can control the Bland account (docs-11/20/27, probe-3/probe-rt-2), which is the opposite role from what the story asks — the voice agent itself acting as an MCP *client* that plugs in external MCP servers as tool sources mid-call. Tool/function calling is documented only via custom HTTP API integrations (bland-docs-4, bland-docs-25), with no mention of the agent consuming MCP servers as a toolset source during calls.
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
- [claimed-docs] “The Bland MCP server lets AI coding agents work with your Bland account through natural language: place and inspect calls, build and validat…”
- [probe] “official MCP server documented at https://docs.bland.ai/integrations/mcp/overview”
- [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP endpoint https://api.bland.ai/v1/mcp returned HTTP 4…”
- [claimed-docs] “The API integration lets you connect your AI agent to any HTTP endpoint.”
- [claimed-docs] “Connect external APIs and take live actions during phone calls.”
developerMy agent can call external APIs and custom functions mid-conversation and speak the result without awkward dead air
weight 3 · round drawnVapi's docs clearly show mid-call function/tool calling via webhook Server URLs and OpenAI-style tool-calls messages, letting the assistant fetch external API results and use them in conversation (vapi-docs-6, vapi-docs-7, vapi-docs-17, vapi-docs-20, vapi-docs-23, vapi-docs-34). However, the pack lacks explicit evidence about mechanisms for avoiding 'dead air' during the API call latency (e.g., async tool config, filler phrases, or interim speech) — only general latency-tuning docs for model selection are present (vapi-docs-15). Missing for 10: explicit documentation/demo of filler/interim speech or async tool handling during function execution, and independent hands-on confirmation that the conversation flow feels seamless during a live tool call.
- [claimed-docs] “Create your own webhook-based tools to extend assistant capabilities”
- [claimed-docs] “Server URL: The endpoint where your function is hosted”
- [claimed-docs] “When tools are triggered, your Server URL receives a `tool-calls` message”
- [claimed-docs] “This guide shows you how to create custom tools, including Function Tools, for your Vapi assistants.”
- [claimed-docs] “Vapi supports OpenAI-style tool/function calling. Assistants can ping your server to perform actions.”
- [claimed-docs] “Custom tools that you create... interact with your systems via webhooks”
- [claimed-docs] “Ultra Fast uses speed-optimized models for lower latency. High Intelligence accepts higher latency for greater capability.”
Bland's pathway/tools docs confirm agents can call external APIs and execute webhooks mid-conversation (bland-docs-4, bland-docs-25, bland-docs-30), which supports live function calling during a call. However, no evidence describes mechanisms for avoiding dead air (e.g., filler speech, streaming partial responses) while waiting on API results. missing for 10: explicit documentation of latency-masking/filler-speech behavior during API calls, and independent/hands-on confirmation of smooth conversational flow.
- [claimed-docs] “The API integration lets you connect your AI agent to any HTTP endpoint.”
- [claimed-docs] “Connect external APIs and take live actions during phone calls.”
- [claimed-docs] “Give your agent instructions on how it should respond at specific points of the conversation. Choose between prompting or fixed sentences. E…”
- [claimed-docs] “Conversational pathways are our new way of prompting Bland that has led to major breakthroughs in realism. Give your agent instructions on h…”
Transcription recording — stories about transcription recording in this arenaTranscription recording
Stories about transcription recording in this arena
Recording
platform-engineerRetrieve full call recordings and transcripts programmatically for every call
weight 2 · round drawnVapinone0/10The evidence pack contains no documentation, API reference, or example demonstrating retrieval of call recordings or transcripts, despite this being a natural, expected capability for a voice AI platform; only tangential tooling (CLI, MCP server, custom tools, testing/evals) is covered.
Transcription
developerGet accurate real-time transcription with control over the STT provider, language models, or key terms
weight 2 · round drawnVapinone0/10The evidence pack contains no documentation about selecting/configuring an STT provider, choosing language models for transcription, or defining custom key terms/vocabulary for real-time transcription — topics like 'transcriber', 'Deepgram', or keyword boosting are absent. While Vapi is clearly a voice AI platform where such controls are a fair axis, none of the provided docs, community items, or probes address it.
Blandnone0/10Bland is a phone-call AI platform where transcription accuracy is clearly relevant, but the evidence pack contains no mention of STT provider selection, language model choice for transcription, or key-term/vocabulary boosting features. missing for 10: STT provider selection, transcription accuracy documentation, custom key terms/vocabulary support, language model configuration for transcription.
Voices tts — stories about voices tts in this arenaVoices tts
Stories about voices tts in this arena
Voices
founderClone a custom brand voice and use it for my agents, with a documented consent process
weight 2 · round to BlandVapinone0/10Vapi docs show you can plug in a 'custom voice' by setting the voice property with a provider ID (vapi-docs-9, vapi-docs-26) and browse a Voice Library (vapi-docs-8, vapi-docs-35), but nothing describes an actual voice-cloning workflow or any documented consent/verification process required before cloning a brand voice. Missing for 10: a described voice-cloning feature/flow, a documented consent or identity-verification process, and any policy language governing voice cloning.
- [claimed-docs] “You can use your own custom voice with any supported provider by setting the `voice` property in your assistant configuration”
- [claimed-docs] “You can use your own custom voice with any supported provider by setting the voice property in your assistant configuration”
- [claimed-docs] “The Voice Library in the Vapi Dashboard lists every voice available to your organization. Browse and preview voices there”
- [claimed-docs] “The **Voice Library** in the [Vapi Dashboard] lists every voice available to your organization. Browse and preview voices there, then copy a…”
Bland documents voice cloning itself (a ~10-second clean sample sets the quality ceiling) which supports the 'clone a custom brand voice' half of the story, but no evidence describes a documented consent process, verification, or authorization requirement for cloning someone's voice. missing for 10: documented consent/verification workflow for voice cloning, legal/compliance guidance on brand-voice rights, independent confirmation of the cloning feature's fidelity.
- [claimed-docs] “A clone needs one clean sample of about ten seconds. Quality of the sample sets the ceiling on quality of the voice”
developerChoose from a broad voice library or plug in multiple TTS providers to get the voice I want
weight 2 · round to VapiDocs confirm a Voice Library with many previewable voices and support for custom voices with any supported provider, indicating multi-provider TTS flexibility, but the evidence never names or lists specific TTS providers (e.g., ElevenLabs, PlayHT, Azure) or details plugging in third-party/custom TTS engines beyond voice ID selection. missing for 10: explicit list of supported TTS providers, documentation of custom/BYO TTS provider integration mechanics, independent hands-on confirmation of voice quality/variety.
- [claimed-docs] “The Voice Library in the Vapi Dashboard lists every voice available to your organization. Browse and preview voices there”
- [claimed-docs] “You can use your own custom voice with any supported provider by setting the `voice` property in your assistant configuration”
- [claimed-docs] “Browse and preview voices there, then copy a voice's ID to use on an assistant.”
- [claimed-docs] “You can use your own custom voice with any supported provider by setting the voice property in your assistant configuration”
- [claimed-docs] “The **Voice Library** in the [Vapi Dashboard] lists every voice available to your organization. Browse and preview voices there, then copy a…”
Docs show voice cloning support (a 10-second sample sets voice quality) but there is no evidence of a broad pre-built voice library to choose from, nor any mention of plugging in multiple third-party TTS providers. Missing for 10: documented voice library/catalog, multi-provider TTS integration, and any comparison of voice options.
- [claimed-docs] “A clone needs one clean sample of about ten seconds. Quality of the sample sets the ceiling on quality of the voice”