Sierra vs Pylon

Actions

1. developerThe agent takes real actions through my APIs — refunds, order changes, subscription updates — with scoped auth per action

   weight 3 · round drawn
   Sierra~partialcommunity5/10

   Sierra's Agent SDK/Studio docs show agents can call customer APIs to execute actions (order changes, refunds implied by 'internal APIs' use), with guardrails, release governance, and human-in-the-loop approval, and a HN commenter confirms it wires directly into a customer's internal APIs. However there's no explicit documentation of per-action scoped auth/permissioning model for API calls. missing for 10: explicit scoped-auth/permission model per action, concrete refund/subscription action examples, independent confirmation of granular auth scoping.

   • [claimed-docs] “write customer journeys as code, track changes, and build sophisticated logic without giving up our development workflows”
   • [claimed-docs] “Understand and rapidly adjust agent behavior by inspecting API calls, logic traces, and more.”
   • [claimed-docs] “Agent Checks and Simulations catching problems proactively, merge approval workflows putting a person in the loop, and split traffic release…”
   • [community] “Having seen this set up for a customer, what is interesting is how it works with internal APIs for customers. It's all point and click in a …”

   proof ↗⚑ flag

   Pylon~partialprobed5/10

   Pylon documents that AI agents can 'take action' and that admins can define Custom API tools so agents can hit internal or third-party API endpoints (e.g., a company's own refund/order APIs), and that Pylon's own API/MCP integration uses OAuth authentication for issues/accounts/contacts. However, there's no concrete example or documentation of granular, action-level scoped auth (e.g., distinct permission scopes for 'refund' vs 'subscription update') — the OAuth scoping described applies to Pylon's own data objects, not to arbitrary external business APIs the agent might call. Missing for 10: explicit examples of refunds/order/subscription actions being executed, and documentation of per-action auth scoping for custom API tools beyond general OAuth.

   • [claimed-docs] “Support Agents can deflect customer questions using content, gather internal context for your team, and take action.”
   • [claimed-docs] “Using a combination of Webhooks, API, and/or Pylon MCP, you can connect your agent to ticketing workflows in Pylon.”
   • [claimed-docs] “If a system does not provide an MCP server, an administrator can define a Custom API tool that agents can use to hit internal or third party…”
   • [claimed-docs] “AI tools authenticate with your Pylon account via OAuth and can read and update issues, accounts, and contacts within the data you can see i…”
   • [probe] “PROBE runtime (recorded 2026-09-10): the public REST API at api.usepylon.com is live and cleanly auth-gated — a keyless GET returned HTTP 40…”
   • [probe] “PROBE runtime (recorded 2026-09-10): the first-party MCP server is live — a keyless JSON-RPC initialize POST to https://mcp.usepylon.com ans…”

   proof ↗⚑ flag

Procedures

1. support ops leadI encode standard operating procedures the agent follows step-by-step for known issue types, with deterministic branching

   weight 2 · round to Sierra
   Sierra✓fullcommunity8/10

   Sierra's Agent SDK/Studio explicitly supports authoring step-by-step journeys ('workflows') as code or no-code, with branching logic ('sophisticated logic'), SOP ingestion via Ghostwriter, and simulation/testing to verify deterministic behavior across scenarios. This directly matches encoding SOPs with deterministic branching for known issue types. missing for 10: independent/hands-on verification of deterministic branching behavior specifically (community evidence is generic, not focused on SOP branching), and docs are login-gated so full workflow-editor detail isn't independently viewable.

   • [claimed-docs] “write customer journeys as code, track changes, and build sophisticated logic without giving up our development workflows”
   • [claimed-docs] “Define step-by-step workflows for your agent from scratch, or use AI to instantly generate journeys from your existing operating procedures.”
   • [claimed-docs] “Upload SOPs, raw transcripts, or even audio interviews of subject matter experts to create customer journeys from scratch.”
   • [claimed-docs] “Verify your agent performs as expected across a wide range of scenarios and avoid regressions.”
   • [community] “Having seen this set up for a customer, what is interesting is how it works with internal APIs for customers. It's all point and click in a …”

   proof ↗⚑ flag

   Pylon~partialclaimed6/10

   Pylon's Runbooks feature lets ops leads write natural-language, step-by-step instructions for the AI Support Agent to follow for specific scenarios, and Skills provide reusable instructions for consistent handling of known workflows — directly matching the SOP-encoding need. However, the docs describe these as natural-language guidance rather than an explicit deterministic branching/decision-tree engine, and Triggers/automations are a separate rule system not tied to agent SOPs. Missing for 10: explicit conditional/branching logic constructs within runbooks, and evidence of deterministic (non-LLM-interpreted) path selection.

   • [claimed-docs] “Write instructions in natural language to help your AI Agent handle specific scenarios.”
   • [claimed-docs] “Skills are reusable instructions that teach Pylon agents how to handle specific workflows consistently.”
   • [claimed-docs] “Build your swarm of AI Agents to help augment your team.”
   • [claimed-docs] “Set up complex workflows and automations to execute your business processes”

   proof ↗⚑ flag

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to Pylon
   Sierra—none0/10

   Direct probes show no llms.txt at sierra.ai (404) and docs.sierra.ai/llms.txt merely resolves to the login SPA HTML shell rather than an actual plain-text agent-oriented index; the real docs are login-gated to contracted customers, so an agent cannot be pointed at a genuine llms.txt or open agent-oriented docs.

   • [probe] “PROBE llms.txt: HTTP 404 at https://sierra.ai/llms.txt”
   • [probe] “PROBE runtime finding (recorded 2026-09-10): docs.sierra.ai has no agent-legible surface — /llms.txt resolves (HTTP 200) to the login SPA's …”

   proof ↗⚑ flag

   Pylon✓fullprobed8/10

   Probes confirm a live llms.txt index at docs.usepylon.com/llms.txt and working per-page .md mirrors (e.g. pylon-mcp.md), so an agent can be pointed at agent-oriented docs and consume them directly. missing for 10: independent third-party confirmation beyond the vendor's own probe, and some per-page md endpoints (e.g. pylon-docs.md) 404 showing coverage is inconsistent.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.usepylon.com/llms.txt # Pylon ## Pylon - [Introduction](https://docs.usepylon.com/pylon-docs/gett…”
   • [probe] “PROBE runtime (recorded 2026-09-10): Pylon's docs are agent-legible — https://docs.usepylon.com/pylon-docs/llms.txt serves a live '# Pylon' …”
   • [probe] “PROBE docs-md: HTTP 200 at https://docs.usepylon.com/pylon-docs.md # Page Not Found The URL `pylon-docs` does not exist. This page may have…”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to Pylon
   Sierra~partialprobed4/10

   Sierra's Agent SDK is described as code-based with dev workflows retained, and one blog post explicitly notes publishing to ChatGPT can be done 'via CI/CD', implying some automation/pipeline support. However there is no dedicated CLI, headless runtime docs, or CI-specific guidance, and docs/API references are login-gated (probe shows /llms.txt is a login SPA and no public OpenAPI spec), so full headless/CI operation is unconfirmed. Missing for 10: explicit CLI/headless execution docs, public API/OpenAPI spec, independent CI usage reports.

   • [claimed-docs] “write customer journeys as code, track changes, and build sophisticated logic without giving up our development workflows”
   • [claimed-docs] “Publish to ChatGPT with one click: Make your agent available on ChatGPT with the push of a button, or via CI/CD.”
   • [probe] “PROBE llms.txt: HTTP 404 at https://sierra.ai/llms.txt”
   • [probe] “PROBE openapi: all candidate paths 404 (https://sierra.ai/openapi.json, https://sierra.ai/swagger.json, https://sierra.ai/api/openapi.json, …”
   • [probe] “PROBE runtime finding (recorded 2026-09-10): docs.sierra.ai has no agent-legible surface — /llms.txt resolves (HTTP 200) to the login SPA's …”

   proof ↗⚑ flag

   Pylon~partialprobed5/10

   Pylon exposes a REST API, webhooks, and triggers that allow programmatic, non-UI interaction with the platform, which supports automation-style usage outside the web UI (pylon-docs-31, pylon-docs-32, pylon-docs-30, pylon-probe-rt-2). However there is no evidence of a CLI, headless mode, or explicit CI/CD integration story — the product is a SaaS support platform accessed via API/OAuth, not something designed to be invoked in a CI pipeline. Missing for 10: explicit CLI/headless execution mode, CI/CD pipeline examples or docs, evidence of automated/scripted runs outside a live SaaS API context.

   • [claimed-docs] “Pylon's API can be used to programmatically access and take action on data within Pylon.”
   • [claimed-docs] “Receive webhooks for various events happening in Pylon.”
   • [claimed-docs] “Set up complex workflows and automations to execute your business processes”
   • [probe] “PROBE runtime (recorded 2026-09-10): the public REST API at api.usepylon.com is live and cleanly auth-gated — a keyless GET returned HTTP 40…”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round to Pylon
   Sierra—none0/10

   No evidence anywhere in the pack mentions MCP servers or the ability to plug external MCP tool servers into Sierra's agents; Sierra's integration mentions are about internal APIs and custom systems, not MCP. missing for 10: any mention of MCP protocol support, MCP client configuration, or third-party tool server integration via MCP.

   ⚑ flag

   Pylon~partialclaimed5/10

   Pylon's agent 'connectors' documentation states that if a third-party system provides an MCP server, agents can use it directly (falling back to a Custom API tool only when none exists), indicating Pylon can consume external MCP servers as tool sources for its AI agents. However, this is a single thin doc line with no setup walkthrough, supported-server list, or hands-on/runtime confirmation of actually connecting to a third-party MCP server. missing for 10: a dedicated 'add MCP server' configuration flow/UI, documentation of supported transports/auth for third-party servers, and independent or runtime evidence of a working third-party MCP connection.

   • [claimed-docs] “If a system does not provide an MCP server, an administrator can define a Custom API tool that agents can use to hit internal or third party…”

   proof ↗⚑ flag

4. ai-native userUse an official CLI

   weight 2 · round drawn
   Sierra—none0/10

   No evidence of an official Sierra CLI tool; the Agent SDK mentions a code-based development workflow but nothing describing a CLI, and probes for llms.txt/openapi return 404s with no CLI reference anywhere in the pack.

   • [claimed-docs] “write customer journeys as code, track changes, and build sophisticated logic without giving up our development workflows”
   • [probe] “PROBE llms.txt: HTTP 404 at https://sierra.ai/llms.txt”
   • [probe] “PROBE openapi: all candidate paths 404 (https://sierra.ai/openapi.json, https://sierra.ai/swagger.json, https://sierra.ai/api/openapi.json, …”

   proof ↗⚑ flag

   Pylon—none0/10

   No evidence of an official Pylon CLI anywhere in docs, changelog, or probes; Pylon offers REST API, webhooks, and an MCP server but no CLI tool is mentioned. missing for 10: any mention of a CLI, CLI installation instructions, or CLI command reference.

   ⚑ flag

5. ai-native userDrive the product through a documented public API

   weight 3 · round to Pylon
   Sierra~partialprobed4/10

   Sierra markets an Agent SDK for building 'customer journeys as code' with API-call inspection, implying a programmatic interface exists, but there's no publicly discoverable OpenAPI spec, llms.txt, or open API reference — docs.sierra.ai is login-gated to contracted customers rather than a documented public API. Missing for 10: publicly accessible API reference/OpenAPI spec, evidence of self-serve API keys or open documentation, independent developer confirmation of using the API without a sales contract.

   • [claimed-docs] “write customer journeys as code, track changes, and build sophisticated logic without giving up our development workflows”
   • [claimed-docs] “Understand and rapidly adjust agent behavior by inspecting API calls, logic traces, and more.”
   • [probe] “PROBE llms.txt: HTTP 404 at https://sierra.ai/llms.txt”
   • [probe] “PROBE openapi: all candidate paths 404 (https://sierra.ai/openapi.json, https://sierra.ai/swagger.json, https://sierra.ai/api/openapi.json, …”
   • [probe] “PROBE runtime finding (recorded 2026-09-10): docs.sierra.ai has no agent-legible surface — /llms.txt resolves (HTTP 200) to the login SPA's …”

   proof ↗⚑ flag

   Pylon✓fullprobed8/10

   Pylon documents a public REST API (docs-31) and webhooks (docs-32), and runtime probes confirm the API is live and properly auth-gated (probe-rt-2). Docs also describe programmatic/agentic access patterns (Custom API tools, OAuth-based AI tool access via docs-34) reinforcing that the API is designed for agentic/programmatic drivers. missing for 10: no discoverable OpenAPI/swagger spec (probe-3 shows 404s across candidate paths) and no independent third-party corroboration of API robustness beyond Pylon's own docs.

   • [claimed-docs] “Pylon's API can be used to programmatically access and take action on data within Pylon.”
   • [claimed-docs] “Receive webhooks for various events happening in Pylon.”
   • [claimed-docs] “AI tools authenticate with your Pylon account via OAuth and can read and update issues, accounts, and contacts within the data you can see i…”
   • [probe] “PROBE runtime (recorded 2026-09-10): the public REST API at api.usepylon.com is live and cleanly auth-gated — a keyless GET returned HTTP 40…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.usepylon.com/openapi.json, https://docs.usepylon.com/swagger.json, https://docs.usepylo…”

   proof ↗⚑ flag

6. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round to Pylon
   Sierra—none0/10

   No evidence anywhere in the pack of scoped or least-privilege API credential/token issuance for agents; docs describe agent building, workflows, channels, and analytics but nothing about credential scoping, permissions, or API key management. Probes even show no public OpenAPI/API docs are accessible (sierra-probe-2, sierra-probe-rt-3), reinforcing the absence of evidence.

   • [probe] “PROBE openapi: all candidate paths 404 (https://sierra.ai/openapi.json, https://sierra.ai/swagger.json, https://sierra.ai/api/openapi.json, …”
   • [probe] “PROBE runtime finding (recorded 2026-09-10): docs.sierra.ai has no agent-legible surface — /llms.txt resolves (HTTP 200) to the login SPA's …”

   proof ↗⚑ flag

   Pylon~partialprobed4/10

   Pylon's MCP integration uses OAuth 2.0 (via AuthKit) and the REST API requires Bearer tokens, so credentials are authenticated and gated (pylon-docs-34, pylon-probe-rt-1/2/3), and access is limited to 'the data you can see in Pylon,' giving a coarse form of least-privilege tied to the issuing user's own permissions. However there is no evidence of a dedicated mechanism to mint distinct, granularly-scoped API keys or OAuth scopes specifically for an agent (e.g., read-only vs write, per-object scoping) separate from a full user account's access. Missing for 10: documented ability to configure fine-grained scopes/permissions per API credential, evidence of agent-specific token restriction UI, and independent confirmation of least-privilege enforcement beyond basic auth gating.

   • [claimed-docs] “AI tools authenticate with your Pylon account via OAuth and can read and update issues, accounts, and contacts within the data you can see i…”
   • [probe] “PROBE runtime (recorded 2026-09-10): Pylon's docs are agent-legible — https://docs.usepylon.com/pylon-docs/llms.txt serves a live '# Pylon' …”
   • [probe] “PROBE runtime (recorded 2026-09-10): the public REST API at api.usepylon.com is live and cleanly auth-gated — a keyless GET returned HTTP 40…”
   • [probe] “PROBE runtime (recorded 2026-09-10): the first-party MCP server is live — a keyless JSON-RPC initialize POST to https://mcp.usepylon.com ans…”

   proof ↗⚑ flag

7. ai-native userBuild against official SDKs

   weight 2 · round to Pylon
   Sierra~partialprobed5/10

   Sierra advertises an official "Agent SDK" for writing customer journeys as code with logic traces and testing (sierra-docs-1/2/4), but runtime probes show no public OpenAPI/swagger spec and the SDK/docs reference is login-gated to contracted customers rather than openly available to any AI-native developer (sierra-probe-2, sierra-probe-rt-3). Missing for 10: publicly accessible API reference/OpenAPI spec, evidence of open sign-up or trial SDK access, and independent developer accounts of building against it outside a paid contract.

   • [claimed-docs] “write customer journeys as code, track changes, and build sophisticated logic without giving up our development workflows”
   • [claimed-docs] “Understand and rapidly adjust agent behavior by inspecting API calls, logic traces, and more.”
   • [claimed-docs] “Verify your agent performs as expected across a wide range of scenarios and avoid regressions.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://sierra.ai/openapi.json, https://sierra.ai/swagger.json, https://sierra.ai/api/openapi.json, …”
   • [probe] “PROBE runtime finding (recorded 2026-09-10): docs.sierra.ai has no agent-legible surface — /llms.txt resolves (HTTP 200) to the login SPA's …”

   proof ↗⚑ flag

   Pylon~partialprobed6/10

   Pylon exposes a documented REST API, webhooks, and an OAuth-secured official MCP server (confirmed live via runtime probes), which together let AI-native developers build integrations and agents against Pylon data. However, there is no evidence of traditional language-specific client SDKs (e.g., Python/Node packages) or an OpenAPI spec (probe found 404s for openapi/swagger endpoints), so 'SDK' support is really API+MCP+webhooks rather than packaged SDKs. Missing for 10: dedicated official SDK packages, publicly discoverable OpenAPI/swagger spec, independent developer corroboration of SDK usage.

   • [claimed-docs] “Pylon's API can be used to programmatically access and take action on data within Pylon.”
   • [claimed-docs] “Receive webhooks for various events happening in Pylon.”
   • [claimed-docs] “AI tools authenticate with your Pylon account via OAuth and can read and update issues, accounts, and contacts within the data you can see i…”
   • [probe] “PROBE runtime (recorded 2026-09-10): the public REST API at api.usepylon.com is live and cleanly auth-gated — a keyless GET returned HTTP 40…”
   • [probe] “PROBE runtime (recorded 2026-09-10): the first-party MCP server is live — a keyless JSON-RPC initialize POST to https://mcp.usepylon.com ans…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.usepylon.com/openapi.json, https://docs.usepylon.com/swagger.json, https://docs.usepylo…”

   proof ↗⚑ flag

8. ai-native userSubscribe to events via webhooks

   weight 2 · round to Pylon
   Sierra—none0/10

   No evidence in the pack mentions webhooks or event subscription capabilities for Sierra; docs pages describe SDK, studio, insights, voice, and channels features but nothing about outbound event notifications. Probes further show no public API spec or open documentation to confirm such a mechanism exists.

   • [probe] “PROBE openapi: all candidate paths 404 (https://sierra.ai/openapi.json, https://sierra.ai/swagger.json, https://sierra.ai/api/openapi.json, …”
   • [probe] “PROBE runtime finding (recorded 2026-09-10): docs.sierra.ai has no agent-legible surface — /llms.txt resolves (HTTP 200) to the login SPA's …”

   proof ↗⚑ flag

   Pylon✓fullclaimed7/10

   Pylon documents a dedicated webhooks feature for receiving events happening in Pylon, directly matching the story. missing for 10: no independent/hands-on corroboration of webhook delivery, no detail on event types/payload schema, or subscription management UI/API specifics.

   • [claimed-docs] “Receive webhooks for various events happening in Pylon.”
   • [claimed-docs] “Using a combination of Webhooks, API, and/or Pylon MCP, you can connect your agent to ticketing workflows in Pylon.”

   proof ↗⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round to Sierra
   Sierra✓fullcommunity8/10

   Sierra's Explorer and Insights products explicitly deliver AI-generated insights: natural-language querying across conversations, automatic weekly briefings on trends/issues with recommendations, and click-to-investigate drill-downs on report data, plus explainability of agent reasoning. This is a first-party documented feature set directly matching the story, with some independent corroboration of Sierra's data-driven production use (sierra-comm-2/3). Missing for 10: independent hands-on validation of the insights/briefing feature specifically (community evidence is about agent setup, not analytics quality), and no detail on data freshness/accuracy limits.

   • [claimed-docs] “Ask any question about your customer experience in natural language, and Explorer identifies the answer across thousands of real conversatio…”
   • [claimed-docs] “Explorer automatically delivers a weekly briefing on key trends, emerging issues, and recommendations — without you having to ask.”
   • [claimed-docs] “Click any data point on a report to launch Explorer and instantly investigate what's driving that trend.”
   • [claimed-docs] “Understand the reasoning behind every agent action or answer—view knowledge sources, systems accessed, and more.”
   • [community] “Having seen this set up for a customer, what is interesting is how it works with internal APIs for customers. It's all point and click in a …”
   • [community] “The magic isn't in a new LLM technology, it is in reliably productionizing a solution for real-world problems... fill in the gap of missing …”

   proof ↗⚑ flag

   Pylon~partialclaimed6/10

   Pylon documents AI summarization of customer conversations synced to CRM accounts (Salesforce), AI-powered knowledge base features, and analytics dashboards, plus AI Support Agents that surface context and take action — all suggesting embedded AI insight generation. However, there's no dedicated 'insights' feature that proactively surfaces suggestions from the user's own data (e.g., trend detection, anomaly alerts) beyond conversation summarization and agent responses. Missing for 10: a first-party 'insights' or 'suggestions' dashboard feature explicitly generating recommendations from account/ticket data, and independent/hands-on validation of insight quality.

   • [claimed-docs] “Every day, Pylon scans your customer channels for conversations that were had. For every conversation, we use AI to summarize the details, a…”
   • [claimed-docs] “Pylon comes with a number of default analytics dashboards with common support metrics. All dashboards are filterable by a wide selection of …”
   • [claimed-docs] “A dedicated place to host your support content, including question-and-answer style articles, internal runbooks, and more -- all supercharge…”
   • [claimed-docs] “Support Agents can deflect customer questions using content, gather internal context for your team, and take action.”
   • [claimed-docs] “The issue log allows you to inspect all the issues the AI agent is associated with and inspect the outcomes and steps taken by the AI.”

   proof ↗⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round drawn
   Sierra~partialcommunity6/10

   Sierra's agents are built to operate autonomously across channels (chat, phone, email, SMS) once deployed, and Explorer explicitly runs in the background to deliver automatic weekly briefings and trend detection 'without you having to ask,' which is genuine unprompted automation. However, there's no documentation of user-configurable scheduled tasks/triggers beyond the always-on conversational agent and the one named automatic-briefing feature. Missing for 10: explicit support for user-defined scheduled/triggered background jobs beyond conversation handling and briefings, and independent/hands-on confirmation of autonomous background execution reliability.

   • [claimed-docs] “Explorer automatically delivers a weekly briefing on key trends, emerging issues, and recommendations — without you having to ask.”
   • [claimed-docs] “Build once and deploy across any channel—chat, phone, email, SMS, and messaging.”
   • [claimed-docs] “Guide care reps through next steps—whether in chat or on calls—automatically capturing and updating customer context as work progresses.”
   • [community] “Having seen this set up for a customer, what is interesting is how it works with internal APIs for customers. It's all point and click in a …”

   proof ↗⚑ flag

   Pylon~partialclaimed6/10

   Pylon documents triggers/automations to execute business processes, webhooks, API access, and AI agents that autonomously handle issues assigned to them with monitoring via issue logs, supporting background autonomous automation. However, evidence is mostly first-party docs with no independent/hands-on confirmation of unattended reliability or long-running autonomy at scale. Missing for 10: independent/hands-on verification of autonomous background execution, detail on failure handling/retries in triggers, and third-party confirmation of agent autonomy in production.

   • [claimed-docs] “Set up complex workflows and automations to execute your business processes”
   • [claimed-docs] “Support Agents can deflect customer questions using content, gather internal context for your team, and take action.”
   • [claimed-docs] “You can assign them to issues you'd like them to take a look at. This means you have full control of which customer issues your AI agent int…”
   • [claimed-docs] “The issue log allows you to inspect all the issues the AI agent is associated with and inspect the outcomes and steps taken by the AI.”
   • [claimed-docs] “Receive webhooks for various events happening in Pylon.”
   • [claimed-docs] “Pylon's API can be used to programmatically access and take action on data within Pylon.”

   proof ↗⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round to Pylon
   Sierra~partialclaimed6/10

   Sierra's Ghostwriter feature lets users delegate agent-building tasks to a built-in AI assistant via natural-language prompts (build/modify agents, generate journeys from SOPs/transcripts), fitting the 'delegate tasks to built-in AI assistant' story. However, this is vendor-only documentation with no independent/hands-on corroboration of Ghostwriter specifically; community evidence instead describes manual point-and-click agent setup, not AI-assistant delegation. Missing for 10: independent/hands-on validation of Ghostwriter's delegation capability, and detail on scope/limits of what can be delegated.

   • [claimed-docs] “Build or modify agents by describing how you want them to behave. Update workflows, systems integrations, guardrails, tone, and style with s…”
   • [claimed-docs] “Upload SOPs, raw transcripts, or even audio interviews of subject matter experts to create customer journeys from scratch.”
   • [claimed-docs] “Define step-by-step workflows for your agent from scratch, or use AI to instantly generate journeys from your existing operating procedures.”

   proof ↗⚑ flag

   Pylon✓fullclaimed8/10

   Pylon ships built-in AI Support Agents that users can build, assign to issues, give runbooks/skills, and monitor outcomes—clearly a built-in assistant users delegate tasks to (assign issues, define workflows, review AI-taken actions). Missing for 10: independent/third-party hands-on validation of assistant task quality and no user-facing UI screenshots/demos beyond docs.

   • [claimed-docs] “Support Agents can deflect customer questions using content, gather internal context for your team, and take action.”
   • [claimed-docs] “Build your swarm of AI Agents to help augment your team.”
   • [claimed-docs] “Write instructions in natural language to help your AI Agent handle specific scenarios.”
   • [claimed-docs] “You can assign them to issues you'd like them to take a look at. This means you have full control of which customer issues your AI agent int…”
   • [claimed-docs] “The issue log allows you to inspect all the issues the AI agent is associated with and inspect the outcomes and steps taken by the AI.”
   • [claimed-docs] “Skills are reusable instructions that teach Pylon agents how to handle specific workflows consistently.”

   proof ↗⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round to Sierra
   Sierra✓fullcommunity7/10

   Sierra provides explicit natural-language control surfaces: Ghostwriter lets users 'build or modify agents by describing how you want them to behave' with 'simple prompts' for workflows, integrations, guardrails, tone and style, and Explorer lets users 'ask any question about your customer experience in natural language.' This directly matches an AI-native user operating the product via NL commands rather than only clicking through UI. Missing for 10: independent hands-on corroboration of NL command reliability/scope beyond vendor docs, and no evidence of NL control over the entire platform (e.g., release governance, channel deployment) rather than just Ghostwriter/Explorer.

   • [claimed-docs] “Build or modify agents by describing how you want them to behave. Update workflows, systems integrations, guardrails, tone, and style with s…”
   • [claimed-docs] “Upload SOPs, raw transcripts, or even audio interviews of subject matter experts to create customer journeys from scratch.”
   • [claimed-docs] “Ask any question about your customer experience in natural language, and Explorer identifies the answer across thousands of real conversatio…”
   • [community] “Having seen this set up for a customer, what is interesting is how it works with internal APIs for customers. It's all point and click in a …”

   proof ↗⚑ flag

   Pylon~partialprobed6/10

   Pylon exposes a live first-party MCP server (confirmed by runtime probes) that lets AI tools authenticate via OAuth and read/update issues, accounts, and contacts, effectively letting an AI-native user drive Pylon via natural-language-mediated agent actions; runbooks and skills also let admins encode natural-language instructions for agent behavior. However, this is achieved through external AI-tool/agent integration rather than a native in-app NL command interface for end users. Missing for 10: evidence of a built-in chat/command-bar UI where a human directly issues natural-language commands to operate Pylon itself, and independent confirmation of real-world NL-driven task completion.

   • [claimed-docs] “AI tools authenticate with your Pylon account via OAuth and can read and update issues, accounts, and contacts within the data you can see i…”
   • [claimed-docs] “Write instructions in natural language to help your AI Agent handle specific scenarios.”
   • [claimed-docs] “Skills are reusable instructions that teach Pylon agents how to handle specific workflows consistently.”
   • [probe] “PROBE runtime (recorded 2026-09-10): Pylon's docs are agent-legible — https://docs.usepylon.com/pylon-docs/llms.txt serves a live '# Pylon' …”
   • [probe] “PROBE runtime (recorded 2026-09-10): the first-party MCP server is live — a keyless JSON-RPC initialize POST to https://mcp.usepylon.com ans…”
   • [claimed-docs] “Using a combination of Webhooks, API, and/or Pylon MCP, you can connect your agent to ticketing workflows in Pylon.”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   Sierra—none0/10

   Probes show no public OpenAPI/interactive reference (404s at openapi.json paths, llms.txt returns login SPA shell), and docs are login-gated to contracted customers rather than an open interactive API reference with runnable examples.

   • [probe] “PROBE llms.txt: HTTP 404 at https://sierra.ai/llms.txt”
   • [probe] “PROBE openapi: all candidate paths 404 (https://sierra.ai/openapi.json, https://sierra.ai/swagger.json, https://sierra.ai/api/openapi.json, …”
   • [probe] “PROBE runtime finding (recorded 2026-09-10): docs.sierra.ai has no agent-legible surface — /llms.txt resolves (HTTP 200) to the login SPA's …”

   proof ↗⚑ flag

   Pylon—none0/10

   Pylon documents a REST API and webhooks (pylon-docs-31, pylon-docs-32) but there is no evidence of an interactive API reference with runnable/try-it examples — probes explicitly found no OpenAPI/Swagger spec at any candidate path (pylon-probe-3), only static markdown docs.

   • [claimed-docs] “Pylon's API can be used to programmatically access and take action on data within Pylon.”
   • [claimed-docs] “Receive webhooks for various events happening in Pylon.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.usepylon.com/openapi.json, https://docs.usepylon.com/swagger.json, https://docs.usepylo…”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   Sierra—none0/10

   Direct probes show no OpenAPI/Swagger spec at common paths (all 404) and no machine-readable llms.txt index; docs are login-gated rather than publicly exposing a spec. missing for 10: publicly downloadable OpenAPI/Swagger file, any machine-readable API spec endpoint.

   • [probe] “PROBE llms.txt: HTTP 404 at https://sierra.ai/llms.txt”
   • [probe] “PROBE openapi: all candidate paths 404 (https://sierra.ai/openapi.json, https://sierra.ai/swagger.json, https://sierra.ai/api/openapi.json, …”
   • [probe] “PROBE runtime finding (recorded 2026-09-10): docs.sierra.ai has no agent-legible surface — /llms.txt resolves (HTTP 200) to the login SPA's …”

   proof ↗⚑ flag

   Pylon—none0/10

   Pylon has a REST API and docs, but explicit probes for OpenAPI/Swagger spec files at standard paths all returned 404, and no evidence anywhere shows a downloadable machine-readable spec (OpenAPI, JSON Schema, etc.) for its API.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.usepylon.com/openapi.json, https://docs.usepylon.com/swagger.json, https://docs.usepylo…”
   • [claimed-docs] “Pylon's API can be used to programmatically access and take action on data within Pylon.”
   • [probe] “PROBE runtime (recorded 2026-09-10): the public REST API at api.usepylon.com is live and cleanly auth-gated — a keyless GET returned HTTP 40…”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round to Sierra
   Sierra~partialclaimed5/10

   Sierra's docs mention scenario testing and regression avoidance (sierra-docs-4) and 'Agent Checks and Simulations' for proactive problem catching (sierra-docs-18), implying some pre-production testing capability, but there is no explicit mention of a dedicated sandbox environment isolated from production data. Missing for 10: explicit sandbox/staging environment documentation, confirmation that test runs don't touch production data, and independent/hands-on verification of this separation.

   • [claimed-docs] “Verify your agent performs as expected across a wide range of scenarios and avoid regressions.”
   • [claimed-docs] “Agent Checks and Simulations catching problems proactively, merge approval workflows putting a person in the loop, and split traffic release…”

   proof ↗⚑ flag

   Pylon~partialclaimed3/10

   Pylon's Support Agent docs mention a 'test' feature to simulate interactions with the AI without deploying it live (pylon-docs-17), which suggests some sandboxed testing before touching real customer issues, but there's no documented sandbox environment, staging API, or test dataset separate from production data. Missing for 10: explicit sandbox/staging environment, test API keys or test accounts isolated from production, and confirmation that simulated interactions don't touch real customer/production data.

   • [claimed-docs] “Simulate an interaction with the AI and see how it handles different questions.”
   • [claimed-docs] “You can assign them to issues you'd like them to take a look at. This means you have full control of which customer issues your AI agent int…”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Sierra—none0/10

   No evidence of a public versioned API reference or a documented deprecation policy; probes show no OpenAPI spec and docs are login-gated, and llms.txt/openapi.json all 404 or resolve to a login shell rather than API docs.

   • [probe] “PROBE llms.txt: HTTP 404 at https://sierra.ai/llms.txt”
   • [probe] “PROBE openapi: all candidate paths 404 (https://sierra.ai/openapi.json, https://sierra.ai/swagger.json, https://sierra.ai/api/openapi.json, …”
   • [probe] “PROBE runtime finding (recorded 2026-09-10): docs.sierra.ai has no agent-legible surface — /llms.txt resolves (HTTP 200) to the login SPA's …”

   proof ↗⚑ flag

   Pylon—none0/10

   Evidence shows Pylon has a REST API, webhooks, and MCP server, but no documentation of API versioning scheme or a deprecation policy anywhere in docs/changelog; openapi spec probes 404 and no version headers or changelog entries reference API deprecations. Missing for 10: versioning scheme (e.g., v1/v2 paths), documented deprecation timeline/policy, changelog entries about breaking changes.

   • [claimed-docs] “Pylon's API can be used to programmatically access and take action on data within Pylon.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.usepylon.com/openapi.json, https://docs.usepylon.com/swagger.json, https://docs.usepylo…”
   • [probe] “PROBE runtime (recorded 2026-09-10): the public REST API at api.usepylon.com is live and cleanly auth-gated — a keyless GET returned HTTP 40…”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Pylon
   Sierra—none0/10

   No evidence describes bulk operations across many items (e.g., batch editing knowledge entries, mass workflow updates, or bulk conversation actions). The docs describe individual agent building, knowledge editing, and analytics tools, but nothing about performing actions at scale across many items simultaneously.

   ⚑ flag

   Pylon~partialprobed5/10

   Pylon's API (pylon-docs-31), MCP server (pylon-docs-34, pylon-probe-rt-1/3), and triggers/automations (pylon-docs-30) give AI agents and scripts programmatic read/update access to issues, accounts, and contacts, which could be used to script bulk changes, but no evidence describes a dedicated bulk-edit/bulk-action UI or batch-endpoint for acting on many items in one call. Missing for 10: an explicit bulk-update/bulk-action API or UI feature, and any documentation or demo of multi-item batch operations rather than single-record CRUD via API/MCP.

   • [claimed-docs] “Pylon's API can be used to programmatically access and take action on data within Pylon.”
   • [claimed-docs] “AI tools authenticate with your Pylon account via OAuth and can read and update issues, accounts, and contacts within the data you can see i…”
   • [claimed-docs] “Set up complex workflows and automations to execute your business processes”
   • [probe] “PROBE runtime (recorded 2026-09-10): Pylon's docs are agent-legible — https://docs.usepylon.com/pylon-docs/llms.txt serves a live '# Pylon' …”
   • [probe] “PROBE runtime (recorded 2026-09-10): the first-party MCP server is live — a keyless JSON-RPC initialize POST to https://mcp.usepylon.com ans…”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to Pylon
   Sierra~partialclaimed5/10

   Sierra supports building workflows/journeys and guardrail-based release governance (e.g., agent checks, split traffic, merge approval) which implies some rule-based triggering, and Ghostwriter/Agent Studio let users define step-by-step logic, but there's no explicit documentation of an event-driven 'if X happens, trigger Y automatically' rules engine for AI-native users to configure independently. missing for 10: explicit event-trigger/rules-engine documentation, API/SDK examples of automated action-on-event configuration, independent verification of this specific automation capability.

   • [claimed-docs] “Define step-by-step workflows for your agent from scratch, or use AI to instantly generate journeys from your existing operating procedures.”
   • [claimed-docs] “Agent Checks and Simulations catching problems proactively, merge approval workflows putting a person in the loop, and split traffic release…”
   • [claimed-docs] “Initiate workflows directly from a conversation—no tab-switching, no referencing instructions, no lost context.”

   proof ↗⚑ flag

   Pylon✓fullprobed8/10

   Pylon has a dedicated Triggers platform docs page describing setting up complex workflows/automations tied to business processes, plus webhooks for events and an API to programmatically act on data, enabling event-driven automated actions. This directly matches rule-based automation triggering actions on events, with runtime probes confirming the API/webhook infrastructure is live. Missing for 10: detailed trigger-rule syntax/examples, independent hands-on validation of complex trigger logic, and explicit AI-agent-specific trigger configuration beyond general workflow automation.

   • [claimed-docs] “Set up complex workflows and automations to execute your business processes”
   • [claimed-docs] “Pylon's API can be used to programmatically access and take action on data within Pylon.”
   • [claimed-docs] “Receive webhooks for various events happening in Pylon.”
   • [probe] “PROBE runtime (recorded 2026-09-10): the public REST API at api.usepylon.com is live and cleanly auth-gated — a keyless GET returned HTTP 40…”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round drawn
   Sierra—none0/10

   Sierra's evidence covers agent building, workflows, channels, analytics, and release governance, but nothing describes scheduling recurring jobs/workflows (e.g., cron-like triggers or timed automation runs) for AI-native users. Missing for 10: any documentation of scheduled/recurring job execution, trigger-based automation, or timer-based workflow runs.

   ⚑ flag

   Pylon—none0/10

   Pylon's docs mention 'triggers' for building automations and workflows (pylon-docs-30) plus API/webhooks for programmatic action, but nothing in the evidence pack specifies time-based or recurring scheduling (e.g., cron-like jobs, scheduled runs) as opposed to event-triggered automations. No mention of a scheduler, recurring job configuration, or interval-based execution.

   • [claimed-docs] “Set up complex workflows and automations to execute your business processes”
   • [claimed-docs] “Pylon's API can be used to programmatically access and take action on data within Pylon.”
   • [claimed-docs] “Receive webhooks for various events happening in Pylon.”

   proof ↗⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round to Sierra
   Sierra~partialprobed5/10

   Sierra's Agent SDK is described as 'journeys as code' with change tracking, and release-governance docs mention merge approval, agent checks/simulations, and split-traffic gradual rollouts — pointing to versioned, reviewable release workflows. However, no explicit rollback/revert mechanism is documented, and docs are login-gated so no independent confirmation exists. missing for 10: explicit rollback capability, independent/hands-on confirmation of version history and revert function.

   • [claimed-docs] “write customer journeys as code, track changes, and build sophisticated logic without giving up our development workflows”
   • [claimed-docs] “Agent Checks and Simulations catching problems proactively, merge approval workflows putting a person in the loop, and split traffic release…”
   • [probe] “PROBE runtime finding (recorded 2026-09-10): docs.sierra.ai has no agent-legible surface — /llms.txt resolves (HTTP 200) to the login SPA's …”

   proof ↗⚑ flag

   Pylon—none0/10

   Pylon has automations/triggers, agent runbooks, and issue logs for monitoring outcomes, but there is no evidence of version history, diffing, review workflows, or rollback capability for automations/triggers/skills/runbooks. Missing for 10: version history for triggers/runbooks, change review/approval process, rollback to a prior automation version.

   • [claimed-docs] “Set up complex workflows and automations to execute your business processes”
   • [claimed-docs] “Write instructions in natural language to help your AI Agent handle specific scenarios.”
   • [claimed-docs] “The issue log allows you to inspect all the issues the AI agent is associated with and inspect the outcomes and steps taken by the AI.”

   proof ↗⚑ flag

Channels

1. support leaderOne agent covers chat, email, and in-app, plus the channels my customers actually use — Slack, WhatsApp, social

   weight 2 · round drawn
   Sierra~partialclaimed5/10

   Sierra's docs confirm one agent can be built once and deployed across chat, phone, email, SMS, and messaging channels (sierra-docs-3), with channel-specific tuning like Voice Personas (sierra-docs-16) and even ChatGPT publishing (sierra-docs-17), showing broad omnichannel intent. However, none of the evidence explicitly names Slack, WhatsApp, or social media as supported channels, leaving the specific channels support leaders care about unconfirmed. Missing for 10: explicit documentation or independent confirmation that Slack, WhatsApp, and social platforms are supported channels, plus hands-on evidence of a single agent operating consistently across these specific channels.

   • [claimed-docs] “Build once and deploy across any channel—chat, phone, email, SMS, and messaging.”
   • [claimed-docs] “Use Voice Personas to design how your agent sounds, speaks, and connects, powered by a constellation of models tuned across 59 languages.”
   • [claimed-docs] “Publish to ChatGPT with one click: Make your agent available on ChatGPT with the push of a button, or via CI/CD.”

   proof ↗⚑ flag

   Pylon~partialclaimed5/10

   Pylon clearly supports one-agent omnichannel routing across chat, email, and in-app (chat widget, Slack, email integrations, omnichannel-support doc), but the evidence pack never explicitly documents WhatsApp or social-media channel integrations, which the story specifically calls out. missing for 10: explicit WhatsApp integration docs, explicit social media (e.g. Twitter/Instagram) channel support, and confirmation these route into the same unified agent workflow as chat/email/Slack.

   • [claimed-docs] “Pylon can support you wherever you interact with your customers.”
   • [claimed-docs] “Pylon watches for customer issues in connected customer Slack channels and automatically bundles together relevant messages and creates trac…”
   • [claimed-docs] “Connect your email addresses with Pylon to track and response to support requests over email.”
   • [claimed-docs] “Fully customizable colors, branding, and workflows * Multiple separate chat widgets with different branding and workflows”
   • [claimed-docs] “Support Agents can deflect customer questions using content, gather internal context for your team, and take action.”

   proof ↗⚑ flag

Languages

1. support leaderThe agent supports customers in many languages, even where my knowledge base exists only in English

   weight 2 · round to Sierra
   Sierra~partialclaimed6/10

   Sierra documents multilingual voice capability explicitly ('Voice Personas... tuned across 59 languages') and general multi-channel deployment, implying broad language coverage for customer-facing conversations. However, there is no direct evidence about handling a knowledge base that exists only in English—no mention of automatic translation/grounding of English-only content into other languages, so the specific claim in the story (KB gap bridging) is unevidenced. missing for 10: explicit documentation on cross-language grounding from English-only knowledge base, independent verification of multilingual quality beyond voice.

   • [claimed-docs] “Use Voice Personas to design how your agent sounds, speaks, and connects, powered by a constellation of models tuned across 59 languages.”
   • [claimed-docs] “Build once and deploy across any channel—chat, phone, email, SMS, and messaging.”

   proof ↗⚑ flag

   Pylon—none0/10

   No evidence in the pack mentions multilingual support, language detection, translation of knowledge base content, or the AI agent responding in customer languages other than English; docs cover agent building, knowledge base, and integrations but never address language coverage.

   ⚑ flag

Voice

1. support leaderThe agent handles phone calls — speech in, speech out — with the same knowledge and actions as chat

   weight 2 · round to Sierra
   Sierra✓fullclaimed8/10

   Sierra explicitly markets voice as a first-class channel with the same agent logic/knowledge as chat ('Build once and deploy across any channel—chat, phone, email, SMS'), plus voice-specific features like Voice Personas across 59 languages, replacing IVR with empathetic voice agents, and phone payments without IVR handoff. Missing for 10: independent/hands-on verification that voice calls actually share identical knowledge/actions with chat in production (only vendor docs, no community confirmation of phone-specific parity).

   • [claimed-docs] “Build once and deploy across any channel—chat, phone, email, SMS, and messaging.”
   • [claimed-docs] “Use Voice Personas to design how your agent sounds, speaks, and connects, powered by a constellation of models tuned across 59 languages.”
   • [claimed-docs] “Collect card and ACH payments entirely over the phone with no IVR handoff.”
   • [claimed-docs] “Replace rigid IVR menus with an empathetic voice agent that understands customer context”
   • [claimed-docs] “Guide care reps through next steps—whether in chat or on calls—automatically capturing and updating customer context as work progresses.”

   proof ↗⚑ flag

   Pylon—none0/10

   Pylon's evidence covers omnichannel support (chat, email, Slack), AI support agents, knowledge bases, and MCP/API integrations, but nothing describes voice/phone call handling with speech-in/speech-out capability. The omnichannel doc (pylon-docs-24) is generic and doesn't mention telephony or voice at all.

   ⚑ flag

Handoff

1. support leaderWhen the agent escalates, the human gets the full conversation, a summary, and collected details — the customer never repeats themselves

   weight 3 · round to Sierra
   Sierra~partialclaimed6/10

   Sierra's Live Assist product explicitly targets escalation handoff, claiming reps are guided through next steps with automatically captured and updated customer context and can pick up conversations 'no tab-switching, no referencing instructions, no lost context' — directly addressing the no-repeat-yourself goal. However, there's no explicit mention of a generated conversation summary artifact or hands-on/independent confirmation that reps actually receive full transcript + summary + collected details in practice. Missing for 10: explicit summary-generation evidence, independent/hands-on validation of the handoff experience, confirmation reps see full conversation history alongside context.

   • [claimed-docs] “Guide care reps through next steps—whether in chat or on calls—automatically capturing and updating customer context as work progresses.”
   • [claimed-docs] “Initiate workflows directly from a conversation—no tab-switching, no referencing instructions, no lost context.”

   proof ↗⚑ flag

   Pylon~partialclaimed5/10

   Pylon's AI agent operates within the same ticket/issue thread that a human later takes over, and the issue log records the agent's steps and outcomes (pylon-docs-19), while the agent explicitly gathers internal context for the team (pylon-docs-13) and elsewhere Pylon auto-summarizes conversations for CRM sync (pylon-docs-33), suggesting summarization capability exists. However, there is no explicit documentation of an escalation-specific handoff package (conversation + AI summary + structured collected details) being automatically attached to a ticket when an agent escalates to a human. missing for 10: explicit escalation-handoff feature docs, evidence the summary/details are surfaced to the human agent at hand-off time, confirmation customer doesn't need to repeat themselves.

   • [claimed-docs] “Support Agents can deflect customer questions using content, gather internal context for your team, and take action.”
   • [claimed-docs] “The issue log allows you to inspect all the issues the AI agent is associated with and inspect the outcomes and steps taken by the AI.”
   • [claimed-docs] “Every day, Pylon scans your customer channels for conversations that were had. For every conversation, we use AI to summarize the details, a…”
   • [claimed-docs] “You can assign them to issues you'd like them to take a look at. This means you have full control of which customer issues your AI agent int…”

   proof ↗⚑ flag

Rules

1. support ops leadI configure when the agent must hand off — by topic, sentiment, customer tier, or explicit request — and it reliably obeys

   weight 2 · round to Pylon
   Sierra~partialclaimed3/10

   Sierra's docs mention configurable guardrails, workflows, and a 'Live Assist' human-handoff product, implying some escalation mechanism exists, but no evidence describes explicit configuration of handoff triggers by topic, sentiment, customer tier, or explicit customer request, nor proof of reliable adherence to such rules. Missing for 10: documentation of specific trigger types (topic/sentiment/tier/request), configuration UI/API for these rules, and evidence (first-party or independent) that the agent reliably obeys them in production.

   • [claimed-docs] “Build or modify agents by describing how you want them to behave. Update workflows, systems integrations, guardrails, tone, and style with s…”
   • [claimed-docs] “Guide care reps through next steps—whether in chat or on calls—automatically capturing and updating customer context as work progresses.”
   • [claimed-docs] “Agent Checks and Simulations catching problems proactively, merge approval workflows putting a person in the loop, and split traffic release…”
   • [claimed-docs] “Define step-by-step workflows for your agent from scratch, or use AI to instantly generate journeys from your existing operating procedures.”

   proof ↗⚑ flag

   Pylon~partialclaimed4/10

   Pylon's Support Agent framework lets admins write natural-language runbooks for 'specific scenarios' and assign only certain issues to the agent, plus monitor outcomes via the issue log — this could be used to script handoff conditions, but there is no explicit documentation of built-in triggers for topic, sentiment, or customer-tier-based escalation, nor any evidence about reliability/guardrails ensuring the agent 'reliably obeys' handoff rules. Missing for 10: dedicated sentiment/tier/topic escalation configuration UI, explicit-request handoff trigger documentation, and evidence (docs or hands-on) of enforcement reliability.

   • [claimed-docs] “Write instructions in natural language to help your AI Agent handle specific scenarios.”
   • [claimed-docs] “You can assign them to issues you'd like them to take a look at. This means you have full control of which customer issues your AI agent int…”
   • [claimed-docs] “The issue log allows you to inspect all the issues the AI agent is associated with and inspect the outcomes and steps taken by the AI.”
   • [claimed-docs] “Skills are reusable instructions that teach Pylon agents how to handle specific workflows consistently.”
   • [claimed-docs] “Support Agents can deflect customer questions using content, gather internal context for your team, and take action.”

   proof ↗⚑ flag

Hallucination

1. ai-native userGuardrails stop the agent from inventing policies, prices, or promises — off-knowledge questions get a safe decline, not a guess

   weight 3 · round to Sierra
   Sierra~partialclaimed4/10

   Sierra's docs mention grounding agents in knowledge/FAQs/policies (sierra-docs-7) and general 'guardrails' as a configurable behavior via Ghostwriter (sierra-docs-8), plus release-governance guardrails like Agent Checks/Simulations (sierra-docs-18) and visibility into reasoning/knowledge sources (sierra-docs-13). However, no evidence explicitly describes a 'safe decline' mechanism for off-knowledge questions or confirms the agent won't invent prices/policies rather than guess. Missing for 10: explicit documentation or hands-on proof of decline-on-unknown behavior, third-party validation that hallucinated policies/prices are prevented, and detail on the unexplored 'trust-and-reliability' product page.

   • [claimed-docs] “View, manage, and edit knowledge data such as Help Center content, FAQs, and policies that ground your agent.”
   • [claimed-docs] “Build or modify agents by describing how you want them to behave. Update workflows, systems integrations, guardrails, tone, and style with s…”
   • [claimed-docs] “Understand the reasoning behind every agent action or answer—view knowledge sources, systems accessed, and more.”
   • [claimed-docs] “Agent Checks and Simulations catching problems proactively, merge approval workflows putting a person in the loop, and split traffic release…”

   proof ↗⚑ flag

   Pylon~partialclaimed3/10

   Pylon's Support Agent is grounded in a curated knowledge base and runbooks/skills that define scenario-specific instructions, which implicitly limits it to trained content rather than free invention, and 'deflect using content' suggests answers are content-bound. However, there is no explicit documentation of a safe-decline behavior for off-knowledge questions, confidence thresholds, or anti-hallucination guardrails around prices/policies. Missing for 10: explicit safe-decline/escalation mechanism for unknown questions, documentation of hallucination prevention, and evidence of guardrails specifically around prices/policies/promises.

   • [claimed-docs] “Support Agents can deflect customer questions using content, gather internal context for your team, and take action.”
   • [claimed-docs] “Write instructions in natural language to help your AI Agent handle specific scenarios.”
   • [claimed-docs] “Skills are reusable instructions that teach Pylon agents how to handle specific workflows consistently.”
   • [claimed-docs] “A dedicated place to host your support content, including question-and-answer style articles, internal runbooks, and more -- all supercharge…”

   proof ↗⚑ flag

Supervision

1. support ops leadLaunch in a supervised mode where the agent drafts replies for human approval before anything reaches a customer

   weight 2 · round drawn
   Sierra—none0/10

   Sierra's evidence covers release governance (merge-approval for deploying agent changes) and Live Assist (guiding human reps in real time), but neither describes a mode where the agent drafts individual customer replies that a human must approve before they are sent. No docs or community evidence mention message-level human-in-the-loop approval for live customer conversations.

   • [claimed-docs] “Agent Checks and Simulations catching problems proactively, merge approval workflows putting a person in the loop, and split traffic release…”
   • [claimed-docs] “Guide care reps through next steps—whether in chat or on calls—automatically capturing and updating customer context as work progresses.”
   • [claimed-docs] “Initiate workflows directly from a conversation—no tab-switching, no referencing instructions, no lost context.”

   proof ↗⚑ flag

   Pylon—none0/10

   Pylon's docs describe agent deployment controls (assigning which issues the agent handles, testing/simulating responses, and an issue log for post-hoc review) but nowhere document a supervised 'draft-for-human-approval-before-send' mode; the language instead emphasizes agents 'taking action' autonomously once deployed.

   • [claimed-docs] “Support Agents can deflect customer questions using content, gather internal context for your team, and take action.”
   • [claimed-docs] “You can assign them to issues you'd like them to take a look at. This means you have full control of which customer issues your AI agent int…”
   • [claimed-docs] “The issue log allows you to inspect all the issues the AI agent is associated with and inspect the outcomes and steps taken by the AI.”
   • [claimed-docs] “Simulate an interaction with the AI and see how it handles different questions.”

   proof ↗⚑ flag

Topic controls

1. support ops leadI mark topics as human-only — legal threats, cancellations, security — and the agent never freelances on them

   weight 2 · round to Pylon
   Sierra—none0/10

   Sierra's guardrail evidence covers release governance (Agent Checks, Simulations, merge approval, split traffic) and general knowledge/workflow editing, but nothing describes topic-level human-only flags for categories like legal threats, cancellations, or security that the agent is barred from handling. Live-assist shows human+AI collaboration but not a mechanism to designate certain topics as strictly human-only with enforced escalation.

   • [claimed-docs] “Agent Checks and Simulations catching problems proactively, merge approval workflows putting a person in the loop, and split traffic release…”
   • [claimed-docs] “Guide care reps through next steps—whether in chat or on calls—automatically capturing and updating customer context as work progresses.”

   proof ↗⚑ flag

   Pylon~partialclaimed4/10

   Pylon documents assignment-based control—agents only interact with issues explicitly assigned to them, giving admins 'full control of which customer issues your AI agent interacts with'—which could be used to withhold sensitive categories from the agent, and runbooks/skills let you write natural-language handling instructions. But there is no documented feature for marking specific topics (legal threats, cancellations, security) as categorically human-only or any guardrail that blocks the agent from acting on flagged topic types. Missing for 10: an explicit topic/category exclusion or block-list mechanism, enforcement guarantees against agent 'freelancing' on excluded topics, and independent verification that assignment-based control reliably prevents agent action on sensitive categories.

   • [claimed-docs] “You can assign them to issues you'd like them to take a look at. This means you have full control of which customer issues your AI agent int…”
   • [claimed-docs] “Write instructions in natural language to help your AI Agent handle specific scenarios.”
   • [claimed-docs] “The issue log allows you to inspect all the issues the AI agent is associated with and inspect the outcomes and steps taken by the AI.”
   • [claimed-docs] “Skills are reusable instructions that teach Pylon agents how to handle specific workflows consistently.”

   proof ↗⚑ flag

Analytics

1. support leaderDashboards show resolution rate, CSAT, handoff rate, and cost per resolution — the numbers I report to my exec team

   weight 3 · round drawn
   Sierra~partialclaimed5/10

   Sierra's Insights and Explorer products provide dashboards/reports with drill-down and natural-language trend analysis (sierra-docs-10, sierra-docs-11, sierra-docs-12, sierra-docs-13), showing the platform surfaces analytics for support leaders, but no evidence specifies the exact metrics named in the story (resolution rate, CSAT, handoff rate, cost per resolution). Missing for 10: explicit documentation naming these specific KPIs, screenshots/examples of the actual dashboard metrics, and independent confirmation that leaders use it for exec-level reporting.

   • [claimed-docs] “Ask any question about your customer experience in natural language, and Explorer identifies the answer across thousands of real conversatio…”
   • [claimed-docs] “Explorer automatically delivers a weekly briefing on key trends, emerging issues, and recommendations — without you having to ask.”
   • [claimed-docs] “Click any data point on a report to launch Explorer and instantly investigate what's driving that trend.”
   • [claimed-docs] “Understand the reasoning behind every agent action or answer—view knowledge sources, systems accessed, and more.”

   proof ↗⚑ flag

   Pylon~partialclaimed5/10

   Pylon docs confirm built-in analytics dashboards with 'common support metrics' filterable by attributes, supporting general resolution/ticket reporting, but no evidence explicitly names CSAT, handoff rate, or cost-per-resolution as tracked metrics. missing for 10: explicit documentation of CSAT scoring, handoff-rate metric, and cost-per-resolution calculation in dashboards.

   • [claimed-docs] “Pylon comes with a number of default analytics dashboards with common support metrics. All dashboards are filterable by a wide selection of …”

   proof ↗⚑ flag

Insights

1. support leaderThe platform clusters conversations by topic and surfaces emerging product issues before they spike ticket volume

   weight 1 · round to Sierra
   Sierra~partialclaimed6/10

   Sierra's Explorer product explicitly surfaces trends and emerging issues via natural-language querying of conversations and proactive weekly briefings, and Insights lets users drill into data points—directly matching the 'surface emerging issues before they spike' theme. However, evidence doesn't explicitly confirm topic clustering methodology, predictive spike detection, or independent/hands-on validation of this specific capability. Missing for 10: explicit description of topic-clustering mechanics, evidence of predictive alerting before ticket-volume spikes (vs. reactive weekly summaries), and independent/customer corroboration of this specific insights capability.

   • [claimed-docs] “Ask any question about your customer experience in natural language, and Explorer identifies the answer across thousands of real conversatio…”
   • [claimed-docs] “Explorer automatically delivers a weekly briefing on key trends, emerging issues, and recommendations — without you having to ask.”
   • [claimed-docs] “Click any data point on a report to launch Explorer and instantly investigate what's driving that trend.”

   proof ↗⚑ flag

   Pylon—none0/10

   Evidence shows generic analytics dashboards with filterable metrics (pylon-docs-29) and AI summarization of Slack conversations synced to Salesforce (pylon-docs-33), but nothing describes topic clustering of conversations or proactive surfacing of emerging product issues before ticket volume spikes.

   • [claimed-docs] “Pylon comes with a number of default analytics dashboards with common support metrics. All dashboards are filterable by a wide selection of …”
   • [claimed-docs] “Every day, Pylon scans your customer channels for conversations that were had. For every conversation, we use AI to summarize the details, a…”

   proof ↗⚑ flag

Helpdesk

1. developerThe agent runs inside my existing helpdesk — Zendesk, Salesforce, Intercom — or standalone, syncing tickets and context both ways

   weight 3 · round to Pylon
   Sierra—none0/10

   No evidence pack citation mentions Zendesk, Salesforce, Intercom, or bidirectional ticket/context syncing with existing helpdesk platforms; only generic 'systems integrations' and 'internal APIs' are referenced without naming any helpdesk system or describing two-way ticket sync.

   ⚑ flag

   Pylon~partialclaimed3/10

   Evidence shows a one-way sync of AI-summarized conversations into Salesforce Accounts and general 'omnichannel' support language, but there is no evidence of the AI agent embedding inside Zendesk or Intercom, nor of two-way ticket/context syncing with any named helpdesk — Pylon's docs otherwise position it as the standalone system of record. Missing for 10: Zendesk/Intercom integration evidence, bidirectional ticket sync (not just one-way conversation summaries), and confirmation the agent itself operates 'inside' another helpdesk's UI/workflow.

   • [claimed-docs] “Every day, Pylon scans your customer channels for conversations that were had. For every conversation, we use AI to summarize the details, a…”
   • [claimed-docs] “Pylon can support you wherever you interact with your customers.”
   • [claimed-docs] “Using a combination of Webhooks, API, and/or Pylon MCP, you can connect your agent to ticketing workflows in Pylon.”

   proof ↗⚑ flag

Freshness

1. support ops leadKnowledge stays current automatically — the agent re-syncs sources on a schedule or on change, not via manual re-uploads

   weight 2 · round to Pylon
   Sierra—none0/10

   Evidence shows Sierra lets teams manually 'view, manage, and edit knowledge data' (sierra-docs-7) but nothing describes automatic re-syncing of knowledge sources on a schedule or on-change detection; docs are login-gated so no further detail is visible.

   • [claimed-docs] “View, manage, and edit knowledge data such as Help Center content, FAQs, and policies that ground your agent.”
   • [probe] “PROBE runtime finding (recorded 2026-09-10): docs.sierra.ai has no agent-legible surface — /llms.txt resolves (HTTP 200) to the login SPA's …”

   proof ↗⚑ flag

   Pylon~partialclaimed4/10

   Pylon's knowledge base and training-data docs mention connecting external knowledge sources (pylon-docs-22, pylon-docs-23), and product-data sync can be disabled mid-run (pylon-docs-8), implying some sync mechanism exists, but there is no documentation of a re-sync schedule, change-detection triggers, or automatic re-indexing cadence for knowledge sources. missing for 10: explicit scheduling/frequency controls for knowledge source re-sync, change-detection or webhook-triggered re-sync of external knowledge, and any evidence contradicting or confirming this works hands-on.

   • [claimed-docs] “Make them even more powerful by connecting any external sources of knowledge you have!”
   • [claimed-docs] “A dedicated place to host your support content, including question-and-answer style articles, internal runbooks, and more -- all supercharge…”
   • [claimed-docs] “Disabling a product-data sync now stops any run already in progress”

   proof ↗⚑ flag

Gaps

1. support ops leadThe platform surfaces knowledge gaps and conflicting content that cause the agent to miss or fumble questions

   weight 1 · round to Sierra
   Sierra~partialclaimed5/10

   Sierra's Explorer and Insights products surface emerging issues, trends, and the reasoning/knowledge sources behind agent answers, and Agent Studio lets teams view/manage knowledge content (FAQs, policies), which could help surface gaps — but no evidence explicitly describes detecting knowledge gaps or conflicting content as a distinct feature. Missing for 10: explicit conflicting-content/contradiction detection, explicit 'knowledge gap' flagging, and independent/hands-on confirmation that Explorer or Insights actually pinpoints such gaps rather than general conversation trends.

   • [claimed-docs] “View, manage, and edit knowledge data such as Help Center content, FAQs, and policies that ground your agent.”
   • [claimed-docs] “Ask any question about your customer experience in natural language, and Explorer identifies the answer across thousands of real conversatio…”
   • [claimed-docs] “Explorer automatically delivers a weekly briefing on key trends, emerging issues, and recommendations — without you having to ask.”
   • [claimed-docs] “Click any data point on a report to launch Explorer and instantly investigate what's driving that trend.”
   • [claimed-docs] “Understand the reasoning behind every agent action or answer—view knowledge sources, systems accessed, and more.”

   proof ↗⚑ flag

   Pylon—none0/10

   Pylon documents training-data ingestion, a knowledge base, agent simulation/testing, and issue logs for reviewing agent outcomes, but nothing in the evidence describes a feature that proactively surfaces knowledge gaps or flags conflicting content driving agent misses. This is a fair axis for a support-agent platform (comparable products ship content-quality/coverage analytics), so absence of evidence yields 'none' rather than 'na'.

   • [claimed-docs] “The issue log allows you to inspect all the issues the AI agent is associated with and inspect the outcomes and steps taken by the AI.”
   • [claimed-docs] “Simulate an interaction with the AI and see how it handles different questions.”
   • [claimed-docs] “Make them even more powerful by connecting any external sources of knowledge you have!”
   • [claimed-docs] “A dedicated place to host your support content, including question-and-answer style articles, internal runbooks, and more -- all supercharge…”

   proof ↗⚑ flag

Grounding

1. ai-native userEvery answer is grounded in my own content and shows which article or source it drew from

   weight 3 · round to Sierra
   Sierra✓fullclaimed7/10

   Sierra explicitly supports grounding agent answers in customer-owned knowledge (Help Center content, FAQs, policies) via Agent Studio, and Insights lets teams 'understand the reasoning behind every agent action or answer—view knowledge sources, systems accessed' which directly maps to showing which source an answer drew from. Missing for 10: independent/hands-on verification that end-user-facing answers visibly cite specific articles, and detail on citation UX rather than just admin-side reasoning traces.

   • [claimed-docs] “View, manage, and edit knowledge data such as Help Center content, FAQs, and policies that ground your agent.”
   • [claimed-docs] “Understand the reasoning behind every agent action or answer—view knowledge sources, systems accessed, and more.”
   • [claimed-docs] “Ask any question about your customer experience in natural language, and Explorer identifies the answer across thousands of real conversatio…”

   proof ↗⚑ flag

   Pylon~partialclaimed5/10

   Pylon's Support Agents are documented to answer/deflect using a dedicated Knowledge Base and connected external knowledge sources (pylon-docs-13, pylon-docs-22, pylon-docs-23), which supports grounding in the user's own content, but there is no evidence that responses explicitly cite or display which specific article/source was used to generate an answer. missing for 10: explicit citation/source-attribution UI in agent answers, evidence of per-answer source linking, independent confirmation of grounding accuracy.

   • [claimed-docs] “Support Agents can deflect customer questions using content, gather internal context for your team, and take action.”
   • [claimed-docs] “Make them even more powerful by connecting any external sources of knowledge you have!”
   • [claimed-docs] “A dedicated place to host your support content, including question-and-answer style articles, internal runbooks, and more -- all supercharge…”
   • [claimed-docs] “Skills are reusable instructions that teach Pylon agents how to handle specific workflows consistently.”

   proof ↗⚑ flag

Ingestion

1. support ops leadThe agent ingests my help center, docs, past tickets, and internal wikis as knowledge sources without manual re-authoring

   weight 3 · round to Sierra
   Sierra~partialprobed6/10

   Sierra's docs explicitly claim agent-studio lets teams 'view, manage, and edit knowledge data such as Help Center content, FAQs, and policies' and Ghostwriter can ingest SOPs, transcripts, and audio interviews to build journeys, supporting knowledge ingestion beyond just help center docs. However, there's no explicit evidence of ingesting past tickets or internal wikis specifically, and docs/reference material is login-gated so independent verification of breadth of source-type ingestion is limited. Missing for 10: documented support for tickets/wiki ingestion specifically, and independent/hands-on confirmation of the ingestion workflow beyond marketing copy.

   • [claimed-docs] “View, manage, and edit knowledge data such as Help Center content, FAQs, and policies that ground your agent.”
   • [claimed-docs] “Upload SOPs, raw transcripts, or even audio interviews of subject matter experts to create customer journeys from scratch.”
   • [probe] “PROBE runtime finding (recorded 2026-09-10): docs.sierra.ai has no agent-legible surface — /llms.txt resolves (HTTP 200) to the login SPA's …”

   proof ↗⚑ flag

   Pylon~partialclaimed5/10

   Pylon documents a 'training-data' page for connecting external knowledge sources and a Knowledge Base feature for hosting support content and runbooks, suggesting some support for knowledge grounding, but there is no concrete evidence of automated ingestion of help center content, past tickets, or internal wikis without manual re-authoring. Missing for 10: explicit documentation of connectors for help center/docs platforms, wiki integrations (Confluence/Notion), automatic ticket-history ingestion, and any hands-on evidence confirming these sources sync without manual re-entry.

   • [claimed-docs] “Make them even more powerful by connecting any external sources of knowledge you have!”
   • [claimed-docs] “A dedicated place to host your support content, including question-and-answer style articles, internal runbooks, and more -- all supercharge…”
   • [claimed-docs] “Support Agents can deflect customer questions using content, gather internal context for your team, and take action.”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Pylon
   Sierra—none0/10

   Sierra markets an Agent SDK for code-based building alongside a separate no-code Agent Studio, but there is no evidence of API/UI feature parity — no public OpenAPI/swagger spec was found (404s), and the SDK reference docs themselves are login-gated to contracted customers, meaning even documented API scope can't be verified as matching UI capabilities.

   • [claimed-docs] “write customer journeys as code, track changes, and build sophisticated logic without giving up our development workflows”
   • [claimed-docs] “Empower teams to build and manage agents—no code required”
   • [probe] “PROBE openapi: all candidate paths 404 (https://sierra.ai/openapi.json, https://sierra.ai/swagger.json, https://sierra.ai/api/openapi.json, …”
   • [probe] “PROBE runtime finding (recorded 2026-09-10): docs.sierra.ai has no agent-legible surface — /llms.txt resolves (HTTP 200) to the login SPA's …”

   proof ↗⚑ flag

   Pylon~partialprobed5/10

   Pylon documents a public REST API and webhooks for programmatically accessing/acting on issues, accounts, and contacts (pylon-docs-31, pylon-docs-32, pylon-docs-4), plus an MCP server that lets AI tools read/update the same core objects (pylon-docs-34, pylon-probe-rt-3). However, there's no evidence the API/MCP surface covers the full breadth of UI capabilities shown in the changelog (custom object forms, analytics dashboards, chat widget branding/customization, knowledge base authoring, triggers/automation config) — these appear UI-only in the evidence pack. Missing for 10: documented API/MCP parity for analytics dashboards, chat widget customization, knowledge base management, and workflow/trigger configuration.

   • [claimed-docs] “Pylon's API can be used to programmatically access and take action on data within Pylon.”
   • [claimed-docs] “Receive webhooks for various events happening in Pylon.”
   • [claimed-docs] “Added a GET /task-statuses API endpoint for enumerating configured task statuses”
   • [claimed-docs] “AI tools authenticate with your Pylon account via OAuth and can read and update issues, accounts, and contacts within the data you can see i…”
   • [probe] “PROBE runtime (recorded 2026-09-10): the public REST API at api.usepylon.com is live and cleanly auth-gated — a keyless GET returned HTTP 40…”
   • [probe] “PROBE runtime (recorded 2026-09-10): the first-party MCP server is live — a keyless JSON-RPC initialize POST to https://mcp.usepylon.com ans…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round drawn
   Sierra—none0/10

   No evidence of any data export or portability feature for customer journeys, knowledge data, or conversation logs in open formats; probes even show docs/API surfaces are gated or unavailable (404s, login-gated /llms.txt). Missing for 10: any export functionality, data portability documentation, or open-format data dump capability.

   • [probe] “PROBE llms.txt: HTTP 404 at https://sierra.ai/llms.txt”
   • [probe] “PROBE openapi: all candidate paths 404 (https://sierra.ai/openapi.json, https://sierra.ai/swagger.json, https://sierra.ai/api/openapi.json, …”
   • [probe] “PROBE runtime finding (recorded 2026-09-10): docs.sierra.ai has no agent-legible surface — /llms.txt resolves (HTTP 200) to the login SPA's …”

   proof ↗⚑ flag

   Pylon—none0/10

   Evidence shows a REST API and webhooks for programmatic access to issues/accounts/contacts, but nothing documents a bulk data-export feature, open-format dump, or data-portability/account-closure workflow that would let a user extract all their data and leave.

   • [claimed-docs] “Pylon's API can be used to programmatically access and take action on data within Pylon.”
   • [claimed-docs] “Receive webhooks for various events happening in Pylon.”
   • [probe] “PROBE runtime (recorded 2026-09-10): the public REST API at api.usepylon.com is live and cleanly auth-gated — a keyless GET returned HTTP 40…”

   proof ↗⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round drawn
   Sierra—none0/10

   Sierra is presented purely as a closed commercial SaaS platform (Agent SDK, Studio, Ghostwriter, Voice, etc.) with no evidence of an open-source license for the product itself; the only open artifact found is the unrelated tau-bench research benchmark repo, not Sierra's product source code, and docs/API surfaces are login-gated or 404. Missing for 10: any open-license repository for Sierra's actual product code, license file, or public source release.

   • [probe] “PROBE runtime (recorded 2026-09-10): Sierra's agent benchmark is genuinely open — the official sierra-research/tau-bench repository README f…”
   • [probe] “PROBE runtime finding (recorded 2026-09-10): docs.sierra.ai has no agent-legible surface — /llms.txt resolves (HTTP 200) to the login SPA's …”
   • [probe] “PROBE llms.txt: HTTP 404 at https://sierra.ai/llms.txt”
   • [probe] “PROBE openapi: all candidate paths 404 (https://sierra.ai/openapi.json, https://sierra.ai/swagger.json, https://sierra.ai/api/openapi.json, …”

   proof ↗⚑ flag

   Pylon—none0/10

   Pylon is a closed-source SaaS product; no evidence of any open-source license or public source repository is present in the pack—only product docs, API/MCP references, and security/compliance pages. missing for 10: any open-source license, public source code repository, or licensing statement.

   ⚑ flag

Pricing

1. support leaderPricing is outcome-based and published — I pay per resolution with caps and controls, not an opaque enterprise quote

   weight 2 · round drawn
   Sierra—none0/10

   No evidence pack item mentions pricing, resolution-based billing, caps, or published rates; docs and probes cover product features and access gating only. Sierra is widely known anecdotally for outcome-based pricing but nothing in this evidence pack substantiates published, capped, self-serve pricing terms.

   ⚑ flag

   Pylon—none0/10

   No evidence anywhere in the pack of outcome-based or per-resolution pricing, published pricing tiers, caps, or usage-based billing controls; Pylon's documentation covers product features, agents, and APIs but is silent on pricing model or billing transparency.

   ⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round drawn
   Sierra—none0/10

   No evidence pack item addresses data residency, region selection, or storage location controls; the docs cover agent building, workflows, and channels but not data governance/residency options. missing for 10: any mention of data region selection, residency guarantees, or storage location controls.

   ⚑ flag

   Pylon—none0/10

   No evidence anywhere in the pack mentions data residency, region selection, or storage location options; only compliance/certification via Vanta is mentioned, which does not address data residency choice. Missing for 10: any mention of regional data centers, residency options, or ability to select storage location.

   • [claimed-docs] “Our compliance and certification are managed by Vanta and our incredible audit partners.”

   proof ↗⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   Sierra—none0/10

   No evidence in the pack addresses data-training opt-out or AI model training policies; Sierra's docs focus on product features (agent building, analytics, channels) with no privacy/data-use policy statements provided.

   ⚑ flag

   Pylon—none0/10

   Evidence pack mentions Pylon's compliance is managed via Vanta (SOC2-type certification) but there is no mention of any AI-training data opt-out, data usage policy for model training, or controls letting users prevent their data from being used to train AI models. Missing for 10: explicit AI-training opt-out/data-usage policy, contractual or product-level control preventing training use, and any documentation addressing this specific privacy concern.

   • [claimed-docs] “Our compliance and certification are managed by Vanta and our incredible audit partners.”

   proof ↗⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round drawn
   Sierra—none0/10

   No evidence pack item addresses data retention controls, deletion policies, or user-facing data lifecycle management; docs cover agent building, analytics, and channels but never mention retention/deletion settings, and docs.sierra.ai is login-gated so no public verification exists.

   ⚑ flag

   Pylon—none0/10

   The evidence pack shows security/compliance certification via Vanta and general data handling but contains no documentation of data retention periods, deletion policies, or user/admin controls to delete or export data. Missing for 10: retention policy documentation, deletion/export controls, data lifecycle settings.

   • [claimed-docs] “Our compliance and certification are managed by Vanta and our incredible audit partners.”

   proof ↗⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Sierra—none0/10

   No evidence pack item discusses telemetry, usage tracking, opt-out controls, or privacy settings for AI-native/developer users; docs focus on product features and docs are gated behind login. Missing for 10: any documentation of telemetry collection practices, opt-out mechanism, or privacy controls.

   ⚑ flag

   Pylon—none0/10

   No evidence Pylon offers any telemetry opt-out or usage-tracking controls; evidence only covers compliance certification (Vanta) and product features, not telemetry settings.

   ⚑ flag

Personalization

1. support leaderAnswers use the customer's live data — plan, order status, account history — not just generic help articles

   weight 2 · round to Sierra
   Sierra~partialcommunity7/10

   Sierra's docs and community evidence show the agent integrates with customers' internal APIs and systems (not just static knowledge), with Insights explicitly noting 'systems accessed' during agent actions, and Live Assist capturing live customer context. However, no evidence explicitly confirms real-time retrieval of specifics like plan, order status, or account history. missing for 10: explicit documented example of pulling live order/plan/account data, independent verification beyond one HN anecdote.

   • [claimed-docs] “Understand and rapidly adjust agent behavior by inspecting API calls, logic traces, and more.”
   • [claimed-docs] “Understand the reasoning behind every agent action or answer—view knowledge sources, systems accessed, and more.”
   • [claimed-docs] “Guide care reps through next steps—whether in chat or on calls—automatically capturing and updating customer context as work progresses.”
   • [community] “Having seen this set up for a customer, what is interesting is how it works with internal APIs for customers. It's all point and click in a …”

   proof ↗⚑ flag

   Pylon~partialclaimed6/10

   Pylon documents product-data syncs, Account Fields, custom objects, and Connectors/Custom API tools/MCP that let AI Support Agents pull in live account/product data (e.g., pylon-docs-8, pylon-docs-21, pylon-docs-22, pylon-docs-33) rather than relying solely on the knowledge base, and Skills/Runbooks let agents act on that data during resolution (pylon-docs-13, pylon-docs-20). However there's no first-party or independent case study showing an actual generated answer citing live plan/order-status/account-history data in a real resolution. Missing for 10: concrete example/transcript of an AI answer referencing live order status or plan data, and independent verification that synced data is actually surfaced in agent responses rather than just accessible to admins.

   • [claimed-docs] “Disabling a product-data sync now stops any run already in progress”
   • [claimed-docs] “If a system does not provide an MCP server, an administrator can define a Custom API tool that agents can use to hit internal or third party…”
   • [claimed-docs] “Make them even more powerful by connecting any external sources of knowledge you have!”
   • [claimed-docs] “Every day, Pylon scans your customer channels for conversations that were had. For every conversation, we use AI to summarize the details, a…”
   • [claimed-docs] “Support Agents can deflect customer questions using content, gather internal context for your team, and take action.”
   • [claimed-docs] “Skills are reusable instructions that teach Pylon agents how to handle specific workflows consistently.”
   • [claimed-docs] “AI tools authenticate with your Pylon account via OAuth and can read and update issues, accounts, and contacts within the data you can see i…”

   proof ↗⚑ flag

Reasoning

1. support leaderThe agent asks clarifying questions and works through multi-step troubleshooting instead of dumping one canned answer

   weight 2 · round to Sierra
   Sierra~partialcommunity6/10

   Sierra's docs describe building step-by-step, multi-step 'customer journeys' and workflows (sierra-docs-6, sierra-docs-9) rather than single canned replies, and community commentary confirms agents are configured to work through processes with internal APIs (sierra-comm-2). However, no evidence explicitly demonstrates the agent proactively asking clarifying questions or a hands-on troubleshooting transcript showing multi-turn dialogue in practice. Missing for 10: concrete transcript/demo of clarifying-question behavior, independent evaluation of troubleshooting depth, and explicit mention of clarification-seeking as a designed capability.

   • [claimed-docs] “Define step-by-step workflows for your agent from scratch, or use AI to instantly generate journeys from your existing operating procedures.”
   • [claimed-docs] “Upload SOPs, raw transcripts, or even audio interviews of subject matter experts to create customer journeys from scratch.”
   • [community] “Having seen this set up for a customer, what is interesting is how it works with internal APIs for customers. It's all point and click in a …”

   proof ↗⚑ flag

   Pylon~partialclaimed5/10

   Pylon's Support Agent docs describe multi-step behavior—runbooks with natural-language instructions for specific scenarios, skills as reusable workflow instructions, gathering internal context, and taking action—plus an issue log showing 'outcomes and steps taken by the AI' (pylon-docs-13, pylon-docs-15, pylon-docs-20, pylon-docs-19). However, there is no explicit documentation that the agent proactively asks clarifying questions to the customer rather than issuing a single canned reply; the story's core 'asks clarifying questions' behavior is only implied, not confirmed. Missing for 10: explicit product documentation or example of the agent asking clarifying follow-up questions mid-conversation, and independent/hands-on evidence of multi-turn troubleshooting flows in practice.

   • [claimed-docs] “Support Agents can deflect customer questions using content, gather internal context for your team, and take action.”
   • [claimed-docs] “Write instructions in natural language to help your AI Agent handle specific scenarios.”
   • [claimed-docs] “Skills are reusable instructions that teach Pylon agents how to handle specific workflows consistently.”
   • [claimed-docs] “The issue log allows you to inspect all the issues the AI agent is associated with and inspect the outcomes and steps taken by the AI.”
   • [claimed-docs] “Simulate an interaction with the AI and see how it handles different questions.”

   proof ↗⚑ flag

Resolution

1. support leaderThe agent fully resolves a meaningful share of conversations end-to-end — measured as resolutions, not mere deflections or bounces

   weight 3 · round to Sierra
   Sierra~partialprobed6/10

   Sierra's marketing emphasizes agent-driven end-to-end workflow execution (journeys, integrations, payments, guardrails, testing) implying resolution rather than mere deflection, and community evidence corroborates real production use handling customer processes end-to-end for at least one customer. However, no quantitative resolution-rate metrics, benchmarks, or third-party validation of 'meaningful share resolved end-to-end' are provided — tau-bench is a research benchmark, not a customer outcome metric. Missing for 10: published resolution-rate statistics, case studies with concrete resolution percentages, independent audits distinguishing resolution from deflection.

   • [claimed-docs] “Define step-by-step workflows for your agent from scratch, or use AI to instantly generate journeys from your existing operating procedures.”
   • [claimed-docs] “Upload SOPs, raw transcripts, or even audio interviews of subject matter experts to create customer journeys from scratch.”
   • [claimed-docs] “Collect card and ACH payments entirely over the phone with no IVR handoff.”
   • [claimed-docs] “Replace rigid IVR menus with an empathetic voice agent that understands customer context”
   • [community] “Having seen this set up for a customer, what is interesting is how it works with internal APIs for customers. It's all point and click in a …”
   • [community] “The magic isn't in a new LLM technology, it is in reliably productionizing a solution for real-world problems... fill in the gap of missing …”
   • [probe] “PROBE runtime (recorded 2026-09-10): Sierra's agent benchmark is genuinely open — the official sierra-research/tau-bench repository README f…”

   proof ↗⚑ flag

   Pylon~partialclaimed3/10

   Pylon's own docs describe the Support Agent primarily in terms of 'deflecting' questions and taking actions, with an issue log to inspect outcomes/steps, but there is no first-party or independent metric quantifying full end-to-end resolutions distinct from deflections or bounces. missing for 10: quantified resolution-rate reporting, evidence distinguishing true resolutions from deflections, independent/customer validation of resolution outcomes.

   • [claimed-docs] “Support Agents can deflect customer questions using content, gather internal context for your team, and take action.”
   • [claimed-docs] “You can assign them to issues you'd like them to take a look at. This means you have full control of which customer issues your AI agent int…”
   • [claimed-docs] “The issue log allows you to inspect all the issues the AI agent is associated with and inspect the outcomes and steps taken by the AI.”
   • [claimed-docs] “Pylon comes with a number of default analytics dashboards with common support metrics. All dashboards are filterable by a wide selection of …”

   proof ↗⚑ flag

Voice

1. support leaderI control the agent's tone and brand voice, and it stays consistent across topics and languages

   weight 1 · round to Sierra
   Sierra~partialclaimed7/10

   Sierra provides explicit tone/brand-voice controls (Ghostwriter prompts for tone and style, Voice Personas tuned across 59 languages) plus testing/simulation tools to verify agent behavior across scenarios and avoid regressions, which supports consistency claims. However, all evidence is vendor-authored with no independent or hands-on verification that voice/tone actually stays consistent across topics and languages in production. Missing for 10: independent case studies or benchmarks confirming cross-topic/cross-language tone consistency, and detail on how brand-voice guardrails are enforced at scale.

   • [claimed-docs] “Build or modify agents by describing how you want them to behave. Update workflows, systems integrations, guardrails, tone, and style with s…”
   • [claimed-docs] “Use Voice Personas to design how your agent sounds, speaks, and connects, powered by a constellation of models tuned across 59 languages.”
   • [claimed-docs] “Verify your agent performs as expected across a wide range of scenarios and avoid regressions.”
   • [claimed-docs] “Agent Checks and Simulations catching problems proactively, merge approval workflows putting a person in the loop, and split traffic release…”

   proof ↗⚑ flag

   Pylon~partialclaimed3/10

   Pylon's AI Agent can be configured with natural-language runbooks and reusable 'skills' that dictate how it handles specific scenarios, which could be used to encode tone/brand-voice guidance, but the docs never explicitly mention persona/tone control or consistency across languages. missing for 10: explicit tone/brand-voice configuration settings, evidence of multilingual consistency, and any customer/independent proof the agent's voice stays consistent across topics.

   • [claimed-docs] “Write instructions in natural language to help your AI Agent handle specific scenarios.”
   • [claimed-docs] “Skills are reusable instructions that teach Pylon agents how to handle specific workflows consistently.”
   • [claimed-docs] “Support Agents can deflect customer questions using content, gather internal context for your team, and take action.”

   proof ↗⚑ flag

Qa

1. support ops leadAI conversations get ongoing QA — scored samples, flagged failures, and a review loop that feeds fixes back into the agent

   weight 1 · round to Sierra
   Sierra~partialclaimed7/10

   Sierra documents Agent Checks and Simulations that proactively catch problems, merge-approval workflows for human review, and split-traffic releases (sierra-docs-18), plus regression testing (sierra-docs-4) and Explorer/Insights tools that surface conversation trends, flag emerging issues, and explain agent reasoning (sierra-docs-10, -11, -13). Ghostwriter lets teams feed fixes back by updating workflows/guardrails via prompts (sierra-docs-8). Missing for 10: explicit description of scored/sampled QA reviews with quantitative scoring rubrics, and independent evidence of the review loop actually closing the gap between flagged failures and shipped fixes.

   • [claimed-docs] “Verify your agent performs as expected across a wide range of scenarios and avoid regressions.”
   • [claimed-docs] “Agent Checks and Simulations catching problems proactively, merge approval workflows putting a person in the loop, and split traffic release…”
   • [claimed-docs] “Ask any question about your customer experience in natural language, and Explorer identifies the answer across thousands of real conversatio…”
   • [claimed-docs] “Explorer automatically delivers a weekly briefing on key trends, emerging issues, and recommendations — without you having to ask.”
   • [claimed-docs] “Understand the reasoning behind every agent action or answer—view knowledge sources, systems accessed, and more.”
   • [claimed-docs] “Build or modify agents by describing how you want them to behave. Update workflows, systems integrations, guardrails, tone, and style with s…”

   proof ↗⚑ flag

   Pylon~partialclaimed5/10

   Pylon documents a monitor-and-iterate loop: a 'test' simulation mode to probe agent behavior, an issue log to inspect outcomes/steps taken by the AI, and runbooks/skills that let ops feed corrections back into the agent's instructions. However, there's no evidence of structured QA scoring of samples or a formal flagged-failure workflow with review states — it's more free-form inspection than a scored QA/review pipeline. Missing for 10: sample scoring/rubrics, dedicated failure-flagging workflow with reviewer roles, and quantitative QA metrics tied back to agent training.

   • [claimed-docs] “Simulate an interaction with the AI and see how it handles different questions.”
   • [claimed-docs] “You can assign them to issues you'd like them to take a look at. This means you have full control of which customer issues your AI agent int…”
   • [claimed-docs] “The issue log allows you to inspect all the issues the AI agent is associated with and inspect the outcomes and steps taken by the AI.”
   • [claimed-docs] “Write instructions in natural language to help your AI Agent handle specific scenarios.”
   • [claimed-docs] “Skills are reusable instructions that teach Pylon agents how to handle specific workflows consistently.”

   proof ↗⚑ flag

Simulation

1. support ops leadI test the agent against historical tickets or simulated conversations before it faces real customers

   weight 2 · round to Sierra
   Sierra✓fullprobed8/10

   Sierra explicitly documents 'Agent Checks and Simulations' and 'verify your agent performs as expected across a wide range of scenarios and avoid regressions' (sierra-docs-4, sierra-docs-18), directly matching pre-release testing against scenarios/regressions. This is further corroborated by Sierra's public tau-bench research benchmark for evaluating conversational agents on simulated user interactions (sierra-probe-rt-1), showing real investment in simulation-based testing methodology. Missing for 10: no explicit mention of testing against historical/real ticket transcripts specifically (only 'scenarios' and simulations), and no independent hands-on customer account of the simulation workflow in practice.

   • [claimed-docs] “Verify your agent performs as expected across a wide range of scenarios and avoid regressions.”
   • [claimed-docs] “Agent Checks and Simulations catching problems proactively, merge approval workflows putting a person in the loop, and split traffic release…”
   • [probe] “PROBE runtime (recorded 2026-09-10): Sierra's agent benchmark is genuinely open — the official sierra-research/tau-bench repository README f…”

   proof ↗⚑ flag

   Pylon~partialclaimed6/10

   Pylon's docs explicitly describe a test/simulate feature ('Simulate an interaction with the AI and see how it handles different questions') plus an issue log to inspect outcomes, which supports pre-deployment validation and monitoring. However, there's no explicit mention of testing against historical ticket datasets in bulk, regression testing, or scoring accuracy across a corpus of past conversations. missing for 10: bulk/historical-ticket backtesting workflow, quantitative accuracy/regression metrics from test runs, independent/hands-on confirmation of the simulate feature's depth.

   • [claimed-docs] “Simulate an interaction with the AI and see how it handles different questions.”
   • [claimed-docs] “You can assign them to issues you'd like them to take a look at. This means you have full control of which customer issues your AI agent int…”
   • [claimed-docs] “The issue log allows you to inspect all the issues the AI agent is associated with and inspect the outcomes and steps taken by the AI.”
   • [claimed-docs] “Write instructions in natural language to help your AI Agent handle specific scenarios.”

   proof ↗⚑ flag