Skip to content
ProductArena

AI Customer Support Agents Arena

AI Customer Support Agents arenaBuyer checklist

Every requirement we judge ai customer support agents products against, as a ready-to-send RFP checklist — with each item's priority, why it matters, and how the top-ranked products score on it today.

53 requirements · 14 themes · verdicts for 6 products · updated 2026-09-16 · priorities mirror the story weights our scoring uses (methodology)

Procurement report →
Show the markdown export
# AI Customer Support Agents — buyer checklist (RFP)

Derived from ProductArena's evidence-graded user-story taxonomy for AI Customer Support Agents: 53 judged requirements. Priorities mirror story weights (3 = must-have, 2 = should-have, 1 = nice-to-have).

## Agent actions

- [ ] **[must-have]** The agent takes real actions through my APIs — refunds, order changes, subscription updates — with scoped auth per action
- [ ] **[should-have]** I encode standard operating procedures the agent follows step-by-step for known issue types, with deterministic branching

## Agenticness

- [ ] **[must-have]** Plug MCP servers into this product so it can use their tools
- [ ] **[must-have]** Connect an agent via an official MCP server
- [ ] **[must-have]** Drive the product through a documented public API
- [ ] **[must-have]** Delegate tasks to a built-in AI assistant inside the product
- [ ] **[should-have]** Point an agent at llms.txt or agent-oriented docs
- [ ] **[should-have]** Run the product headlessly / in CI for automation
- [ ] **[should-have]** Use an official CLI
- [ ] **[should-have]** Issue scoped/least-privilege API credentials for an agent
- [ ] **[should-have]** Build against official SDKs
- [ ] **[should-have]** Subscribe to events via webhooks
- [ ] **[should-have]** Get AI-generated insights and suggestions from my data inside the product
- [ ] **[should-have]** Set up automations that run autonomously in the background
- [ ] **[should-have]** Operate the product with natural-language commands
- [ ] **[should-have]** Explore an interactive API reference with runnable examples
- [ ] **[should-have]** Download a machine-readable API spec (OpenAPI or equivalent)
- [ ] **[should-have]** Rely on versioned APIs with a documented deprecation policy
- [ ] **[nice-to-have]** Test against a sandbox environment without touching production data

## Automation depth

- [ ] **[must-have]** Define rules that trigger actions automatically on events
- [ ] **[should-have]** Perform bulk operations across many items at once
- [ ] **[should-have]** Schedule recurring jobs or workflows
- [ ] **[nice-to-have]** Version, review, and roll back my automations

## Channels languages

- [ ] **[should-have]** One agent covers chat, email, and in-app, plus the channels my customers actually use — Slack, WhatsApp, social
- [ ] **[should-have]** The agent supports customers in many languages, even where my knowledge base exists only in English
- [ ] **[should-have]** The agent handles phone calls — speech in, speech out — with the same knowledge and actions as chat

## Escalation handoff

- [ ] **[must-have]** When the agent escalates, the human gets the full conversation, a summary, and collected details — the customer never repeats themselves
- [ ] **[should-have]** I configure when the agent must hand off — by topic, sentiment, customer tier, or explicit request — and it reliably obeys

## Guardrails safety

- [ ] **[must-have]** Guardrails stop the agent from inventing policies, prices, or promises — off-knowledge questions get a safe decline, not a guess
- [ ] **[should-have]** Launch in a supervised mode where the agent drafts replies for human approval before anything reaches a customer
- [ ] **[should-have]** I mark topics as human-only — legal threats, cancellations, security — and the agent never freelances on them

## Insights analytics

- [ ] **[must-have]** Dashboards show resolution rate, CSAT, handoff rate, and cost per resolution — the numbers I report to my exec team
- [ ] **[nice-to-have]** The platform clusters conversations by topic and surfaces emerging product issues before they spike ticket volume

## Integrations platform

- [ ] **[must-have]** The agent runs inside my existing helpdesk — Zendesk, Salesforce, Intercom — or standalone, syncing tickets and context both ways

## Knowledge grounding

- [ ] **[must-have]** Every answer is grounded in my own content and shows which article or source it drew from
- [ ] **[must-have]** The agent ingests my help center, docs, past tickets, and internal wikis as knowledge sources without manual re-authoring
- [ ] **[should-have]** Knowledge stays current automatically — the agent re-syncs sources on a schedule or on change, not via manual re-uploads
- [ ] **[nice-to-have]** The platform surfaces knowledge gaps and conflicting content that cause the agent to miss or fumble questions

## Openness

- [ ] **[must-have]** Export all of my data in open formats and leave
- [ ] **[must-have]** Self-host the core product
- [ ] **[should-have]** Do everything through the API that I can do in the UI
- [ ] **[should-have]** Read the product's source under an open license

## Pricing economics

- [ ] **[should-have]** Pricing is outcome-based and published — I pay per resolution with caps and controls, not an opaque enterprise quote

## Privacy posture

- [ ] **[must-have]** Prevent my data from being used to train AI models
- [ ] **[should-have]** Choose where my data is stored (region/residency)
- [ ] **[should-have]** Control data retention and deletion
- [ ] **[should-have]** Opt out of telemetry and usage tracking

## Resolution quality

- [ ] **[must-have]** The agent fully resolves a meaningful share of conversations end-to-end — measured as resolutions, not mere deflections or bounces
- [ ] **[should-have]** Answers use the customer's live data — plan, order status, account history — not just generic help articles
- [ ] **[should-have]** The agent asks clarifying questions and works through multi-step troubleshooting instead of dumping one canned answer
- [ ] **[nice-to-have]** I control the agent's tone and brand voice, and it stays consistent across topics and languages

## Testing qa

- [ ] **[should-have]** I test the agent against historical tickets or simulated conversations before it faces real customers
- [ ] **[nice-to-have]** AI conversations get ongoing QA — scored samples, flagged failures, and a review loop that feeds fixes back into the agent

---

Source: https://ultrametric.ai/productarena/arena/ai-support-agents (evidence-graded verdicts for 6 products) · methodology: https://ultrametric.ai/productarena/methodology

Chips show the top 5 ranked products' current verdict on each requirement — ✓ full · ~ partial · ! disputed · — none · n/a not applicable.

Agent actions — stories about agent actions in this arenaAgent actions· 2 items

Stories about agent actions in this arena

  • developerThe agent takes real actions through my APIs — refunds, order changes, subscription updates — with scoped auth per action

    Core requirement — weighs 3× in arena scoring · 2 of 6 products fully deliver this today

    LorikeetPylonParahelpFinSierra
    must-have

  • support ops leadI encode standard operating procedures the agent follows step-by-step for known issue types, with deterministic branching

    Important, not disqualifying — weighs 2× in arena scoring · 4 of 6 products fully deliver this today

    LorikeetPylonParahelpFinSierra
    should-have

Agenticness — how well agents can access and operate the productAgenticness· 17 items

How well agents can access and operate the product

Automation depth — how much of the product can run unattendedAutomation depth· 4 items

How much of the product can run unattended

Channels languages — stories about channels languages in this arenaChannels languages· 3 items

Stories about channels languages in this arena

  • support leaderOne agent covers chat, email, and in-app, plus the channels my customers actually use — Slack, WhatsApp, social

    Important, not disqualifying — weighs 2× in arena scoring · 1 of 6 products fully deliver this today

    LorikeetPylonParahelpFinSierra
    should-have

  • support leaderThe agent supports customers in many languages, even where my knowledge base exists only in English

    Important, not disqualifying — weighs 2× in arena scoring · 1 of 6 products fully deliver this today

    LorikeetPylonParahelpFinSierra
    should-have

  • support leaderThe agent handles phone calls — speech in, speech out — with the same knowledge and actions as chat

    Important, not disqualifying — weighs 2× in arena scoring · 2 of 6 products fully deliver this today

    LorikeetPylonParahelpFinSierra
    should-have

Escalation handoff — stories about escalation handoff in this arenaEscalation handoff· 2 items

Stories about escalation handoff in this arena

  • support leaderWhen the agent escalates, the human gets the full conversation, a summary, and collected details — the customer never repeats themselves

    Core requirement — weighs 3× in arena scoring · no product fully delivers this yet

    LorikeetPylonParahelpFinSierra
    must-have

  • support ops leadI configure when the agent must hand off — by topic, sentiment, customer tier, or explicit request — and it reliably obeys

    Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet

    LorikeetPylonParahelpFinSierra
    should-have

Guardrails safety — stories about guardrails safety in this arenaGuardrails safety· 3 items

Stories about guardrails safety in this arena

  • ai-native userGuardrails stop the agent from inventing policies, prices, or promises — off-knowledge questions get a safe decline, not a guess

    Core requirement — weighs 3× in arena scoring · 1 of 6 products fully deliver this today

    LorikeetPylonParahelpFinSierra
    must-have

  • support ops leadLaunch in a supervised mode where the agent drafts replies for human approval before anything reaches a customer

    Important, not disqualifying — weighs 2× in arena scoring · 1 of 6 products fully deliver this today

    LorikeetPylonParahelpFinSierra
    should-have

  • support ops leadI mark topics as human-only — legal threats, cancellations, security — and the agent never freelances on them

    Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet

    LorikeetPylonParahelpFinSierra
    should-have

Insights analytics — stories about insights analytics in this arenaInsights analytics· 2 items

Stories about insights analytics in this arena

  • support leaderDashboards show resolution rate, CSAT, handoff rate, and cost per resolution — the numbers I report to my exec team

    Core requirement — weighs 3× in arena scoring · no product fully delivers this yet

    LorikeetPylonParahelpFinSierra
    must-have

  • support leaderThe platform clusters conversations by topic and surfaces emerging product issues before they spike ticket volume

    Differentiator, not a dealbreaker — weighs 1× in arena scoring · no product fully delivers this yet

    LorikeetPylonParahelpFinSierra
    nice-to-have

Integrations platform — stories about integrations platform in this arenaIntegrations platform· 1 item

Stories about integrations platform in this arena

  • developerThe agent runs inside my existing helpdesk — Zendesk, Salesforce, Intercom — or standalone, syncing tickets and context both ways

    Core requirement — weighs 3× in arena scoring · 2 of 6 products fully deliver this today

    LorikeetPylonParahelpFinSierra
    must-have

Knowledge grounding — stories about knowledge grounding in this arenaKnowledge grounding· 4 items

Stories about knowledge grounding in this arena

  • ai-native userEvery answer is grounded in my own content and shows which article or source it drew from

    Core requirement — weighs 3× in arena scoring · 1 of 6 products fully deliver this today

    LorikeetPylonParahelpFinSierra
    must-have

  • support ops leadThe agent ingests my help center, docs, past tickets, and internal wikis as knowledge sources without manual re-authoring

    Core requirement — weighs 3× in arena scoring · 2 of 6 products fully deliver this today

    LorikeetPylonParahelpFinSierra
    must-have

  • support ops leadKnowledge stays current automatically — the agent re-syncs sources on a schedule or on change, not via manual re-uploads

    Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet

    LorikeetPylonParahelpFinSierra
    should-have

  • support ops leadThe platform surfaces knowledge gaps and conflicting content that cause the agent to miss or fumble questions

    Differentiator, not a dealbreaker — weighs 1× in arena scoring · 2 of 6 products fully deliver this today

    LorikeetPylonParahelpFinSierra
    nice-to-have

Openness — open source, data portability, and self-hosting storiesOpenness· 4 items

Open source, data portability, and self-hosting stories

Pricing economics — stories about pricing economics in this arenaPricing economics· 1 item

Stories about pricing economics in this arena

  • support leaderPricing is outcome-based and published — I pay per resolution with caps and controls, not an opaque enterprise quote

    Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet

    LorikeetPylonParahelpFinSierra
    should-have

Privacy posture — data-handling and privacy storiesPrivacy posture· 4 items

Data-handling and privacy stories

Resolution quality — stories about resolution quality in this arenaResolution quality· 4 items

Stories about resolution quality in this arena

  • support leaderThe agent fully resolves a meaningful share of conversations end-to-end — measured as resolutions, not mere deflections or bounces

    Core requirement — weighs 3× in arena scoring · 1 of 6 products fully deliver this today

    LorikeetPylonParahelpFinSierra
    must-have

  • support leaderAnswers use the customer's live data — plan, order status, account history — not just generic help articles

    Important, not disqualifying — weighs 2× in arena scoring · 2 of 6 products fully deliver this today

    LorikeetPylonParahelpFinSierra
    should-have

  • support leaderThe agent asks clarifying questions and works through multi-step troubleshooting instead of dumping one canned answer

    Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet

    LorikeetPylonParahelpFinSierra
    should-have

  • support leaderI control the agent's tone and brand voice, and it stays consistent across topics and languages

    Differentiator, not a dealbreaker — weighs 1× in arena scoring · no product fully delivers this yet

    LorikeetPylonParahelpFinSierra
    nice-to-have

Testing qa — stories about testing qa in this arenaTesting qa· 2 items

Stories about testing qa in this arena

  • support ops leadI test the agent against historical tickets or simulated conversations before it faces real customers

    Important, not disqualifying — weighs 2× in arena scoring · 5 of 6 products fully deliver this today

    LorikeetPylonParahelpFinSierra
    should-have

  • support ops leadAI conversations get ongoing QA — scored samples, flagged failures, and a review loop that feeds fixes back into the agent

    Differentiator, not a dealbreaker — weighs 1× in arena scoring · 3 of 6 products fully deliver this today

    LorikeetPylonParahelpFinSierra
    nice-to-have

Full evidence behind every verdict lives on the arena page and each product page — chips above deep-link straight to the judged story.