AI Assistants arenaBuyer checklist
Every requirement we judge ai assistants products against, as a ready-to-send RFP checklist — with each item's priority, why it matters, and how the top-ranked products score on it today.
52 requirements · 12 themes · verdicts for 9 products · updated 2026-09-16 · priorities mirror the story weights our scoring uses (methodology)
Show the markdown export
# AI Assistants — buyer checklist (RFP) Derived from ProductArena's evidence-graded user-story taxonomy for AI Assistants: 52 judged requirements. Priorities mirror story weights (3 = must-have, 2 = should-have, 1 = nice-to-have). ## Agenticness - [ ] **[must-have]** Plug MCP servers into this product so it can use their tools - [ ] **[must-have]** Connect an agent via an official MCP server - [ ] **[must-have]** Drive the product through a documented public API - [ ] **[must-have]** Delegate tasks to a built-in AI assistant inside the product - [ ] **[should-have]** Point an agent at llms.txt or agent-oriented docs - [ ] **[should-have]** Run the product headlessly / in CI for automation - [ ] **[should-have]** Use an official CLI - [ ] **[should-have]** Issue scoped/least-privilege API credentials for an agent - [ ] **[should-have]** Build against official SDKs - [ ] **[should-have]** Subscribe to events via webhooks - [ ] **[should-have]** Get AI-generated insights and suggestions from my data inside the product - [ ] **[should-have]** Set up automations that run autonomously in the background - [ ] **[should-have]** Operate the product with natural-language commands - [ ] **[should-have]** Explore an interactive API reference with runnable examples - [ ] **[should-have]** Download a machine-readable API spec (OpenAPI or equivalent) - [ ] **[should-have]** Rely on versioned APIs with a documented deprecation policy - [ ] **[nice-to-have]** Test against a sandbox environment without touching production data ## Agents tasks - [ ] **[must-have]** Delegate a multi-step task that the assistant works on autonomously in the background and returns for my review - [ ] **[must-have]** Have the assistant operate a web browser on my behalf to research and complete tasks on websites - [ ] **[should-have]** Let the assistant see and operate applications on my computer to complete work - [ ] **[should-have]** Schedule recurring or one-off tasks that run automatically and come back to me with results ## Apps devices - [ ] **[should-have]** Use an official desktop app with OS-level shortcuts and access to what is on my screen - [ ] **[should-have]** Use full-featured official mobile apps for iOS and Android - [ ] **[should-have]** Build and share custom assistants with their own instructions and knowledge ## Automation depth - [ ] **[must-have]** Define rules that trigger actions automatically on events - [ ] **[should-have]** Perform bulk operations across many items at once - [ ] **[should-have]** Schedule recurring jobs or workflows - [ ] **[nice-to-have]** Version, review, and roll back my automations ## Connectors apps - [ ] **[must-have]** Connect my cloud drive, email, and calendar so the assistant can search and use them in answers - [ ] **[should-have]** Browse a directory of third-party apps and connectors and add them to the assistant ## Files analysis - [ ] **[must-have]** Have the assistant write and run code on my data to produce charts, computed answers, and downloadable files - [ ] **[must-have]** Upload documents, spreadsheets, and PDFs and get accurate analysis of their contents - [ ] **[should-have]** Have the assistant create and iteratively edit documents, presentations, and other files I can export ## Memory context - [ ] **[must-have]** Have the assistant remember relevant context from previous chats and apply it in new conversations - [ ] **[should-have]** Organize related chats and files into a project or space that shares context and instructions - [ ] **[nice-to-have]** Set persistent custom instructions and preferences that shape every response ## Multimodal - [ ] **[should-have]** Generate and edit images from natural-language prompts - [ ] **[should-have]** Share screenshots and photos and have the assistant accurately interpret what is in them - [ ] **[should-have]** Have a natural, real-time voice conversation with the assistant ## Openness - [ ] **[must-have]** Export all of my data in open formats and leave - [ ] **[must-have]** Self-host the core product - [ ] **[should-have]** Do everything through the API that I can do in the UI - [ ] **[should-have]** Read the product's source under an open license ## Privacy posture - [ ] **[must-have]** Prevent my data from being used to train AI models - [ ] **[should-have]** Choose where my data is stored (region/residency) - [ ] **[should-have]** Control data retention and deletion - [ ] **[should-have]** Opt out of telemetry and usage tracking ## Research answers - [ ] **[must-have]** Launch a deep research run that autonomously searches many sources and returns a cited report - [ ] **[should-have]** Get answers grounded in current web results with citations back to the sources ## Trust controls - [ ] **[must-have]** Control whether my conversations are used to train models - [ ] **[should-have]** Manage members, permissions, and data policies for my organization's workspace - [ ] **[nice-to-have]** Export my complete chat history and account data --- Source: https://ultrametric.ai/productarena/arena/ai-assistants (evidence-graded verdicts for 9 products) · methodology: https://ultrametric.ai/productarena/methodology
Chips show the top 5 ranked products' current verdict on each requirement — ✓ full · ~ partial · ! disputed · — none · n/a not applicable.
Agenticness — how well agents can access and operate the productAgenticness· 17 items
How well agents can access and operate the product
- must-have
ai-native userPlug MCP servers into this product so it can use their tools
Core requirement — weighs 3× in arena scoring · 4 of 9 products fully deliver this today
- must-have
ai-native userConnect an agent via an official MCP server
Core requirement — weighs 3× in arena scoring · all 2 products fully deliver this today
- must-have
ai-native userDrive the product through a documented public API
Core requirement — weighs 3× in arena scoring · 2 of 9 products fully deliver this today
- must-have
ai-native userDelegate tasks to a built-in AI assistant inside the product
Core requirement — weighs 3× in arena scoring · 6 of 9 products fully deliver this today
- should-have
ai-native userPoint an agent at llms.txt or agent-oriented docs
Important, not disqualifying — weighs 2× in arena scoring · 5 of 8 products fully deliver this today
- should-have
ai-native userRun the product headlessly / in CI for automation
Important, not disqualifying — weighs 2× in arena scoring · 2 of 6 products fully deliver this today
- should-have
ai-native userUse an official CLI
Important, not disqualifying — weighs 2× in arena scoring · 2 of 8 products fully deliver this today
- should-have
ai-native userIssue scoped/least-privilege API credentials for an agent
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
- should-have
ai-native userBuild against official SDKs
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
- should-have
ai-native userSubscribe to events via webhooks
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
- should-have
ai-native userGet AI-generated insights and suggestions from my data inside the product
Important, not disqualifying — weighs 2× in arena scoring · 3 of 9 products fully deliver this today
- should-have
ai-native userSet up automations that run autonomously in the background
Important, not disqualifying — weighs 2× in arena scoring · 3 of 9 products fully deliver this today
- should-have
ai-native userOperate the product with natural-language commands
Important, not disqualifying — weighs 2× in arena scoring · 6 of 9 products fully deliver this today
- should-have
ai-native userExplore an interactive API reference with runnable examples
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
- should-have
ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)
Important, not disqualifying — weighs 2× in arena scoring · 2 of 8 products fully deliver this today
- should-have
ai-native userRely on versioned APIs with a documented deprecation policy
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
- nice-to-have
ai-native userTest against a sandbox environment without touching production data
Differentiator, not a dealbreaker — weighs 1× in arena scoring · no product fully delivers this yet
Agents tasks — stories about agents tasks in this arenaAgents tasks· 4 items
Stories about agents tasks in this arena
- must-have
power-userDelegate a multi-step task that the assistant works on autonomously in the background and returns for my review
Core requirement — weighs 3× in arena scoring · 2 of 9 products fully deliver this today
- must-have
power-userHave the assistant operate a web browser on my behalf to research and complete tasks on websites
Core requirement — weighs 3× in arena scoring · 2 of 9 products fully deliver this today
- should-have
power-userLet the assistant see and operate applications on my computer to complete work
Important, not disqualifying — weighs 2× in arena scoring · 2 of 9 products fully deliver this today
- should-have
power-userSchedule recurring or one-off tasks that run automatically and come back to me with results
Important, not disqualifying — weighs 2× in arena scoring · 2 of 9 products fully deliver this today
Apps devices — stories about apps devices in this arenaApps devices· 3 items
Stories about apps devices in this arena
- should-have
power-userUse an official desktop app with OS-level shortcuts and access to what is on my screen
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
- should-have
knowledge-workerUse full-featured official mobile apps for iOS and Android
Important, not disqualifying — weighs 2× in arena scoring · 3 of 9 products fully deliver this today
- should-have
power-userBuild and share custom assistants with their own instructions and knowledge
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
Automation depth — how much of the product can run unattendedAutomation depth· 4 items
How much of the product can run unattended
- must-have
ai-native userDefine rules that trigger actions automatically on events
Core requirement — weighs 3× in arena scoring · 1 of 9 products fully deliver this today
- should-have
ai-native userPerform bulk operations across many items at once
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
- should-have
ai-native userSchedule recurring jobs or workflows
Important, not disqualifying — weighs 2× in arena scoring · 2 of 9 products fully deliver this today
- nice-to-have
ai-native userVersion, review, and roll back my automations
Differentiator, not a dealbreaker — weighs 1× in arena scoring · no product fully delivers this yet
Connectors apps — stories about connectors apps in this arenaConnectors apps· 2 items
Stories about connectors apps in this arena
- must-have
knowledge-workerConnect my cloud drive, email, and calendar so the assistant can search and use them in answers
Core requirement — weighs 3× in arena scoring · 4 of 9 products fully deliver this today
- should-have
power-userBrowse a directory of third-party apps and connectors and add them to the assistant
Important, not disqualifying — weighs 2× in arena scoring · 3 of 9 products fully deliver this today
Files analysis — stories about files analysis in this arenaFiles analysis· 3 items
Stories about files analysis in this arena
- must-have
power-userHave the assistant write and run code on my data to produce charts, computed answers, and downloadable files
Core requirement — weighs 3× in arena scoring · 2 of 8 products fully deliver this today
- must-have
knowledge-workerUpload documents, spreadsheets, and PDFs and get accurate analysis of their contents
Core requirement — weighs 3× in arena scoring · 3 of 9 products fully deliver this today
- should-have
knowledge-workerHave the assistant create and iteratively edit documents, presentations, and other files I can export
Important, not disqualifying — weighs 2× in arena scoring · 2 of 8 products fully deliver this today
Memory context — stories about memory context in this arenaMemory context· 3 items
Stories about memory context in this arena
- must-have
power-userHave the assistant remember relevant context from previous chats and apply it in new conversations
Core requirement — weighs 3× in arena scoring · 3 of 9 products fully deliver this today
- should-have
knowledge-workerOrganize related chats and files into a project or space that shares context and instructions
Important, not disqualifying — weighs 2× in arena scoring · 3 of 8 products fully deliver this today
- nice-to-have
power-userSet persistent custom instructions and preferences that shape every response
Differentiator, not a dealbreaker — weighs 1× in arena scoring · 1 of 9 products fully deliver this today
Multimodal — stories about multimodal in this arenaMultimodal· 3 items
Stories about multimodal in this arena
- should-have
knowledge-workerGenerate and edit images from natural-language prompts
Important, not disqualifying — weighs 2× in arena scoring · 3 of 8 products fully deliver this today
- should-have
knowledge-workerShare screenshots and photos and have the assistant accurately interpret what is in them
Important, not disqualifying — weighs 2× in arena scoring · 3 of 9 products fully deliver this today
- should-have
knowledge-workerHave a natural, real-time voice conversation with the assistant
Important, not disqualifying — weighs 2× in arena scoring · 5 of 9 products fully deliver this today
Openness — open source, data portability, and self-hosting storiesOpenness· 4 items
Open source, data portability, and self-hosting stories
- must-have
ai-native userExport all of my data in open formats and leave
Core requirement — weighs 3× in arena scoring · no product fully delivers this yet
- must-have
ai-native userSelf-host the core product
Core requirement — weighs 3× in arena scoring · no product fully delivers this yet
- should-have
ai-native userDo everything through the API that I can do in the UI
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
- should-have
ai-native userRead the product's source under an open license
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
Privacy posture — data-handling and privacy storiesPrivacy posture· 4 items
Data-handling and privacy stories
- must-have
ai-native userPrevent my data from being used to train AI models
Core requirement — weighs 3× in arena scoring · 1 of 9 products fully deliver this today
- should-have
ai-native userChoose where my data is stored (region/residency)
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
- should-have
ai-native userControl data retention and deletion
Important, not disqualifying — weighs 2× in arena scoring · 2 of 9 products fully deliver this today
- should-have
ai-native userOpt out of telemetry and usage tracking
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
Research answers — stories about research answers in this arenaResearch answers· 2 items
Stories about research answers in this arena
- must-have
knowledge-workerLaunch a deep research run that autonomously searches many sources and returns a cited report
Core requirement — weighs 3× in arena scoring · 2 of 9 products fully deliver this today
- should-have
knowledge-workerGet answers grounded in current web results with citations back to the sources
Important, not disqualifying — weighs 2× in arena scoring · 2 of 9 products fully deliver this today
Trust controls — stories about trust controls in this arenaTrust controls· 3 items
Stories about trust controls in this arena
- must-have
knowledge-workerControl whether my conversations are used to train models
Core requirement — weighs 3× in arena scoring · 2 of 9 products fully deliver this today
- should-have
team-adminManage members, permissions, and data policies for my organization's workspace
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
- nice-to-have
knowledge-workerExport my complete chat history and account data
Differentiator, not a dealbreaker — weighs 1× in arena scoring · 1 of 9 products fully deliver this today
Full evidence behind every verdict lives on the arena page and each product page — chips above deep-link straight to the judged story.