Product Analytics — procurement report
ProductArena · rankings as of 2026-09-16 · evidence as of 2026-09-16 · 4 products · 53 judged requirements · 212 judged cells
Methodology: Every product is judged against a shared taxonomy of user stories using cited evidence — hands-on probes > repository code > independent community sources > vendor claims — never opinion. Full writeup: https://ultrametric.ai/productarena/methodology
Leaderboard
| # | Product | PA Score | Coverage score | Applicable cells | Confidence |
|---|---|---|---|---|---|
| 1 | PostHog | 41.2 | 43.9 | 53/53 | B |
| 2 | Amplitude | 31.5 | 40.0 | 53/53 | B |
| 3 | Mixpanel | 25.3 | 35.1 | 52/53 | C |
| 4 | Plausible Analytics | 18.9 | 21.0 | 47/53 | B |
PA Score = agent-readiness blend (see methodology). Coverage score = weighted share of judged requirements met. Confidence = how much of the score rests on tested vs claimed evidence (A–D).
Uncertainty note
The current #1/#2 gap in this arena is not close enough to qualify for the multi-judge uncertainty pass (or the pass has not covered it yet) — no extra caveat applies beyond the per-product confidence grades above.
Buyer checklist (RFP)
The arena's 53 judged user stories as requirements, grouped by theme. Priorities mirror the story weights our scoring uses (3 = must-have, 2 = should-have, 1 = nice-to-have). Interactive version with per-requirement verdicts for the top products: /arena/product-analytics/checklist
Agenticness — how well agents can access and operate the productAgenticness
How well agents can access and operate the product
- ai-native userPlug MCP servers into this product so it can use their toolsmust-have
- ai-native userConnect an agent via an official MCP servermust-have
- ai-native userDrive the product through a documented public APImust-have
- ai-native userDelegate tasks to a built-in AI assistant inside the productmust-have
- ai-native userPoint an agent at llms.txt or agent-oriented docsshould-have
- ai-native userRun the product headlessly / in CI for automationshould-have
- ai-native userUse an official CLIshould-have
- ai-native userIssue scoped/least-privilege API credentials for an agentshould-have
- ai-native userBuild against official SDKsshould-have
- ai-native userSubscribe to events via webhooksshould-have
- ai-native userGet AI-generated insights and suggestions from my data inside the productshould-have
- ai-native userSet up automations that run autonomously in the backgroundshould-have
- ai-native userOperate the product with natural-language commandsshould-have
- ai-native userExplore an interactive API reference with runnable examplesshould-have
- ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)should-have
- ai-native userRely on versioned APIs with a documented deprecation policyshould-have
- ai-native userTest against a sandbox environment without touching production datanice-to-have
Automation depth — how much of the product can run unattendedAutomation depth
How much of the product can run unattended
- ai-native userDefine rules that trigger actions automatically on eventsmust-have
- ai-native userPerform bulk operations across many items at onceshould-have
- ai-native userSchedule recurring jobs or workflowsshould-have
- ai-native userVersion, review, and roll back my automationsnice-to-have
Dashboards reporting — stories about dashboards reporting in this arenaDashboards reporting
Stories about dashboards reporting in this arena
- ai-native userHave an agent build a dashboard of my key metrics end-to-end via the API or MCP servermust-have
- developerCreate and manage dashboards and saved insights programmatically via the APIshould-have
- product-managerCompose saved insights into shareable dashboards for my teamshould-have
- product-managerSubscribe to dashboards and get alerted when a key metric moves abnormallynice-to-have
Event ingestion — stories about event ingestion in this arenaEvent ingestion
Stories about event ingestion in this arena
- developerSend events from web, mobile, and backend apps through official SDKs for the major languages and platformsmust-have
- developerIdentify users across devices and merge anonymous pre-signup activity into their identified profileshould-have
- product-managerCapture clicks, pageviews, and form interactions automatically without writing tracking code for each eventshould-have
- developerBulk-import historical events from another analytics tool or a data export into the platformnice-to-have
Flags experiments — stories about flags experiments in this arenaFlags experiments
Stories about flags experiments in this arena
- product-managerRun A/B experiments with goal metrics and statistical significance reported on the resultsmust-have
- developerRoll out features behind flags targeted by user properties, cohorts, and percentage rolloutsmust-have
- ai-native userHave an agent create, target, and toggle feature flags through the API or MCP servershould-have
Funnels retention — stories about funnels retention in this arenaFunnels retention
Stories about funnels retention in this arena
- product-managerBuild multi-step funnels with filters and breakdowns to find where users drop offmust-have
- product-managerMeasure retention over time and slice it by behavioral cohortsmust-have
- founderAnswer "which activation step loses users?" myself with funnels and drop-off analysis, without an analystmust-have
- product-managerAnalyze behavior at the account or company level, not just per user, for B2B productsshould-have
- product-managerExplore the paths users actually take before or after a key eventshould-have
Integrations — connecting to other tools — breadth and depth of built-in integrationsIntegrations
Connecting to other tools — breadth and depth of built-in integrations
- developerStream events onward to third-party destinations (CRM, ad platforms, webhooks) from the analytics platformshould-have
Openness — open source, data portability, and self-hosting storiesOpenness
Open source, data portability, and self-hosting stories
- ai-native userExport all of my data in open formats and leavemust-have
- ai-native userSelf-host the core productmust-have
- ai-native userDo everything through the API that I can do in the UIshould-have
- ai-native userRead the product's source under an open licenseshould-have
Privacy cookieless — stories about privacy cookieless in this arenaPrivacy cookieless
Stories about privacy cookieless in this arena
- founderTrack product usage without third-party cookies so I can minimize or avoid cookie consent bannersshould-have
Privacy posture — data-handling and privacy storiesPrivacy posture
Data-handling and privacy stories
- ai-native userPrevent my data from being used to train AI modelsmust-have
- ai-native userChoose where my data is stored (region/residency)should-have
- ai-native userControl data retention and deletionshould-have
- ai-native userOpt out of telemetry and usage trackingshould-have
Session replay — stories about session replay in this arenaSession replay
Stories about session replay in this arena
- product-managerWatch recordings of real user sessions with sensitive input masked by defaultmust-have
- product-managerJump from a funnel drop-off or an error event straight to session replays of the affected usersshould-have
Warehouse sql — stories about warehouse sql in this arenaWarehouse sql
Stories about warehouse sql in this arena
- ai-native userHave an agent answer growth questions from live product data by running queries through the API or MCP servermust-have
- developerRun analytical queries and pull raw event data back through a documented query/export APImust-have
- developerQuery my raw events with SQL (or an SQL-equivalent query language) inside the platformmust-have
- developerSync events between the platform and my own data warehouse (query warehouse tables in-product or export events continuously to it)should-have
Appendix: recorded probes
No replayable probe recordings exist for this arena yet. Probe-tier evidence (hands-on checks) still backs verdicts where cited — see each product page for the evidence trail.
Cite as: ProductArena by Ultrametric Inc, Product Analytics arena, rankings as of 2026-09-16 — https://ultrametric.ai/productarena/arena/product-analytics
License: © 2026 Ultrametric Inc. Brief quotation of individual verdicts, scores, or evidence excerpts is permitted with attribution to "ProductArena by Ultrametric Inc (ultrametric.ai/productarena)", as is use of the data to evaluate, contest, or contribute corrections. Bulk copying, redistribution, or use to build competing datasets requires prior written permission (see DATA-LICENSE in the repository).
No liability: rankings, verdicts, and scores are research outputs derived from the cited evidence at a point in time, provided "as is", without warranties. Ultrametric Inc accepts no responsibility for procurement, purchasing, or other decisions made in reliance on them — verify against the cited evidence before acting (https://ultrametric.ai/productarena/terms).