Incident Management & On-call Arena
Incident Management & On-call — procurement report
ProductArena · rankings as of 2026-09-07 · evidence as of 2026-09-11 · 5 products · 54 judged requirements · 270 judged cells
Methodology: Every product is judged against a shared taxonomy of user stories using cited evidence — hands-on probes > repository code > independent community sources > vendor claims — never opinion. Full writeup: https://ultrametric.ai/productarena/methodology
Leaderboard
| # | Product | PA Score | Coverage score | Applicable cells | Confidence |
|---|---|---|---|---|---|
| 1 | incident.io | 42.3 | 39.5 | 52/54 | B |
| 2 | PagerDuty | 37.3 | 37.6 | 52/54 | B |
| 3 | Rootly | 26.2 | 37.5 | 54/54 | C |
| 4 | FireHydrant | 25.6 | 30.9 | 53/54 | C |
| 5 | Better Stack | 23.1 | 33.3 | 50/54 | C |
PA Score = agent-readiness blend (see methodology). Coverage score = weighted share of judged requirements met. Confidence = how much of the score rests on tested vs claimed evidence (A–D).
Uncertainty note
The current #1/#2 gap in this arena is not close enough to qualify for the multi-judge uncertainty pass (or the pass has not covered it yet) — no extra caveat applies beyond the per-product confidence grades above.
Buyer checklist (RFP)
The arena's 54 judged user stories as requirements, grouped by theme. Priorities mirror the story weights our scoring uses (3 = must-have, 2 = should-have, 1 = nice-to-have). Interactive version with per-requirement verdicts for the top products: /arena/incident-management/checklist
Agenticness — how well agents can access and operate the productAgenticness
How well agents can access and operate the product
- ai-native userPlug MCP servers into this product so it can use their toolsmust-have
- ai-native userConnect an agent via an official MCP servermust-have
- ai-native userDrive the product through a documented public APImust-have
- ai-native userDelegate tasks to a built-in AI assistant inside the productmust-have
- ai-native userPoint an agent at llms.txt or agent-oriented docsshould-have
- ai-native userRun the product headlessly / in CI for automationshould-have
- ai-native userUse an official CLIshould-have
- ai-native userIssue scoped/least-privilege API credentials for an agentshould-have
- ai-native userBuild against official SDKsshould-have
- ai-native userSubscribe to events via webhooksshould-have
- ai-native userGet AI-generated insights and suggestions from my data inside the productshould-have
- ai-native userSet up automations that run autonomously in the backgroundshould-have
- ai-native userOperate the product with natural-language commandsshould-have
- ai-native userExplore an interactive API reference with runnable examplesshould-have
- ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)should-have
- ai-native userRely on versioned APIs with a documented deprecation policyshould-have
- ai-native userTest against a sandbox environment without touching production datanice-to-have
Ai incident — stories about ai incident in this arenaAi incident
Stories about ai incident in this arena
- ai-native userMy agent can acknowledge, escalate, and resolve incidents end to end through a documented API or MCP connection — no dashboard in the loopmust-have
- ai-native userAI writes the incident as it happens — live summaries, drafted updates, and scribed call notes — so responders respond instead of typingmust-have
- ai-native userAn AI investigator digs into the probable cause — correlating changes, telemetry, and similar past incidents — before a human even asksshould-have
- ai-native userAI drafts the postmortem from the incident record — timeline, contributing factors, follow-ups — ready for human reviewshould-have
Alerting escalation — stories about alerting escalation in this arenaAlerting escalation
Stories about alerting escalation in this arena
- sreEscalation policies walk unacknowledged pages through multiple steps — delays, fallback responders, and repeat rounds — until someone acknowledgesmust-have
- sreAlerts from my monitoring tools are ingested through documented sources and routed to the right team by conditions I definemust-have
- sreDuplicate and related alerts are deduplicated and grouped so one incident pages one human, not fiftyshould-have
- on-call engineerPages reach me over the channels I choose — push, SMS, phone call, and email — with per-channel notification rulesshould-have
Analytics reliability — stories about analytics reliability in this arenaAnalytics reliability
Stories about analytics reliability in this arena
- engineering leaderI get reliability analytics — MTTA/MTTR trends, incident load, on-call health — to see whether we are actually improvingshould-have
Automation depth — how much of the product can run unattendedAutomation depth
How much of the product can run unattended
- ai-native userDefine rules that trigger actions automatically on eventsmust-have
- ai-native userPerform bulk operations across many items at onceshould-have
- ai-native userSchedule recurring jobs or workflowsshould-have
- ai-native userVersion, review, and roll back my automationsnice-to-have
Automation runbooks — stories about automation runbooks in this arenaAutomation runbooks
Stories about automation runbooks in this arena
- sreRunbooks attach to incidents and their steps can trigger automatically — creating channels, assigning tasks, running diagnosticsshould-have
- sreA condition-based workflow engine automates the toil — updates, reminders, field changes — triggered by incident eventsshould-have
Incident response — stories about incident response in this arenaIncident response
Stories about incident response in this arena
- on-call engineerDeclare and run an incident from chat — Slack or Teams — with channels, roles, and updates created for memust-have
- sreThe incident timeline is captured automatically — alerts, actions, and chat decisions — and I can edit or annotate it afterwardsshould-have
- engineering leaderInternal stakeholders get structured incident updates they can subscribe to, without joining the war roomnice-to-have
- engineering leaderIncidents carry defined roles (commander, comms lead) and task checklists so response stays coordinated under pressurenice-to-have
Integrations observability — stories about integrations observability in this arenaIntegrations observability
Stories about integrations observability in this arena
- sreFirst-party integrations cover my observability stack — Datadog, Grafana, Prometheus, CloudWatch, Sentry — with documented setupmust-have
- sreThe platform integrates with the tools around the incident — Jira, Slack, Teams, Zoom, GitHub — so state flows both waysshould-have
Mobile experience — stories about mobile experience in this arenaMobile experience
Stories about mobile experience in this arena
- on-call engineerA full mobile app lets me acknowledge, escalate, and resolve from my phone at 3amshould-have
On call scheduling — stories about on call scheduling in this arenaOn call scheduling
Stories about on call scheduling in this arena
- sreBuild on-call schedules with rotations, layers, time zones, and round-robin coverage that match how my teams actually workmust-have
- on-call engineerTake an override, swap a shift, or request coverage without an admin rebuilding the scheduleshould-have
- on-call engineerMy shifts sync to my personal calendar via a feed so I always know when I'm on the hooknice-to-have
Openness — open source, data portability, and self-hosting storiesOpenness
Open source, data portability, and self-hosting stories
- ai-native userExport all of my data in open formats and leavemust-have
- ai-native userSelf-host the core productmust-have
- ai-native userDo everything through the API that I can do in the UIshould-have
- ai-native userRead the product's source under an open licenseshould-have
Postmortems learning — stories about postmortems learning in this arenaPostmortems learning
Stories about postmortems learning in this arena
- engineering leaderPostmortems follow a real workflow — templates, drafting from the timeline, review, and publicationmust-have
- engineering leaderFollow-up actions from incidents are tracked to completion and sync to our issue trackershould-have
Privacy posture — data-handling and privacy storiesPrivacy posture
Data-handling and privacy stories
- ai-native userPrevent my data from being used to train AI modelsmust-have
- ai-native userChoose where my data is stored (region/residency)should-have
- ai-native userControl data retention and deletionshould-have
- ai-native userOpt out of telemetry and usage trackingshould-have
Status communication — stories about status communication in this arenaStatus communication
Stories about status communication in this arena
- engineering leaderPublish a hosted public status page — custom domain, subscriber notifications — driven from incident stateshould-have
- engineering leaderRun private or internal status pages with access control for customer-specific or employee-only audiencesnice-to-have
Appendix: recorded probes
Hands-on probe recordings — transcripts/videos a human can replay, the strongest evidence tier. Watch them at https://ultrametric.ai/productarena/proofs
- Better Stack
curl -si https://uptime.betterstack.com/api/v2/monitors | head -4terminal · recorded 2026-09-07 · exit 0 - FireHydrant
printf '<jsonrpc initialize>' | npx -y firehydrant-mcp start --transport stdio # full keyless stdio handshaketerminal · recorded 2026-09-07 · exit 0 - incident.io
curl -si -X POST https://mcp.incident.io/mcp -H 'Content-Type: application/json' -d '<jsonrpc initialize>'terminal · recorded 2026-09-07 · exit 0 - incident.io
curl -s https://docs.incident.io/openapi/tags/incidents-v2.json | head -8terminal · recorded 2026-09-07 · exit 0 - PagerDuty
curl -si -X POST https://events.pagerduty.com/v2/enqueue -H 'Content-Type: application/json' -d '{}' # live endpoint answers with a structured validation errorterminal · recorded 2026-09-07 · exit 0 - PagerDuty
curl -si -X POST https://mcp.pagerduty.com/mcp -H 'Content-Type: application/json' -d '<jsonrpc initialize>'terminal · recorded 2026-09-07 · exit 0 - Rootly
curl -si -X POST https://mcp.rootly.com/mcp -H 'Content-Type: application/json' -d '<jsonrpc initialize>'terminal · recorded 2026-09-07 · exit 0 - Rootly
printf '<jsonrpc initialize>' | uvx rootly-mcp-server # pypi install + server_start audit line, then documented ROOTLY_API_TOKEN gateterminal · recorded 2026-09-07 · exit 0
Cite as: ProductArena by Ultrametric Inc, Incident Management & On-call arena, rankings as of 2026-09-07 — https://ultrametric.ai/productarena/arena/incident-management
License: © 2026 Ultrametric Inc. Brief quotation of individual verdicts, scores, or evidence excerpts is permitted with attribution to "ProductArena by Ultrametric Inc (ultrametric.ai/productarena)", as is use of the data to evaluate, contest, or contribute corrections. Bulk copying, redistribution, or use to build competing datasets requires prior written permission (see DATA-LICENSE in the repository).
No liability: rankings, verdicts, and scores are research outputs derived from the cited evidence at a point in time, provided "as is", without warranties. Ultrametric Inc accepts no responsibility for procurement, purchasing, or other decisions made in reliance on them — verify against the cited evidence before acting (https://ultrametric.ai/productarena/terms).