How Rollbar’s scores are calculated

The full audit trail, recomputed from the verdict data at build time through the same code that produced the leaderboard: verdict × quality × story weight per cell, cells sum to dimension scores, dimensions blend into the PA Score. Every number on the product page is reproducible from this page alone; for why the formula looks like this, see the methodology.

verdict factors: full ×1.0 · partial ×0.6 · disputed ×0.3 · none ×0.0 · n/a excluded from both sides · cell points = weight × quality × factor · cell max = weight × 10

PA Score26/100

Agent-ready 44.9 × 0.30 = 13.47

API quality 5.1 × 0.20 = 1.02

Openness 12.6 × 0.20 = 2.52

Built-in AI 34.7 × 0.15 = 5.21

Automation 27.6 × 0.15 = 4.14

(13.47 + 1.02 + 2.52 + 5.21 + 4.14) ÷ (0.30 + 0.20 + 0.20 + 0.15 + 0.15) = 26.35 ÷ 1.00 = 26.4

Scores are stored to 1 decimal; the product page’s pills round to whole numbers for display. Each dimension below shows the stories, verdicts, and cited evidence behind its number.

Agent-ready44.9/100×0.30 of the PA blend

Outside-in: can YOUR agent reach and drive this product — API, MCP, CLI, headless runs, agent docs.

Point an agent at llms.txt or agent-oriented docsweight 2

2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max

[probe] https://docs.rollbar.com/llms.txt“PROBE llms.txt: HTTP 200 at https://docs.rollbar.com/llms.txt # Rollbar Docs Documentation > Rollbar docs help developers install SDKs, configure error monitoring, manage projects,”
[claimed-docs] https://docs.rollbar.com/docs/mcp-server-setup“An MCP server is a component of the Model Context Protocol (MCP), which is an open standard for connecting AI systems with external data sources and tools.”
[probe] https://docs.rollbar.com/docs/mcp-server-setup“official MCP server documented at https://docs.rollbar.com/docs/mcp-server-setup”
[probe] https://docs.rollbar.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.rollbar.com/openapi.json, https://docs.rollbar.com/swagger.json, https://docs.rollbar.com/api/openapi.json, https://docs.rollbar.com/.well-known/openapi.json)”

Run the product headlessly / in CI for automationweight 2

2 (weight) × 5 (quality) × 0.6 (partial) = 6.0 of 20 max

[claimed-docs] https://docs.rollbar.com/docs/deploy-tracking“If you notify Rollbar every time you deploy or release your app, you'll unlock several features that will help your debugging process.”
[claimed-docs] https://docs.rollbar.com/reference“The Rollbar API provides a RESTful interface to much of the data in the system.”
[claimed-docs] https://docs.rollbar.com/docs/access-tokens“Newly created tokens will now be encrypted and will no longer be accessible via the UI or API after creation.”
[claimed-docs] https://docs.rollbar.com/docs/deploy-tracking“If you notify Rollbar every time you deploy or release your app, you'll unlock several features that will help your debuggi”
[claimed-docs] https://docs.rollbar.com/reference“The Rollbar API provides a RESTful interface to much of the data in the system. It is used by our official SDKs to report errors/exceptions, deploys, and other mes”
[claimed-docs] https://docs.rollbar.com/docs/access-tokens“Managing Account & Project Access Tokens”

Plug MCP servers into this product so it can use their toolsweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

[claimed-docs] https://docs.rollbar.com/docs/mcp-server-setup“An MCP server is a component of the Model Context Protocol (MCP), which is an open standard for connecting AI systems with external data sources and tools.”
[probe] https://docs.rollbar.com/docs/mcp-server-setup“official MCP server documented at https://docs.rollbar.com/docs/mcp-server-setup”
[claimed-docs] https://docs.rollbar.com/docs/root-cause-analysis“Resolve is an AI agent in Rollbar that takes a production error and produces a pull request you can review.”

Connect an agent via an official MCP serverweight 3

3 (weight) × 7 (quality) × 1.0 (full) = 21.0 of 30 max

[claimed-docs] https://docs.rollbar.com/docs/mcp-server-setup“An MCP server is a component of the Model Context Protocol (MCP), which is an open standard for connecting AI systems with external data sources and tools.”
[probe] https://docs.rollbar.com/docs/mcp-server-setup“official MCP server documented at https://docs.rollbar.com/docs/mcp-server-setup”

Use an official CLIweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Drive the product through a documented public APIweight 3

3 (weight) × 8 (quality) × 1.0 (full) = 24.0 of 30 max

[claimed-docs] https://docs.rollbar.com/reference“The Rollbar API provides a RESTful interface to much of the data in the system.”
[claimed-docs] https://docs.rollbar.com/reference/list-all-items“Get the first page of items that are error or critical, in the production environment”
[claimed-docs] https://docs.rollbar.com/reference“The Rollbar API provides a RESTful interface to much of the data in the system. It is used by our official SDKs to report errors/exceptions, deploys, and other mes”
[claimed-docs] https://docs.rollbar.com/reference/list-all-items“Get the 101st through 199th active items:”
[claimed-docs] https://docs.rollbar.com/docs/access-tokens“Newly created tokens will now be encrypted and will no longer be accessible via the UI or API after creation.”
[claimed-docs] https://docs.rollbar.com/docs/access-tokens“Managing Account & Project Access Tokens”
[claimed-docs] https://docs.rollbar.com/docs/mcp-server-setup“An MCP server is a component of the Model Context Protocol (MCP), which is an open standard for connecting AI systems with external data sources and tools.”
[probe] https://docs.rollbar.com/docs/mcp-server-setup“official MCP server documented at https://docs.rollbar.com/docs/mcp-server-setup”
[probe] https://docs.rollbar.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.rollbar.com/openapi.json, https://docs.rollbar.com/swagger.json, https://docs.rollbar.com/api/openapi.json, https://docs.rollbar.com/.well-known/openapi.json)”

Issue scoped/least-privilege API credentials for an agentweight 2

2 (weight) × 5 (quality) × 0.6 (partial) = 6.0 of 20 max

[claimed-docs] https://docs.rollbar.com/docs/preventing-client-side-access-token-abuse.md“Use a separate client-side token with post_client_item scope only. (This is the default post_client_item token.) The client-side access token can only be used to send events and only from client-side platforms.”
[claimed-docs] https://docs.rollbar.com/docs/access-tokens“Newly created tokens will now be encrypted and will no longer be accessible via the UI or API after creation.”
[claimed-docs] https://docs.rollbar.com/docs/access-tokens“Managing Account & Project Access Tokens”
[claimed-docs] https://docs.rollbar.com/docs/mcp-server-setup“An MCP server is a component of the Model Context Protocol (MCP), which is an open standard for connecting AI systems with external data sources and tools.”
[community] https://news.ycombinator.com/item?id=37439926“It looks like every project by default has a standing token with read privileges. It never expires and its not opt-in. Unless I'm missing something, that seems ridiculous.”

Build against official SDKsweight 2

2 (weight) × 6 (quality) × 0.6 (partial) = 7.2 of 20 max

[claimed-docs] https://docs.rollbar.com“Rollbar supports over a dozen platforms, and our community has built many more.”
[github] https://github.com/rollbar/rollbar.js“Cross platform: Rollbar.js supports both server-side and client-side Javascript, including frameworks such as React, Angular, Express, Next.js and more.”
[claimed-docs] https://docs.rollbar.com/reference“The Rollbar API provides a RESTful interface to much of the data in the system. It is used by our official SDKs to report errors/exceptions, deploys, and other mes”
[claimed-docs] https://docs.rollbar.com/reference“The Rollbar API provides a RESTful interface to much of the data in the system.”

Subscribe to events via webhooksweight 2

2 (weight) × 7 (quality) × 1.0 (full) = 14.0 of 20 max

[claimed-docs] https://docs.rollbar.com/docs/webhooks“You can set up webhooks to make Rollbar push data to any arbitrary external service.”
[claimed-docs] https://docs.rollbar.com/docs/sending-discord-notifications-using-webhooks-and-hookdeck.md“Using Hookdeck, you can transform a Rollbar Webhook to one that Discord can display”

Agent-ready = 94.2 ÷ 210 × 100 = 44.9

API quality5.1/100×0.20 of the PA blend

The programmable surface once an agent is there — machine-readable spec, interactive docs, sandbox, versioning discipline.

Explore an interactive API reference with runnable examplesweight 2

2 (weight) × 3 (quality) × 0.6 (partial) = 3.6 of 20 max

[claimed-docs] https://docs.rollbar.com/reference“The Rollbar API provides a RESTful interface to much of the data in the system.”
[claimed-docs] https://docs.rollbar.com/reference/list-all-items“Get the first page of items that are error or critical, in the production environment”
[claimed-docs] https://docs.rollbar.com/reference“The Rollbar API provides a RESTful interface to much of the data in the system. It is used by our official SDKs to report errors/exceptions, deploys, and other mes”
[claimed-docs] https://docs.rollbar.com/reference/list-all-items“Get the 101st through 199th active items:”
[probe] https://docs.rollbar.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.rollbar.com/openapi.json, https://docs.rollbar.com/swagger.json, https://docs.rollbar.com/api/openapi.json, https://docs.rollbar.com/.well-known/openapi.json)”

Download a machine-readable API spec (OpenAPI or equivalent)weight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

[claimed-docs] https://docs.rollbar.com/reference“The Rollbar API provides a RESTful interface to much of the data in the system.”
[claimed-docs] https://docs.rollbar.com/reference“The Rollbar API provides a RESTful interface to much of the data in the system. It is used by our official SDKs to report errors/exceptions, deploys, and other mes”
[probe] https://docs.rollbar.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.rollbar.com/openapi.json, https://docs.rollbar.com/swagger.json, https://docs.rollbar.com/api/openapi.json, https://docs.rollbar.com/.well-known/openapi.json)”

Test against a sandbox environment without touching production dataweight 1

1 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 10 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Rely on versioned APIs with a documented deprecation policyweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

[claimed-docs] https://docs.rollbar.com/reference“The Rollbar API provides a RESTful interface to much of the data in the system.”
[claimed-docs] https://docs.rollbar.com/reference“The Rollbar API provides a RESTful interface to much of the data in the system. It is used by our official SDKs to report errors/exceptions, deploys, and other mes”
[probe] https://docs.rollbar.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.rollbar.com/openapi.json, https://docs.rollbar.com/swagger.json, https://docs.rollbar.com/api/openapi.json, https://docs.rollbar.com/.well-known/openapi.json)”

API quality = 3.6 ÷ 70 × 100 = 5.1

Openness12.6/100×0.20 of the PA blend

Can you leave, inspect, or self-host — data export, open source, portability.

Do everything through the API that I can do in the UIweight 2

2 (weight) × 6 (quality) × 0.6 (partial) = 7.2 of 20 max

[claimed-docs] https://docs.rollbar.com/reference“The Rollbar API provides a RESTful interface to much of the data in the system.”
[claimed-docs] https://docs.rollbar.com/reference/list-all-items“Get the first page of items that are error or critical, in the production environment”
[claimed-docs] https://docs.rollbar.com/reference“The Rollbar API provides a RESTful interface to much of the data in the system. It is used by our official SDKs to report errors/exceptions, deploys, and other mes”
[claimed-docs] https://docs.rollbar.com/reference/list-all-items“Get the 101st through 199th active items:”
[claimed-docs] https://docs.rollbar.com/docs/mcp-server-setup“An MCP server is a component of the Model Context Protocol (MCP), which is an open standard for connecting AI systems with external data sources and tools.”
[probe] https://docs.rollbar.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.rollbar.com/openapi.json, https://docs.rollbar.com/swagger.json, https://docs.rollbar.com/api/openapi.json, https://docs.rollbar.com/.well-known/openapi.json)”
[probe] https://docs.rollbar.com/docs/mcp-server-setup“official MCP server documented at https://docs.rollbar.com/docs/mcp-server-setup”

Export all of my data in open formats and leaveweight 3

3 (weight) × 3 (quality) × 0.6 (partial) = 5.4 of 30 max

[claimed-docs] https://docs.rollbar.com/reference“The Rollbar API provides a RESTful interface to much of the data in the system.”
[claimed-docs] https://docs.rollbar.com/reference/list-all-items“Get the first page of items that are error or critical, in the production environment”
[claimed-docs] https://docs.rollbar.com/reference“The Rollbar API provides a RESTful interface to much of the data in the system. It is used by our official SDKs to report errors/exceptions, deploys, and other mes”
[claimed-docs] https://docs.rollbar.com/docs/data-retention“Rollbar data retention period can be customized on all paid plans.”

Read the product's source under an open licenseweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Self-host the core productweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Openness = 12.6 ÷ 100 × 100 = 12.6

Built-in AI34.7/100×0.15 of the PA blend

Inside-out: how agentic the product itself is for its users — built-in assistants, autonomous features.

Get AI-generated insights and suggestions from my data inside the productweight 2

2 (weight) × 6 (quality) × 0.6 (partial) = 7.2 of 20 max

[claimed-docs] https://docs.rollbar.com/docs/root-cause-analysis“Resolve is an AI agent in Rollbar that takes a production error and produces a pull request you can review.”
[claimed-docs] https://docs.rollbar.com/docs/root-cause-analysis“Resolve is an AI agent in Rollbar that takes a prod”
[claimed-docs] https://docs.rollbar.com/docs/mcp-server-setup“An MCP server is a component of the Model Context Protocol (MCP), which is an open standard for connecting AI systems with external data sources and tools.”
[probe] https://docs.rollbar.com/docs/mcp-server-setup“official MCP server documented at https://docs.rollbar.com/docs/mcp-server-setup”

Set up automations that run autonomously in the backgroundweight 2

2 (weight) × 6 (quality) × 0.6 (partial) = 7.2 of 20 max

[claimed-docs] https://docs.rollbar.com/docs/root-cause-analysis“Resolve is an AI agent in Rollbar that takes a production error and produces a pull request you can review.”
[claimed-docs] https://docs.rollbar.com/docs/root-cause-analysis“Resolve is an AI agent in Rollbar that takes a prod”
[claimed-docs] https://docs.rollbar.com/docs/mcp-server-setup“An MCP server is a component of the Model Context Protocol (MCP), which is an open standard for connecting AI systems with external data sources and tools.”
[probe] https://docs.rollbar.com/docs/mcp-server-setup“official MCP server documented at https://docs.rollbar.com/docs/mcp-server-setup”
[claimed-docs] https://docs.rollbar.com/docs/webhooks“You can set up webhooks to make Rollbar push data to any arbitrary external service.”

Delegate tasks to a built-in AI assistant inside the productweight 3

3 (weight) × 6 (quality) × 0.6 (partial) = 10.8 of 30 max

[claimed-docs] https://docs.rollbar.com/docs/root-cause-analysis“Resolve is an AI agent in Rollbar that takes a production error and produces a pull request you can review.”
[claimed-docs] https://docs.rollbar.com/docs/root-cause-analysis“Resolve is an AI agent in Rollbar that takes a prod”

Operate the product with natural-language commandsweight 2

2 (weight) × 5 (quality) × 0.6 (partial) = 6.0 of 20 max

[claimed-docs] https://docs.rollbar.com/docs/mcp-server-setup“An MCP server is a component of the Model Context Protocol (MCP), which is an open standard for connecting AI systems with external data sources and tools.”
[probe] https://docs.rollbar.com/docs/mcp-server-setup“official MCP server documented at https://docs.rollbar.com/docs/mcp-server-setup”
[claimed-docs] https://docs.rollbar.com/docs/root-cause-analysis“Resolve is an AI agent in Rollbar that takes a production error and produces a pull request you can review.”

Built-in AI = 31.2 ÷ 90 × 100 = 34.7

Automation27.6/100×0.15 of the PA blend

Depth of automation primitives — rules, scheduling, bulk operations, webhooks.

Perform bulk operations across many items at onceweight 2

2 (weight) × 4 (quality) × 0.6 (partial) = 4.8 of 20 max

[claimed-docs] https://docs.rollbar.com/reference“The Rollbar API provides a RESTful interface to much of the data in the system.”
[claimed-docs] https://docs.rollbar.com/reference/list-all-items“Get the first page of items that are error or critical, in the production environment”
[claimed-docs] https://docs.rollbar.com/docs/merge-items“Item merging allows you to combine multiple items into one 'group' for easier management and more accurate metrics.”
[claimed-docs] https://docs.rollbar.com/docs/webhooks“You can set up webhooks to make Rollbar push data to any arbitrary external service.”
[probe] https://docs.rollbar.com/docs/mcp-server-setup“official MCP server documented at https://docs.rollbar.com/docs/mcp-server-setup”

Define rules that trigger actions automatically on eventsweight 3

3 (weight) × 5 (quality) × 0.6 (partial) = 9.0 of 30 max

[claimed-docs] https://docs.rollbar.com/docs/webhooks“You can set up webhooks to make Rollbar push data to any arbitrary external service.”
[claimed-docs] https://docs.rollbar.com/docs/notifications“Rollbar supports email and many messaging and incident management tools where your team can get notified about errors and important events by real-time alerts.”
[claimed-docs] https://docs.rollbar.com/docs/slack“Click **Connect with Slack**”
[claimed-docs] https://docs.rollbar.com/docs/pagerduty“Rollbar can create, update, and resolve Incidents in PagerDuty.”
[claimed-docs] https://docs.rollbar.com/docs/notifications“Rollbar provides real-time alerts through the project-level Notifications menu.”

Schedule recurring jobs or workflowsweight 2

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Version, review, and roll back my automationsweight 1

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Automation = 13.8 ÷ 50 × 100 = 27.6