Skip to content
ProductArena

How BugSnag’s scores are calculated

The full audit trail, recomputed from the verdict data at build time through the same code that produced the leaderboard: verdict × quality × story weight per cell, cells sum to dimension scores, dimensions blend into the PA Score. Every number on the product page is reproducible from this page alone; for why the formula looks like this, see the methodology.

verdict factors: full ×1.0 · partial ×0.6 · disputed ×0.3 · none ×0.0 · n/a excluded from both sides · cell points = weight × quality × factor · cell max = weight × 10

PA Score21/100

Agent-ready 36.7 × 0.30 = 11.01

API quality 0.0 × 0.20 = 0.00

Openness 24.6 × 0.20 = 4.92

Built-in AI 13.3 × 0.15 = 2.00

Automation 18.0 × 0.15 = 2.70

(11.01 + 0.00 + 4.92 + 2.00 + 2.70) ÷ (0.30 + 0.20 + 0.20 + 0.15 + 0.15) = 20.63 ÷ 1.00 = 20.6

Scores are stored to 1 decimal; the product page’s pills round to whole numbers for display. Each dimension below shows the stories, verdicts, and cited evidence behind its number.

Agent-ready36.7/100×0.30 of the PA blend

Outside-in: can YOUR agent reach and drive this product — API, MCP, CLI, headless runs, agent docs.

Point an agent at llms.txt or agent-oriented docsweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [probe] https://docs.bugsnag.com/llms.txt“PROBE llms.txt: HTTP 404 at https://docs.bugsnag.com/llms.txt”

Run the product headlessly / in CI for automationweight 2

2 (weight) × 5 (quality) × 0.6 (partial) = 6.0 of 20 max

  • [claimed-docs] https://docs.bugsnag.com/build-integrations/bugsnag-cli/“Allows you to create a build within BugSnag to enrich releases shown in the BugSnag dashboard.”
  • [probe] https://docs.bugsnag.com/build-integrations/bugsnag-cli/“official CLI documented at https://docs.bugsnag.com/build-integrations/bugsnag-cli/”
  • [claimed-docs] https://docs.bugsnag.com/platforms/javascript/source-maps/“The BugSnag error monitoring SDK supports using source maps to reverse JavaScript transformations.”

Plug MCP servers into this product so it can use their toolsweight 3

n/a — not applicable to this product: excluded from numerator and denominator

  • [claimed-docs] https://docs.bugsnag.com/whats-new/“Self Hosted MCP Server OAuth 2.0 support — securely connect to BugSnag via browser-based login instead of manual API tokens.”
  • [probe] https://github.com/SmartBear/smartbear-mcp“official MCP server documented at https://github.com/SmartBear/smartbear-mcp”

Connect an agent via an official MCP serverweight 3

3 (weight) × 6 (quality) × 0.6 (partial) = 10.8 of 30 max

  • [claimed-docs] https://docs.bugsnag.com/whats-new/“Self Hosted MCP Server OAuth 2.0 support — securely connect to BugSnag via browser-based login instead of manual API tokens.”
  • [probe] https://github.com/SmartBear/smartbear-mcp“official MCP server documented at https://github.com/SmartBear/smartbear-mcp”

Use an official CLIweight 2

2 (weight) × 6 (quality) × 1.0 (full) = 12.0 of 20 max

  • [claimed-docs] https://docs.bugsnag.com/build-integrations/bugsnag-cli/“Allows you to create a build within BugSnag to enrich releases shown in the BugSnag dashboard.”
  • [probe] https://docs.bugsnag.com/build-integrations/bugsnag-cli/“official CLI documented at https://docs.bugsnag.com/build-integrations/bugsnag-cli/”

Drive the product through a documented public APIweight 3

3 (weight) × 4 (quality) × 0.6 (partial) = 7.2 of 30 max

  • [claimed-docs] https://docs.bugsnag.com/build-integrations/bugsnag-cli/“Allows you to create a build within BugSnag to enrich releases shown in the BugSnag dashboard.”
  • [probe] https://docs.bugsnag.com/build-integrations/bugsnag-cli/“official CLI documented at https://docs.bugsnag.com/build-integrations/bugsnag-cli/”
  • [claimed-docs] https://docs.bugsnag.com/product/integrations/data-forwarding/webhook/“The webhook plugin allows access to error and project data as soon as they are received and processed by BugSnag.”
  • [probe] https://github.com/SmartBear/smartbear-mcp“official MCP server documented at https://github.com/SmartBear/smartbear-mcp”
  • [probe] https://docs.bugsnag.com/llms.txt“PROBE llms.txt: HTTP 404 at https://docs.bugsnag.com/llms.txt”
  • [probe] https://docs.bugsnag.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.bugsnag.com/openapi.json, https://docs.bugsnag.com/swagger.json, https://docs.bugsnag.com/api/openapi.json, https://docs.bugsnag.com/.well-known/openapi.json)”

Issue scoped/least-privilege API credentials for an agentweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Build against official SDKsweight 2

2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max

  • [github] https://github.com/bugsnag/bugsnag-js“Automatically detect JavaScript errors in the browser, Node.js and React Native, with plugins for React, Vue, Angular, Express, Restify and Koa.”
  • [github] https://github.com/bugsnag/bugsnag-js“Report handled exceptions using Bugsnag.notify()”
  • [claimed-docs] https://docs.bugsnag.com/platforms/“BugSnag provides error reporting libraries for over 50 different platforms, listed in the navigation to the left.”
  • [claimed-docs] https://docs.bugsnag.com/platforms/“BugSnag provides error reporting libraries for over 50 different platforms”
  • [claimed-docs] https://docs.bugsnag.com/platforms/javascript/source-maps/“The BugSnag error monitoring SDK supports using source maps to reverse JavaScript transformations.”
  • [claimed-docs] https://docs.bugsnag.com/build-integrations/bugsnag-cli/“Allows you to create a build within BugSnag to enrich releases shown in the BugSnag dashboard.”
  • [probe] https://docs.bugsnag.com/build-integrations/bugsnag-cli/“official CLI documented at https://docs.bugsnag.com/build-integrations/bugsnag-cli/”

Subscribe to events via webhooksweight 2

2 (weight) × 7 (quality) × 1.0 (full) = 14.0 of 20 max

  • [claimed-docs] https://docs.bugsnag.com/product/integrations/data-forwarding/webhook/“The webhook plugin allows access to error and project data as soon as they are received and processed by BugSnag.”

Agent-ready = 66.0 ÷ 180 × 100 = 36.7

API quality0.0/100×0.20 of the PA blend

The programmable surface once an agent is there — machine-readable spec, interactive docs, sandbox, versioning discipline.

Explore an interactive API reference with runnable examplesweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [probe] https://docs.bugsnag.com/llms.txt“PROBE llms.txt: HTTP 404 at https://docs.bugsnag.com/llms.txt”
  • [probe] https://docs.bugsnag.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.bugsnag.com/openapi.json, https://docs.bugsnag.com/swagger.json, https://docs.bugsnag.com/api/openapi.json, https://docs.bugsnag.com/.well-known/openapi.json)”

Download a machine-readable API spec (OpenAPI or equivalent)weight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [probe] https://docs.bugsnag.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.bugsnag.com/openapi.json, https://docs.bugsnag.com/swagger.json, https://docs.bugsnag.com/api/openapi.json, https://docs.bugsnag.com/.well-known/openapi.json)”

Test against a sandbox environment without touching production dataweight 1

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Rely on versioned APIs with a documented deprecation policyweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [probe] https://docs.bugsnag.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.bugsnag.com/openapi.json, https://docs.bugsnag.com/swagger.json, https://docs.bugsnag.com/api/openapi.json, https://docs.bugsnag.com/.well-known/openapi.json)”
  • [probe] https://docs.bugsnag.com/llms.txt“PROBE llms.txt: HTTP 404 at https://docs.bugsnag.com/llms.txt”

API quality = 0.0 ÷ 60 × 100 = 0.0

Openness24.6/100×0.20 of the PA blend

Can you leave, inspect, or self-host — data export, open source, portability.

Do everything through the API that I can do in the UIweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [probe] https://docs.bugsnag.com/llms.txt“PROBE llms.txt: HTTP 404 at https://docs.bugsnag.com/llms.txt”
  • [probe] https://docs.bugsnag.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.bugsnag.com/openapi.json, https://docs.bugsnag.com/swagger.json, https://docs.bugsnag.com/api/openapi.json, https://docs.bugsnag.com/.well-known/openapi.json)”
  • [claimed-docs] https://docs.bugsnag.com/build-integrations/bugsnag-cli/“Allows you to create a build within BugSnag to enrich releases shown in the BugSnag dashboard.”
  • [probe] https://github.com/SmartBear/smartbear-mcp“official MCP server documented at https://github.com/SmartBear/smartbear-mcp”
  • [github] https://github.com/bugsnag/bugsnag-js“Report handled exceptions using Bugsnag.notify()”

Export all of my data in open formats and leaveweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

  • [claimed-docs] https://docs.bugsnag.com/product/integrations/data-forwarding/webhook/“The webhook plugin allows access to error and project data as soon as they are received and processed by BugSnag.”
  • [probe] https://docs.bugsnag.com/llms.txt“PROBE llms.txt: HTTP 404 at https://docs.bugsnag.com/llms.txt”
  • [probe] https://docs.bugsnag.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.bugsnag.com/openapi.json, https://docs.bugsnag.com/swagger.json, https://docs.bugsnag.com/api/openapi.json, https://docs.bugsnag.com/.well-known/openapi.json)”

Read the product's source under an open licenseweight 2

2 (weight) × 3 (quality) × 0.6 (partial) = 3.6 of 20 max

  • [github] https://github.com/bugsnag/bugsnag-js“Automatically detect JavaScript errors in the browser, Node.js and React Native, with plugins for React, Vue, Angular, Express, Restify and Koa.”
  • [github] https://github.com/bugsnag/bugsnag-js“Report handled exceptions using Bugsnag.notify()”

Self-host the core productweight 3

3 (weight) × 7 (quality) × 1.0 (full) = 21.0 of 30 max

  • [claimed-docs] https://docs.bugsnag.com/on-premise/“BugSnag On-premise offers a self-hosted version of BugSnag for organizations that need or want to manage their own data. It can be run in your existing PCI or HIPAA compliant environments.”

Openness = 24.6 ÷ 100 × 100 = 24.6

Built-in AI13.3/100×0.15 of the PA blend

Inside-out: how agentic the product itself is for its users — built-in assistants, autonomous features.

Get AI-generated insights and suggestions from my data inside the productweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [claimed-docs] https://docs.bugsnag.com/whats-new/“Self Hosted MCP Server OAuth 2.0 support — securely connect to BugSnag via browser-based login instead of manual API tokens.”
  • [probe] https://github.com/SmartBear/smartbear-mcp“official MCP server documented at https://github.com/SmartBear/smartbear-mcp”
  • [claimed-docs] https://docs.bugsnag.com/product/error-grouping/“BugSnag aims to group instances of the same event together to give you a clear view of which issues are having the biggest impact on your users”
  • [claimed-docs] https://docs.bugsnag.com/product/stability/“Monitor the stability of your application and measure progress against stability targets.”

Set up automations that run autonomously in the backgroundweight 2

2 (weight) × 5 (quality) × 0.6 (partial) = 6.0 of 20 max

  • [claimed-docs] https://docs.bugsnag.com/product/automatic-error-assignment/“BugSnag allows you to create rules to automate this process.”
  • [claimed-docs] https://docs.bugsnag.com/product/integrations/data-forwarding/webhook/“The webhook plugin allows access to error and project data as soon as they are received and processed by BugSnag.”
  • [claimed-docs] https://docs.bugsnag.com/product/integrations/“Get alerts in Slack, Microsoft Teams, or your team's communication tool.”
  • [claimed-docs] https://docs.bugsnag.com/product/integrations/slack/“The Slack integration can send notifications to any Slack channel and allows you to take action on errors straight from Slack.”
  • [claimed-docs] https://docs.bugsnag.com/product/integrations/pagerduty/“The PagerDuty integration creates incidents in PagerDuty when error activity is reported to BugSnag.”

Delegate tasks to a built-in AI assistant inside the productweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Operate the product with natural-language commandsweight 2

2 (weight) × 5 (quality) × 0.6 (partial) = 6.0 of 20 max

  • [claimed-docs] https://docs.bugsnag.com/whats-new/“Self Hosted MCP Server OAuth 2.0 support — securely connect to BugSnag via browser-based login instead of manual API tokens.”
  • [probe] https://github.com/SmartBear/smartbear-mcp“official MCP server documented at https://github.com/SmartBear/smartbear-mcp”

Built-in AI = 12.0 ÷ 90 × 100 = 13.3

Automation18.0/100×0.15 of the PA blend

Depth of automation primitives — rules, scheduling, bulk operations, webhooks.

Perform bulk operations across many items at onceweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [claimed-docs] https://docs.bugsnag.com/product/automatic-error-assignment/“BugSnag allows you to create rules to automate this process.”
  • [claimed-docs] https://docs.bugsnag.com/product/error-status-and-actions/“All errors introduced in the last 24 hours”
  • [claimed-docs] https://docs.bugsnag.com/product/event-usage/“Take control over which events we save and count towards your event quota.”
  • [claimed-docs] https://docs.bugsnag.com/product/error-status-and-actions/“All errors except those that are fixed, snoozed or ignored”

Define rules that trigger actions automatically on eventsweight 3

3 (weight) × 5 (quality) × 0.6 (partial) = 9.0 of 30 max

  • [claimed-docs] https://docs.bugsnag.com/product/automatic-error-assignment/“BugSnag allows you to create rules to automate this process.”
  • [claimed-docs] https://docs.bugsnag.com/product/integrations/“Get alerts in Slack, Microsoft Teams, or your team's communication tool.”
  • [claimed-docs] https://docs.bugsnag.com/product/integrations/slack/“The Slack integration can send notifications to any Slack channel and allows you to take action on errors straight from Slack.”
  • [claimed-docs] https://docs.bugsnag.com/product/integrations/pagerduty/“The PagerDuty integration creates incidents in PagerDuty when error activity is reported to BugSnag.”
  • [claimed-docs] https://docs.bugsnag.com/product/integrations/data-forwarding/webhook/“The webhook plugin allows access to error and project data as soon as they are received and processed by BugSnag.”

Schedule recurring jobs or workflowsweight 2

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Version, review, and roll back my automationsweight 1

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Automation = 9.0 ÷ 50 × 100 = 18.0