Skip to content
ProductArena

How Raygun’s scores are calculated

The full audit trail, recomputed from the verdict data at build time through the same code that produced the leaderboard: verdict × quality × story weight per cell, cells sum to dimension scores, dimensions blend into the PA Score. Every number on the product page is reproducible from this page alone; for why the formula looks like this, see the methodology.

verdict factors: full ×1.0 · partial ×0.6 · disputed ×0.3 · none ×0.0 · n/a excluded from both sides · cell points = weight × quality × factor · cell max = weight × 10

PA Score13/100

Agent-ready 12.3 × 0.30 = 3.69

API quality 0.0 × 0.20 = 0.00

Openness 9.0 × 0.20 = 1.80

Built-in AI 25.3 × 0.15 = 3.79

Automation 27.6 × 0.15 = 4.14

(3.69 + 0.00 + 1.80 + 3.79 + 4.14) ÷ (0.30 + 0.20 + 0.20 + 0.15 + 0.15) = 13.43 ÷ 1.00 = 13.4

Scores are stored to 1 decimal; the product page’s pills round to whole numbers for display. Each dimension below shows the stories, verdicts, and cited evidence behind its number.

Agent-ready12.3/100×0.30 of the PA blend

Outside-in: can YOUR agent reach and drive this product — API, MCP, CLI, headless runs, agent docs.

Point an agent at llms.txt or agent-oriented docsweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [probe] https://raygun.com/llms.txt“PROBE llms.txt: HTTP 404 at https://raygun.com/llms.txt”

Run the product headlessly / in CI for automationweight 2

2 (weight) × 4 (quality) × 0.6 (partial) = 4.8 of 20 max

  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/external-access-token/“You'll need your external access token for the following services: Deployments API, Sending dSYMS to Raygun, Uploading ProGuard mapping files, Uploading JavaScript files directly to Raygun”
  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/external-access-token/“You'll need your external access token for the following services:”
  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/exporting-data/“The Raygun Crash Reporting export feature allows customers to export all error groups or selected errors groups from within the Crash Reporting dashboard.”
  • [probe] https://raygun.com/openapi.json“PROBE openapi: all candidate paths 404 (https://raygun.com/openapi.json, https://raygun.com/swagger.json, https://raygun.com/api/openapi.json, https://raygun.com/.well-known/openapi.json)”

Plug MCP servers into this product so it can use their toolsweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

  • [probe] https://github.com/MindscapeHQ/mcp-server-raygun“official MCP server documented at https://github.com/MindscapeHQ/mcp-server-raygun”
  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/ai-resolution/“You can chat with this tool collaboratively. Typing indicators are shown whenever teammates are typing, so it's always clear who is using the chat.”

Connect an agent via an official MCP serverweight 3

3 (weight) × 5 (quality) × 0.6 (partial) = 9.0 of 30 max

  • [probe] https://github.com/MindscapeHQ/mcp-server-raygun“official MCP server documented at https://github.com/MindscapeHQ/mcp-server-raygun”

Use an official CLIweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Drive the product through a documented public APIweight 3

3 (weight) × 4 (quality) × 0.6 (partial) = 7.2 of 30 max

  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/external-access-token/“You'll need your external access token for the following services: Deployments API, Sending dSYMS to Raygun, Uploading ProGuard mapping files, Uploading JavaScript files directly to Raygun”
  • [claimed-docs] https://raygun.com/documentation/language-guides/javascript/crash-reporting/source-maps/“If the automatic option does not work for your application you can also manually upload source maps either through our website using the `JS source map center` or via our API”
  • [probe] https://raygun.com/openapi.json“PROBE openapi: all candidate paths 404 (https://raygun.com/openapi.json, https://raygun.com/swagger.json, https://raygun.com/api/openapi.json, https://raygun.com/.well-known/openapi.json)”
  • [probe] https://github.com/MindscapeHQ/mcp-server-raygun“official MCP server documented at https://github.com/MindscapeHQ/mcp-server-raygun”

Issue scoped/least-privilege API credentials for an agentweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/external-access-token/“You'll need your external access token for the following services: Deployments API, Sending dSYMS to Raygun, Uploading ProGuard mapping files, Uploading JavaScript files directly to Raygun”
  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/external-access-token/“You'll need your external access token for the following services:”
  • [probe] https://github.com/MindscapeHQ/mcp-server-raygun“official MCP server documented at https://github.com/MindscapeHQ/mcp-server-raygun”

Build against official SDKsweight 2

2 (weight) × 4 (quality) × 0.6 (partial) = 4.8 of 20 max

  • [claimed-docs] https://raygun.com/documentation/language-guides/javascript/crash-reporting/source-maps/“If your source maps are publicly available on your website Raygun will fetch them automatically without you having to do anything.”
  • [claimed-docs] https://raygun.com/documentation/language-guides/javascript/crash-reporting/source-maps/“If the automatic option does not work for your application you can also manually upload source maps either through our website using the `JS source map center` or via our API”
  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/external-access-token/“You'll need your external access token for the following services: Deployments API, Sending dSYMS to Raygun, Uploading ProGuard mapping files, Uploading JavaScript files directly to Raygun”
  • [community] https://news.ycombinator.com/item?id=5211350“I've been using the product since Alpha, needless to say I love it... Their .NET support is awesome... WinRT support is GREAT! Picked up a few errors in my Windows Store App!”

Subscribe to events via webhooksweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [claimed-docs] https://raygun.com/documentation/product-guides/apm/integrations/slack/“Select the Slack channel you would like APM trace messages to be sent to and click "Authorize".”
  • [claimed-docs] https://raygun.com/documentation/product-guides/apm/integrations/pagerduty/“Now when an issue is triggered from this rule, a PagerDuty incident will be created and your incident team will be notified by PagerDuty.”
  • [probe] https://raygun.com/llms.txt“PROBE llms.txt: HTTP 404 at https://raygun.com/llms.txt”
  • [probe] https://raygun.com/openapi.json“PROBE openapi: all candidate paths 404 (https://raygun.com/openapi.json, https://raygun.com/swagger.json, https://raygun.com/api/openapi.json, https://raygun.com/.well-known/openapi.json)”

Agent-ready = 25.8 ÷ 210 × 100 = 12.3

API quality0.0/100×0.20 of the PA blend

The programmable surface once an agent is there — machine-readable spec, interactive docs, sandbox, versioning discipline.

Explore an interactive API reference with runnable examplesweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [probe] https://raygun.com/llms.txt“PROBE llms.txt: HTTP 404 at https://raygun.com/llms.txt”
  • [probe] https://raygun.com/openapi.json“PROBE openapi: all candidate paths 404 (https://raygun.com/openapi.json, https://raygun.com/swagger.json, https://raygun.com/api/openapi.json, https://raygun.com/.well-known/openapi.json)”

Download a machine-readable API spec (OpenAPI or equivalent)weight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [probe] https://raygun.com/openapi.json“PROBE openapi: all candidate paths 404 (https://raygun.com/openapi.json, https://raygun.com/swagger.json, https://raygun.com/api/openapi.json, https://raygun.com/.well-known/openapi.json)”
  • [probe] https://raygun.com/llms.txt“PROBE llms.txt: HTTP 404 at https://raygun.com/llms.txt”

Test against a sandbox environment without touching production dataweight 1

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Rely on versioned APIs with a documented deprecation policyweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [probe] https://raygun.com/openapi.json“PROBE openapi: all candidate paths 404 (https://raygun.com/openapi.json, https://raygun.com/swagger.json, https://raygun.com/api/openapi.json, https://raygun.com/.well-known/openapi.json)”
  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/external-access-token/“You'll need your external access token for the following services: Deployments API, Sending dSYMS to Raygun, Uploading ProGuard mapping files, Uploading JavaScript files directly to Raygun”

API quality = 0.0 ÷ 60 × 100 = 0.0

Openness9.0/100×0.20 of the PA blend

Can you leave, inspect, or self-host — data export, open source, portability.

Do everything through the API that I can do in the UIweight 2

2 (weight) × 3 (quality) × 0.6 (partial) = 3.6 of 20 max

  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/external-access-token/“You'll need your external access token for the following services: Deployments API, Sending dSYMS to Raygun, Uploading ProGuard mapping files, Uploading JavaScript files directly to Raygun”
  • [claimed-docs] https://raygun.com/documentation/language-guides/javascript/crash-reporting/source-maps/“If the automatic option does not work for your application you can also manually upload source maps either through our website using the `JS source map center` or via our API”
  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/exporting-data/“The Raygun Crash Reporting export feature allows customers to export all error groups or selected errors groups from within the Crash Reporting dashboard.”
  • [probe] https://raygun.com/llms.txt“PROBE llms.txt: HTTP 404 at https://raygun.com/llms.txt”
  • [probe] https://raygun.com/openapi.json“PROBE openapi: all candidate paths 404 (https://raygun.com/openapi.json, https://raygun.com/swagger.json, https://raygun.com/api/openapi.json, https://raygun.com/.well-known/openapi.json)”

Export all of my data in open formats and leaveweight 3

3 (weight) × 3 (quality) × 0.6 (partial) = 5.4 of 30 max

  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/exporting-data/“The Raygun Crash Reporting export feature allows customers to export all error groups or selected errors groups from within the Crash Reporting dashboard.”
  • [claimed-docs] https://raygun.com/changelog“Aligned **CSV exports** with applied top-level filters. Exported data now matches filtered views for: Most Requested Pages, Tags, Country codes, Other top-level filters”
  • [claimed-docs] https://raygun.com/changelog“Exported data now matches filtered views for:”
  • [probe] https://raygun.com/openapi.json“PROBE openapi: all candidate paths 404 (https://raygun.com/openapi.json, https://raygun.com/swagger.json, https://raygun.com/api/openapi.json, https://raygun.com/.well-known/openapi.json)”

Read the product's source under an open licenseweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Self-host the core productweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Openness = 9.0 ÷ 100 × 100 = 9.0

Built-in AI25.3/100×0.15 of the PA blend

Inside-out: how agentic the product itself is for its users — built-in assistants, autonomous features.

Get AI-generated insights and suggestions from my data inside the productweight 2

2 (weight) × 6 (quality) × 0.6 (partial) = 7.2 of 20 max

  • [claimed-docs] https://raygun.com/platform/crash-reporting“Resolve issues with unprecedented speed and accuracy using AI-powered suggestions on the root cause of errors and how to fix them.”
  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/ai-resolution/“You can chat with this tool collaboratively. Typing indicators are shown whenever teammates are typing, so it's always clear who is using the chat.”
  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/ai-resolution/“This feature allows you to identify and tackle errors swiftly with just a click, using advanced AI technology powered by OpenAI and Azure OpenAI.”

Set up automations that run autonomously in the backgroundweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [claimed-docs] https://raygun.com/platform/crash-reporting“Resolve issues with unprecedented speed and accuracy using AI-powered suggestions on the root cause of errors and how to fix them.”
  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/ai-resolution/“You can chat with this tool collaboratively. Typing indicators are shown whenever teammates are typing, so it's always clear who is using the chat.”
  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/ai-resolution/“This feature allows you to identify and tackle errors swiftly with just a click, using advanced AI technology powered by OpenAI and Azure OpenAI.”
  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/error-statuses/“We provide three types of conditions: Ignore for a specific time period ... Ignore for a specific number of occurrences ... Ignore by rule”
  • [claimed-docs] https://raygun.com/documentation/product-guides/apm/integrations/pagerduty/“Now when an issue is triggered from this rule, a PagerDuty incident will be created and your incident team will be notified by PagerDuty.”

Delegate tasks to a built-in AI assistant inside the productweight 3

3 (weight) × 6 (quality) × 0.6 (partial) = 10.8 of 30 max

  • [claimed-docs] https://raygun.com/platform/crash-reporting“Resolve issues with unprecedented speed and accuracy using AI-powered suggestions on the root cause of errors and how to fix them.”
  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/ai-resolution/“You can chat with this tool collaboratively. Typing indicators are shown whenever teammates are typing, so it's always clear who is using the chat.”
  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/ai-resolution/“Simply press the comment button in the AI chat, and it will create a link back to your conversation to be posted into the comments section.”
  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/ai-resolution/“This feature allows you to identify and tackle errors swiftly with just a click, using advanced AI technology powered by OpenAI and Azure OpenAI.”

Operate the product with natural-language commandsweight 2

2 (weight) × 4 (quality) × 0.6 (partial) = 4.8 of 20 max

  • [claimed-docs] https://raygun.com/platform/crash-reporting“Resolve issues with unprecedented speed and accuracy using AI-powered suggestions on the root cause of errors and how to fix them.”
  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/ai-resolution/“You can chat with this tool collaboratively. Typing indicators are shown whenever teammates are typing, so it's always clear who is using the chat.”
  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/ai-resolution/“This feature allows you to identify and tackle errors swiftly with just a click, using advanced AI technology powered by OpenAI and Azure OpenAI.”
  • [probe] https://github.com/MindscapeHQ/mcp-server-raygun“official MCP server documented at https://github.com/MindscapeHQ/mcp-server-raygun”

Built-in AI = 22.8 ÷ 90 × 100 = 25.3

Automation27.6/100×0.15 of the PA blend

Depth of automation primitives — rules, scheduling, bulk operations, webhooks.

Perform bulk operations across many items at onceweight 2

2 (weight) × 4 (quality) × 0.6 (partial) = 4.8 of 20 max

  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/exporting-data/“The Raygun Crash Reporting export feature allows customers to export all error groups or selected errors groups from within the Crash Reporting dashboard.”
  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/error-statuses/“We provide three types of conditions: Ignore for a specific time period ... Ignore for a specific number of occurrences ... Ignore by rule”
  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/error-statuses/“When you ignore an error group you can specify the condition you would like the error group to be ignored for.”

Define rules that trigger actions automatically on eventsweight 3

3 (weight) × 5 (quality) × 0.6 (partial) = 9.0 of 30 max

  • [claimed-docs] https://raygun.com/documentation/product-guides/apm/integrations/pagerduty/“Now when an issue is triggered from this rule, a PagerDuty incident will be created and your incident team will be notified by PagerDuty.”
  • [claimed-docs] https://raygun.com/documentation/product-guides/apm/integrations/slack/“Select the Slack channel you would like APM trace messages to be sent to and click "Authorize".”
  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/error-statuses/“We provide three types of conditions: Ignore for a specific time period ... Ignore for a specific number of occurrences ... Ignore by rule”
  • [claimed-docs] https://raygun.com/documentation/product-guides/crash-reporting/error-statuses/“When you ignore an error group you can specify the condition you would like the error group to be ignored for.”

Schedule recurring jobs or workflowsweight 2

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Version, review, and roll back my automationsweight 1

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Automation = 13.8 ÷ 50 × 100 = 27.6