How Orb’s scores are calculated
The full audit trail, recomputed from the verdict data at build time through the same code that produced the leaderboard: verdict × quality × story weight per cell, cells sum to dimension scores, dimensions blend into the PA Score. Every number on the product page is reproducible from this page alone; for why the formula looks like this, see the methodology.
verdict factors: full ×1.0 · partial ×0.6 · disputed ×0.3 · none ×0.0 · n/a excluded from both sides · cell points = weight × quality × factor · cell max = weight × 10
PA Score18/100
Agent-ready 33.1 × 0.30 = 9.93
API quality 0.0 × 0.20 = 0.00
Openness 14.4 × 0.20 = 2.88
Built-in AI 5.3 × 0.15 = 0.79
Automation 26.6 × 0.15 = 3.99
(9.93 + 0.00 + 2.88 + 0.79 + 3.99) ÷ (0.30 + 0.20 + 0.20 + 0.15 + 0.15) = 17.59 ÷ 1.00 = 17.6
Scores are stored to 1 decimal; the product page’s pills round to whole numbers for display. Each dimension below shows the stories, verdicts, and cited evidence behind its number.
Agent-ready33.1/100×0.30 of the PA blend
Outside-in: can YOUR agent reach and drive this product — API, MCP, CLI, headless runs, agent docs.
Point an agent at llms.txt or agent-oriented docsweight 2
2 (weight) × 9 (quality) × 1.0 (full) = 18.0 of 20 max
- [probe] https://docs.withorb.com/llms.txt“PROBE llms.txt: HTTP 200 at https://docs.withorb.com/llms.txt # Orb - [Overview](https://docs.withorb.com/overview.md) - [What is Orb?](https://docs.withorb.com/introduction.md) - [”
Run the product headlessly / in CI for automationweight 2
2 (weight) × 3 (quality) × 0.6 (partial) = 3.6 of 20 max
- [claimed-docs] https://docs.withorb.com/product-catalog/creating-subscriptions.md“You can create a subscription either from the Orb UI or via the subscription creation API endpoint.”
- [claimed-docs] https://docs.withorb.com/events-and-metrics/event-ingestion.md“Orb provides per-event idempotency through the API to guarantee that duplicates are never processed within the account grace period.”
- [claimed-docs] https://docs.withorb.com/integrations-and-exports/webhooks.md“Orb provides a powerful, real-time webhooks system that can issue calls to an endpoint of your choosing when certain events take place.”
Plug MCP servers into this product so it can use their toolsweight 3
n/a — not applicable to this product: excluded from numerator and denominator
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
Connect an agent via an official MCP serverweight 3
3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
Use an official CLIweight 2
2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
Drive the product through a documented public APIweight 3
3 (weight) × 8 (quality) × 1.0 (full) = 24.0 of 30 max
- [claimed-docs] https://docs.withorb.com/product-catalog/creating-subscriptions.md“You can create a subscription either from the Orb UI or via the subscription creation API endpoint.”
- [claimed-docs] https://docs.withorb.com/api-reference/error-responses.md“If you are processing an error programmatically, consider using the type URI along with the status code.”
- [claimed-docs] https://docs.withorb.com/events-and-metrics/event-ingestion.md“Orb provides per-event idempotency through the API to guarantee that duplicates are never processed within the account grace period.”
- [claimed-docs] https://docs.withorb.com/integrations-and-exports/webhooks.md“Orb provides a powerful, real-time webhooks system that can issue calls to an endpoint of your choosing when certain events take place.”
- [probe] https://docs.withorb.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.withorb.com/openapi.json, https://docs.withorb.com/swagger.json, https://docs.withorb.com/api/openapi.json, https://docs.withorb.com/.well-known/openapi.json)”
Issue scoped/least-privilege API credentials for an agentweight 2
2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
Build against official SDKsweight 2
2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max
- [claimed-docs] https://docs.withorb.com/product-catalog/creating-subscriptions.md“You can create a subscription either from the Orb UI or via the subscription creation API endpoint.”
- [claimed-docs] https://docs.withorb.com/api-reference/error-responses.md“If you are processing an error programmatically, consider using the type URI along with the status code.”
- [probe] https://docs.withorb.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.withorb.com/openapi.json, https://docs.withorb.com/swagger.json, https://docs.withorb.com/api/openapi.json, https://docs.withorb.com/.well-known/openapi.json)”
Subscribe to events via webhooksweight 2
2 (weight) × 7 (quality) × 1.0 (full) = 14.0 of 20 max
- [claimed-docs] https://docs.withorb.com/integrations-and-exports/webhooks.md“Orb provides a powerful, real-time webhooks system that can issue calls to an endpoint of your choosing when certain events take place.”
Agent-ready = 59.6 ÷ 180 × 100 = 33.1
API quality0.0/100×0.20 of the PA blend
The programmable surface once an agent is there — machine-readable spec, interactive docs, sandbox, versioning discipline.
Explore an interactive API reference with runnable examplesweight 2
2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max
- [claimed-docs] https://docs.withorb.com/api-reference/error-responses.md“If you are processing an error programmatically, consider using the type URI along with the status code.”
- [probe] https://docs.withorb.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.withorb.com/openapi.json, https://docs.withorb.com/swagger.json, https://docs.withorb.com/api/openapi.json, https://docs.withorb.com/.well-known/openapi.json)”
Download a machine-readable API spec (OpenAPI or equivalent)weight 2
2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max
- [probe] https://docs.withorb.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.withorb.com/openapi.json, https://docs.withorb.com/swagger.json, https://docs.withorb.com/api/openapi.json, https://docs.withorb.com/.well-known/openapi.json)”
- [probe] https://docs.withorb.com/llms.txt“PROBE llms.txt: HTTP 200 at https://docs.withorb.com/llms.txt # Orb - [Overview](https://docs.withorb.com/overview.md) - [What is Orb?](https://docs.withorb.com/introduction.md) - [”
- [claimed-docs] https://docs.withorb.com/api-reference/error-responses.md“If you are processing an error programmatically, consider using the type URI along with the status code.”
Test against a sandbox environment without touching production dataweight 1
1 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 10 max
- [claimed-docs] https://docs.withorb.com“Test pricing changes against historical usage before deploying.”
Rely on versioned APIs with a documented deprecation policyweight 2
2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
API quality = 0.0 ÷ 70 × 100 = 0.0
Openness14.4/100×0.20 of the PA blend
Can you leave, inspect, or self-host — data export, open source, portability.
Do everything through the API that I can do in the UIweight 2
2 (weight) × 6 (quality) × 0.6 (partial) = 7.2 of 20 max
- [claimed-docs] https://docs.withorb.com/product-catalog/creating-subscriptions.md“You can create a subscription either from the Orb UI or via the subscription creation API endpoint.”
- [claimed-docs] https://docs.withorb.com/product-catalog/editing-subscriptions.md“Subscription Edits let you change a single subscription's pricing configuration ... without changing the plan itself.”
- [claimed-docs] https://docs.withorb.com/data-exports/introduction.md“Orb supports managed, daily exports of resource types to your data destination of your choice.”
- [claimed-docs] https://docs.withorb.com/events-and-metrics/construct-metrics.md“A billable metric consists of event filters as well as an aggregation clause.”
- [claimed-docs] https://docs.withorb.com/integrations-and-exports/webhooks.md“Orb provides a powerful, real-time webhooks system that can issue calls to an endpoint of your choosing when certain events take place.”
- [probe] https://docs.withorb.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.withorb.com/openapi.json, https://docs.withorb.com/swagger.json, https://docs.withorb.com/api/openapi.json, https://docs.withorb.com/.well-known/openapi.json)”
Export all of my data in open formats and leaveweight 3
3 (weight) × 4 (quality) × 0.6 (partial) = 7.2 of 30 max
- [claimed-docs] https://docs.withorb.com/data-exports/introduction.md“Orb supports managed, daily exports of resource types to your data destination of your choice.”
- [claimed-docs] https://docs.withorb.com“Track recognized and deferred revenue with exports for financial systems.”
Read the product's source under an open licenseweight 2
2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
Self-host the core productweight 3
3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
Openness = 14.4 ÷ 100 × 100 = 14.4
Built-in AI5.3/100×0.15 of the PA blend
Inside-out: how agentic the product itself is for its users — built-in assistants, autonomous features.
Get AI-generated insights and suggestions from my data inside the productweight 2
2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
Set up automations that run autonomously in the backgroundweight 2
2 (weight) × 4 (quality) × 0.6 (partial) = 4.8 of 20 max
- [claimed-docs] https://docs.withorb.com/financial-operations/essential-dunning.md“Orb will automatically retry the charge after 1, 4, and 9 days from the initial payment attempt.”
- [claimed-docs] https://docs.withorb.com/invoicing/invoice-issuance.md“Automatic Issuance ... Invoices auto-issue after grace periods and buffers ... Manual Issuance ... All invoices require manual approval before issuance”
- [claimed-docs] https://docs.withorb.com“Trigger alerts at usage thresholds to power product-led growth.”
- [claimed-docs] https://docs.withorb.com/integrations-and-exports/webhooks.md“Orb provides a powerful, real-time webhooks system that can issue calls to an endpoint of your choosing when certain events take place.”
Delegate tasks to a built-in AI assistant inside the productweight 3
3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
Operate the product with natural-language commandsweight 2
2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max
- [claimed-docs] https://docs.withorb.com/product-catalog/creating-subscriptions.md“You can create a subscription either from the Orb UI or via the subscription creation API endpoint.”
- [claimed-docs] https://docs.withorb.com“Build complex billing metrics with full SQL control.”
- [probe] https://docs.withorb.com/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.withorb.com/openapi.json, https://docs.withorb.com/swagger.json, https://docs.withorb.com/api/openapi.json, https://docs.withorb.com/.well-known/openapi.json)”
Built-in AI = 4.8 ÷ 90 × 100 = 5.3
Automation26.6/100×0.15 of the PA blend
Depth of automation primitives — rules, scheduling, bulk operations, webhooks.
Perform bulk operations across many items at onceweight 2
2 (weight) × 5 (quality) × 0.6 (partial) = 6.0 of 20 max
- [claimed-docs] https://docs.withorb.com“Handle 1M+ billing events per second in production deployments.”
- [claimed-docs] https://docs.withorb.com/migration-guides/stripe-billing.md“If you want Orb to generate accurate invoices or analytics based on prior usage data, you can backfill events.”
- [claimed-docs] https://docs.withorb.com/events-and-metrics/event-ingestion.md“Orb provides per-event idempotency through the API to guarantee that duplicates are never processed within the account grace period.”
- [claimed-docs] https://docs.withorb.com/data-exports/introduction.md“Orb supports managed, daily exports of resource types to your data destination of your choice.”
Define rules that trigger actions automatically on eventsweight 3
3 (weight) × 5 (quality) × 0.6 (partial) = 9.0 of 30 max
- [claimed-docs] https://docs.withorb.com“Trigger alerts at usage thresholds to power product-led growth.”
- [claimed-docs] https://docs.withorb.com/integrations-and-exports/webhooks.md“Orb provides a powerful, real-time webhooks system that can issue calls to an endpoint of your choosing when certain events take place.”
- [claimed-docs] https://docs.withorb.com/financial-operations/essential-dunning.md“Orb will automatically retry the charge after 1, 4, and 9 days from the initial payment attempt.”
- [claimed-docs] https://docs.withorb.com/invoicing/invoice-issuance.md“Automatic Issuance ... Invoices auto-issue after grace periods and buffers ... Manual Issuance ... All invoices require manual approval before issuance”
Schedule recurring jobs or workflowsweight 2
2 (weight) × 3 (quality) × 0.6 (partial) = 3.6 of 20 max
- [claimed-docs] https://docs.withorb.com/data-exports/introduction.md“Orb supports managed, daily exports of resource types to your data destination of your choice.”
- [claimed-docs] https://docs.withorb.com/financial-operations/essential-dunning.md“Orb will automatically retry the charge after 1, 4, and 9 days from the initial payment attempt.”
- [claimed-docs] https://docs.withorb.com/invoicing/invoice-issuance.md“Automatic Issuance ... Invoices auto-issue after grace periods and buffers ... Manual Issuance ... All invoices require manual approval before issuance”
- [claimed-docs] https://docs.withorb.com/integrations-and-exports/webhooks.md“Orb provides a powerful, real-time webhooks system that can issue calls to an endpoint of your choosing when certain events take place.”
Version, review, and roll back my automationsweight 1
n/a — not applicable to this product: excluded from numerator and denominator
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
Automation = 18.6 ÷ 70 × 100 = 26.6