Recurly vs Orb
Recurly
Recurly, Inc.
Orb wins · 11–14 (24 drawn)
Agenticness — how well agents can access and operate the productAgenticness
How well agents can access and operate the product
Agent access
ai-native userPoint an agent at llms.txt or agent-oriented docs
weight 2 · round drawnA live llms.txt is confirmed via direct probe (HTTP 200) at docs.recurly.com/llms.txt, and Recurly also documents a public MCP server (Compass) that connects AI dev tools directly to its documentation and API knowledge, giving agents a first-party structured path to Recurly's docs. missing for 10: independent/community confirmation of agents actually using these endpoints successfully.
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.recurly.com/llms.txt # Recurly Documentation Documentation > Welcome to the Recurly Documentation …”
- [probe] “official MCP server documented at https://docs.recurly.com/recurly-subscriptions/docs/compass-public-mcp-server”
- [claimed-docs] “Recurly Compass MCP connects your AI development tools directly to Recurly's documentation and API knowledge.”
A live probe confirms Orb serves a proper llms.txt at docs.withorb.com/llms.txt returning HTTP 200 with structured links to docs, directly satisfying the agent-oriented docs discovery story. Missing for 10: no independent third-party corroboration of agent usage/adoption beyond the probe itself.
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.withorb.com/llms.txt # Orb - [Overview](https://docs.withorb.com/overview.md) - [What is Orb?](htt…”
ai-native userRun the product headlessly / in CI for automation
weight 2 · round to RecurlyRecurly offers a full REST API and official client libraries (Ruby, Node, Python, .NET, Java, PHP, Go) which could be scripted headlessly in CI, but there is no explicit documentation of a CLI, CI/CD examples, or automation-focused tooling for headless/agentic workflows. missing for 10: explicit CI/headless usage examples, a documented CLI tool, evidence of non-interactive/automation-first design beyond generic API/SDK availability.
- [claimed-docs] “Official Client Libraries ### Ruby ... ### Node.js ... ### Python ... ### Dotnet ... ### Java ... ### PHP ... ### Go”
- [claimed-docs] “The Recurly Native Mobile SDKs allow you to integrate recurrent payments in your existing Android or iOS apps in a matter of minutes.”
Orb is API-first for core actions (e.g., subscription creation, event ingestion, webhooks) which implies operations can be automated without the UI, but there is no evidence of a CLI, SDK, or explicit CI/pipeline integration story — the evidence never discusses running Orb 'headlessly' in an automation/CI context. missing for 10: CLI or SDK for automation, explicit CI/pipeline integration examples, documentation framing API use as headless/agentic automation.
- [claimed-docs] “You can create a subscription either from the Orb UI or via the subscription creation API endpoint.”
- [claimed-docs] “Orb provides per-event idempotency through the API to guarantee that duplicates are never processed within the account grace period.”
- [claimed-docs] “Orb provides a powerful, real-time webhooks system that can issue calls to an endpoint of your choosing when certain events take place.”
ai-native userConnect an agent via an official MCP server
weight 3 · round to RecurlyRecurly ships an official 'Compass' MCP server that connects AI dev tools directly to Recurly's documentation and API knowledge, confirmed both in docs and via a live probe of the docs page. However, evidence only describes it as a documentation/knowledge connector, not a full agentic interface for performing billing actions (e.g., managing subscriptions, invoices) via MCP. Missing for 10: evidence of MCP tool-calling for actual account/subscription operations, independent hands-on validation of the MCP server in use.
- [claimed-docs] “Recurly Compass MCP connects your AI development tools directly to Recurly's documentation and API knowledge.”
- [probe] “official MCP server documented at https://docs.recurly.com/recurly-subscriptions/docs/compass-public-mcp-server”
ai-native userUse an official CLI
weight 2 · round drawnRecurlynone0/10No evidence of an official Recurly CLI; documentation instead references client libraries (Ruby, Node.js, Python, etc.), mobile SDKs, and an MCP server for AI tools, but nothing about a command-line interface.
- [claimed-docs] “Official Client Libraries ### Ruby ... ### Node.js ... ### Python ... ### Dotnet ... ### Java ... ### PHP ... ### Go”
- [claimed-docs] “The Recurly Native Mobile SDKs allow you to integrate recurrent payments in your existing Android or iOS apps in a matter of minutes.”
- [claimed-docs] “Recurly Compass MCP connects your AI development tools directly to Recurly's documentation and API knowledge.”
ai-native userDrive the product through a documented public API
weight 3 · round drawnRecurly documents a full public API with official client libraries in seven languages, entitlements checkable via API, and even an MCP server (Compass) exposing API/doc knowledge to AI tools, indicating strong API-first, AI-accessible design. missing for 10: independent developer corroboration of API robustness/rate limits and no evidence of API versioning or webhooks documentation depth.
- [claimed-docs] “Entitlements can be applied across platforms (web, mobile, and app stores) and checked in real time via the Recurly API.”
- [claimed-docs] “Recurly Compass MCP connects your AI development tools directly to Recurly's documentation and API knowledge.”
- [claimed-docs] “Official Client Libraries ### Ruby ... ### Node.js ... ### Python ... ### Dotnet ... ### Java ... ### PHP ... ### Go”
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.recurly.com/llms.txt # Recurly Documentation Documentation > Welcome to the Recurly Documentation …”
- [probe] “official MCP server documented at https://docs.recurly.com/recurly-subscriptions/docs/compass-public-mcp-server”
Orb's docs explicitly reference creating subscriptions via API, an api-reference error-responses page, per-event idempotency through the API, and a real-time webhooks system, indicating a documented public API spanning core billing operations. Missing for 10: a discoverable machine-readable OpenAPI/swagger spec (probe returned 404 on all candidates) and independent third-party corroboration of API usage.
- [claimed-docs] “You can create a subscription either from the Orb UI or via the subscription creation API endpoint.”
- [claimed-docs] “If you are processing an error programmatically, consider using the type URI along with the status code.”
- [claimed-docs] “Orb provides per-event idempotency through the API to guarantee that duplicates are never processed within the account grace period.”
- [claimed-docs] “Orb provides a powerful, real-time webhooks system that can issue calls to an endpoint of your choosing when certain events take place.”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.withorb.com/openapi.json, https://docs.withorb.com/swagger.json, https://docs.withorb.c…”
ai-native userIssue scoped/least-privilege API credentials for an agent
weight 2 · round drawnRecurlynone0/10No evidence of scoped or least-privilege API key/credential issuance for agents; the MCP server (recurly-docs-12, recurly-probe-2) only connects AI tools to documentation, not scoped API credentials, and no API key management or permission-scoping docs are present.
- [claimed-docs] “Recurly Compass MCP connects your AI development tools directly to Recurly's documentation and API knowledge.”
- [probe] “official MCP server documented at https://docs.recurly.com/recurly-subscriptions/docs/compass-public-mcp-server”
ai-native userBuild against official SDKs
weight 2 · round to RecurlyRecurly documents official client libraries across seven languages (Ruby, Node.js, Python, .NET, Java, PHP, Go) plus native mobile SDKs for iOS/Android, giving AI-native developers a solid multi-language SDK surface, and an official Compass MCP server ties AI tools into its API docs. Missing for 10: independent hands-on developer confirmation of SDK quality/reliability and no explicit mention of AI-specific SDK tooling beyond the MCP docs connector.
- [claimed-docs] “Official Client Libraries ### Ruby ... ### Node.js ... ### Python ... ### Dotnet ... ### Java ... ### PHP ... ### Go”
- [claimed-docs] “The Recurly Native Mobile SDKs allow you to integrate recurrent payments in your existing Android or iOS apps in a matter of minutes.”
- [claimed-docs] “Recurly Compass MCP connects your AI development tools directly to Recurly's documentation and API knowledge.”
- [probe] “official MCP server documented at https://docs.recurly.com/recurly-subscriptions/docs/compass-public-mcp-server”
Orbnone0/10The evidence pack shows Orb has a REST API (endpoints, error responses, webhooks) but contains no mention of official language SDKs (e.g., Python, Node, Java client libraries), and the OpenAPI probe returned 404s, suggesting no public spec/SDK generation surface. Missing for 10: any documentation of official SDKs, client libraries, or language-specific packages for building AI-native integrations.
- [claimed-docs] “You can create a subscription either from the Orb UI or via the subscription creation API endpoint.”
- [claimed-docs] “If you are processing an error programmatically, consider using the type URI along with the status code.”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.withorb.com/openapi.json, https://docs.withorb.com/swagger.json, https://docs.withorb.c…”
ai-native userSubscribe to events via webhooks
weight 2 · round to OrbThe only mention of webhooks in the evidence pack is an indirect community comparison noting Recurly automates retention 'without needing to implement webhooks' for dunning, implying webhook functionality exists, but no first-party documentation, API reference, or event-list evidence describes how to subscribe to or configure webhooks. Missing for 10: official webhook API/docs, list of subscribable events, setup instructions, and any AI-agent-specific webhook integration guidance.
- [community] “Comparing Stripe and Recurly: It's faster to get going with Stripe (no merchant account required) but Recurly appears to offer more automate…”
Orb documents a real-time webhooks system that issues calls to an endpoint for account events, directly enabling event subscription. Missing for 10: details on event payload schema, list of supported event types, signature verification/security, and independent/hands-on confirmation beyond docs.
- [claimed-docs] “Orb provides a powerful, real-time webhooks system that can issue calls to an endpoint of your choosing when certain events take place.”
Agentic features
ai-native userGet AI-generated insights and suggestions from my data inside the product
weight 2 · round drawnRecurlynone0/10No evidence Recurly generates AI-driven insights or suggestions from a merchant's own subscription/billing data; the only AI-related feature (Compass MCP) merely connects AI dev tools to documentation/API knowledge, not to account data analytics.
- [claimed-docs] “Recurly Compass MCP connects your AI development tools directly to Recurly's documentation and API knowledge.”
- [probe] “official MCP server documented at https://docs.recurly.com/recurly-subscriptions/docs/compass-public-mcp-server”
Orbnone0/10Orb is a billing/usage-metering platform; evidence covers forecasting, revenue recognition, invoicing, and pricing analytics, but none describes AI-generated insights or suggestions surfaced within the product itself. No mention of AI/ML features, copilots, or generative recommendations anywhere in the docs pack.
ai-native userSet up automations that run autonomously in the background
weight 2 · round drawnRecurly ships built-in autonomous background processes—dunning campaigns (up to 50 targeted campaigns), automatic backup payment method fallback, and involuntary churn recovery tools—that run without manual intervention once configured. However, there is no evidence of a general-purpose automation/workflow builder, triggers, or agent-configurable rules engine that an AI-native user could set up beyond these fixed billing-retention features. Missing for 10: a configurable automation/rules engine, evidence of user-defined triggers/actions beyond dunning, and any AI-agent-specific automation tooling.
- [claimed-docs] “merchants on Professional and Elite plans can create up to 50 targeted campaigns for different customer segments or plans”
- [claimed-docs] “Involuntary churn happens when customers are lost due to factors outside their intent — expired cards, bank changes, or failed payments. Rec…”
- [claimed-docs] “Let customers designate a backup payment method so service continues automatically if their primary payment fails.”
Orb has several autonomous background processes (auto dunning retries, automatic invoice issuance after grace periods, usage-threshold alerts, and real-time webhooks) that run without manual intervention, which partially satisfies 'automations that run in the background.' However there is no evidence of an AI-native automation builder, agent-configurable workflows, or general-purpose autonomous task orchestration beyond these fixed billing behaviors. Missing for 10: user-configurable automation/workflow engine, AI-agent-triggered automation setup, evidence of arbitrary background task scheduling beyond billing-specific triggers.
- [claimed-docs] “Orb will automatically retry the charge after 1, 4, and 9 days from the initial payment attempt.”
- [claimed-docs] “Automatic Issuance ... Invoices auto-issue after grace periods and buffers ... Manual Issuance ... All invoices require manual approval befo…”
- [claimed-docs] “Trigger alerts at usage thresholds to power product-led growth.”
- [claimed-docs] “Orb provides a powerful, real-time webhooks system that can issue calls to an endpoint of your choosing when certain events take place.”
ai-native userDelegate tasks to a built-in AI assistant inside the product
weight 3 · round drawnRecurlynone0/10Recurly is a subscription billing platform; evidence shows an MCP server for connecting AI dev tools to documentation/API knowledge, but nothing indicates a built-in AI assistant within the product that users can delegate tasks to.
- [claimed-docs] “Recurly Compass MCP connects your AI development tools directly to Recurly's documentation and API knowledge.”
- [probe] “official MCP server documented at https://docs.recurly.com/recurly-subscriptions/docs/compass-public-mcp-server”
ai-native userOperate the product with natural-language commands
weight 2 · round to RecurlyRecurly's Compass MCP server lets AI dev tools query Recurly's documentation and API knowledge, which is a step toward AI-native/agentic interaction, but the evidence describes it as a knowledge/docs connector rather than a natural-language interface for actually executing billing operations (e.g., changing subscriptions, issuing credits) end-to-end. Missing for 10: evidence of natural-language command execution for actual account/subscription actions, hands-on proof of the MCP server performing operations (not just doc lookup), and any client-facing chat/NL interface.
- [claimed-docs] “Recurly Compass MCP connects your AI development tools directly to Recurly's documentation and API knowledge.”
- [probe] “official MCP server documented at https://docs.recurly.com/recurly-subscriptions/docs/compass-public-mcp-server”
Orbnone0/10No evidence anywhere in the pack of a natural-language interface, chat/AI assistant, or NL-driven command execution for Orb; all documented interactions are via UI or REST/SQL APIs.
- [claimed-docs] “You can create a subscription either from the Orb UI or via the subscription creation API endpoint.”
- [claimed-docs] “Build complex billing metrics with full SQL control.”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.withorb.com/openapi.json, https://docs.withorb.com/swagger.json, https://docs.withorb.c…”
Api quality
ai-native userExplore an interactive API reference with runnable examples
weight 2 · round drawnRecurlynone0/10Evidence shows client libraries, SDKs, an MCP server for docs/API knowledge, and an llms.txt file, but there is no mention of an interactive API reference with runnable/live-executable examples (e.g., a try-it-now console). Missing for 10: interactive API console, runnable code examples, in-browser execution/testing of API calls.
- [claimed-docs] “Recurly Compass MCP connects your AI development tools directly to Recurly's documentation and API knowledge.”
- [claimed-docs] “Official Client Libraries ### Ruby ... ### Node.js ... ### Python ... ### Dotnet ... ### Java ... ### PHP ... ### Go”
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.recurly.com/llms.txt # Recurly Documentation Documentation > Welcome to the Recurly Documentation …”
- [probe] “official MCP server documented at https://docs.recurly.com/recurly-subscriptions/docs/compass-public-mcp-server”
Orbnone0/10Evidence only shows standard API reference docs (e.g., error-responses.md) but no mention of an interactive explorer or runnable code examples; the openapi spec probe returned 404s, suggesting no discoverable machine-readable spec to power such interactivity.
- [claimed-docs] “If you are processing an error programmatically, consider using the type URI along with the status code.”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.withorb.com/openapi.json, https://docs.withorb.com/swagger.json, https://docs.withorb.c…”
ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)
weight 2 · round drawnRecurlynone0/10The evidence pack shows official client libraries, mobile SDKs, an llms.txt file, and an MCP server, but no mention of a downloadable OpenAPI/Swagger spec or any machine-readable API schema file. Since Recurly is an API-based product, this axis clearly applies, but there is no evidence of the specific capability requested.
- [claimed-docs] “Official Client Libraries ### Ruby ... ### Node.js ... ### Python ... ### Dotnet ... ### Java ... ### PHP ... ### Go”
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.recurly.com/llms.txt # Recurly Documentation Documentation > Welcome to the Recurly Documentation …”
- [probe] “official MCP server documented at https://docs.recurly.com/recurly-subscriptions/docs/compass-public-mcp-server”
Orbnone0/10The evidence pack shows explicit probes for a machine-readable OpenAPI spec (openapi.json, swagger.json, etc.) all returning 404, and no docs page offers a downloadable API spec — only an llms.txt index and general API reference mentions. Missing for 10: any working OpenAPI/Swagger endpoint, a documented spec download link, or third-party confirmation of spec availability.
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.withorb.com/openapi.json, https://docs.withorb.com/swagger.json, https://docs.withorb.c…”
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.withorb.com/llms.txt # Orb - [Overview](https://docs.withorb.com/overview.md) - [What is Orb?](htt…”
- [claimed-docs] “If you are processing an error programmatically, consider using the type URI along with the status code.”
ai-native userTest against a sandbox environment without touching production data
weight 1 · round drawnRecurlynone0/10No evidence pack item mentions a sandbox, test/dev environment, or test API keys for Recurly; documentation covers subscription changes, billing, and MCP tooling but not isolated test environments. Missing for 10: any mention of sandbox/test mode, test credentials, or separation from production data.
Orbnone0/10The axis applies since billing/API platforms commonly offer sandbox/test environments, but the evidence pack never mentions a distinct sandbox or test-mode environment isolated from production data — only 'testing pricing changes against historical usage' (orb-docs-1), which is backtesting, not a sandbox. missing for 10: explicit sandbox/test-mode environment, API keys or docs distinguishing test vs live data, confirmation that test data never touches production.
- [claimed-docs] “Test pricing changes against historical usage before deploying.”
ai-native userRely on versioned APIs with a documented deprecation policy
weight 2 · round drawnRecurlynone0/10The evidence pack covers subscription features, client libraries, and an MCP server, but contains no documentation of API versioning schemes or a deprecation policy. Missing for 10: any mention of API version headers/URLs, changelog, or a documented deprecation/sunset policy.
Orbnone0/10No evidence pack item documents API versioning scheme or a deprecation policy; only error-response handling and general docs are covered. Missing for 10: versioning scheme (e.g., date-based API versions), deprecation/sunset policy, changelog or migration guides for breaking changes.
Automation depth — how much of the product can run unattendedAutomation depth
How much of the product can run unattended
ai-native userPerform bulk operations across many items at once
weight 2 · round to OrbRecurly's docs mention bulk data import/export to populate accounts and analyze records, which implies some bulk operation support, and API client libraries could theoretically be used to script batch changes. However, there is no explicit documentation of a bulk-update/bulk-cancel/bulk-modify API for subscriptions, invoices, or entitlements, nor any AI-native tooling for orchestrating such bulk actions. missing for 10: explicit bulk subscription/invoice update API, bulk operation endpoints or SDK examples, evidence of AI-driven or agentic bulk workflows, independent confirmation of bulk import scale/performance.
- [claimed-docs] “Use it to populate your account, analyze performance and customer behavior, and keep records for compliance.”
- [claimed-docs] “Import customer data to populate accounts and records across your Recurly account.”
- [claimed-docs] “Official Client Libraries ### Ruby ... ### Node.js ... ### Python ... ### Dotnet ... ### Java ... ### PHP ... ### Go”
Orb documents high-throughput event ingestion (1M+ events/sec) and backfilling historical usage events, which are forms of bulk data operations, but there is no explicit documentation of a bulk/batch API for operations like mass-editing subscriptions, invoices, or other resources. missing for 10: explicit bulk/batch API endpoints for subscriptions, invoices, or other catalog items; UI or API support for multi-item batch edits.
- [claimed-docs] “Handle 1M+ billing events per second in production deployments.”
- [claimed-docs] “If you want Orb to generate accurate invoices or analytics based on prior usage data, you can backfill events.”
- [claimed-docs] “Orb provides per-event idempotency through the API to guarantee that duplicates are never processed within the account grace period.”
- [claimed-docs] “Orb supports managed, daily exports of resource types to your data destination of your choice.”
ai-native userDefine rules that trigger actions automatically on events
weight 3 · round to OrbRecurly ships built-in automated responses to specific billing events—dunning campaigns triggered by failed payments (up to 50 targeted campaigns) and automatic fallback to backup payment methods—but these are pre-defined behaviors within fixed billing workflows, not a general-purpose rule engine where users define arbitrary event→action logic. Missing for 10: a configurable trigger/condition/action framework (e.g., custom webhooks-driven rules, if-this-then-that automation builder) and evidence of user-defined (not just Recurly-provided) automation logic.
- [claimed-docs] “merchants on Professional and Elite plans can create up to 50 targeted campaigns for different customer segments or plans”
- [claimed-docs] “Involuntary churn happens when customers are lost due to factors outside their intent — expired cards, bank changes, or failed payments. Rec…”
- [claimed-docs] “Let customers designate a backup payment method so service continues automatically if their primary payment fails.”
- [community] “Comparing Stripe and Recurly: It's faster to get going with Stripe (no merchant account required) but Recurly appears to offer more automate…”
Orb offers usage-threshold alerts and a real-time webhooks system that fires on defined events, which functions as event-triggered automation, and dunning/invoice issuance include automatic rule-based actions (auto-retry, auto-issue). However, these are fixed, product-specific automations rather than a general user-defined rules/actions engine (no evidence of custom conditional logic, multi-step workflows, or arbitrary action definitions). Missing for 10: a general-purpose rule builder, evidence of custom conditional logic beyond thresholds, and support for arbitrary automated actions beyond alerts/webhooks/dunning.
- [claimed-docs] “Trigger alerts at usage thresholds to power product-led growth.”
- [claimed-docs] “Orb provides a powerful, real-time webhooks system that can issue calls to an endpoint of your choosing when certain events take place.”
- [claimed-docs] “Orb will automatically retry the charge after 1, 4, and 9 days from the initial payment attempt.”
- [claimed-docs] “Automatic Issuance ... Invoices auto-issue after grace periods and buffers ... Manual Issuance ... All invoices require manual approval befo…”
ai-native userSchedule recurring jobs or workflows
weight 2 · round to RecurlyRecurly's core product is built around recurring billing cycles that run automatically (subscription renewals, usage-based billing at cycle end, dunning email campaigns), which functions as a domain-specific recurring workflow engine, but it offers no general-purpose job/task scheduler an AI-native user could configure for arbitrary recurring workflows. Missing for 10: a generic scheduling/automation API or cron-like mechanism for arbitrary AI-driven tasks, and any evidence of AI-agent-triggered scheduling beyond built-in billing cycles.
- [claimed-docs] “Recurly's usage-based billing model lets you charge customers at the end of each billing cycle based on their actual consumption”
- [claimed-docs] “merchants on Professional and Elite plans can create up to 50 targeted campaigns for different customer segments or plans”
- [claimed-docs] “Involuntary churn happens when customers are lost due to factors outside their intent — expired cards, bank changes, or failed payments. Rec…”
- [claimed-docs] “Recurly bills only the difference during an immediate change when the underlying plan is unchanged.”
Orb offers some built-in recurring automated processes—daily scheduled data exports, automatic dunning retries on fixed day intervals, and automatic invoice issuance after grace periods—which are recurring, schedule-driven jobs, but these are fixed billing-domain behaviors rather than a general-purpose scheduler or workflow engine that an AI-native user could configure for arbitrary recurring tasks. Missing for 10: a documented job/workflow scheduling API or UI, support for custom recurrence rules or triggers beyond billing events, and any evidence of user-defined automation workflows.
- [claimed-docs] “Orb supports managed, daily exports of resource types to your data destination of your choice.”
- [claimed-docs] “Orb will automatically retry the charge after 1, 4, and 9 days from the initial payment attempt.”
- [claimed-docs] “Automatic Issuance ... Invoices auto-issue after grace periods and buffers ... Manual Issuance ... All invoices require manual approval befo…”
- [claimed-docs] “Orb provides a powerful, real-time webhooks system that can issue calls to an endpoint of your choosing when certain events take place.”
Billing agent access — stories about billing agent access in this arenaBilling agent access
Stories about billing agent access in this arena
Agent operations
ai-native userAn agent can operate my billing — create a subscription, apply a credit, change a plan — through a programmatic surface with scoped permissions
weight 3 · round drawnRecurly exposes a full REST API and client libraries covering subscription changes, credits, and invoices (recurly-docs-1, -2, -9, -13), which could technically be driven by an agent. However, the only AI-specific surface documented, Recurly Compass MCP, is explicitly a documentation/knowledge connector for AI dev tools, not an operational interface for executing billing actions (recurly-docs-12, recurly-probe-2), and there is no evidence of scoped/restricted API keys or permission models designed for autonomous agent use. Missing for 10: an agent-oriented action interface (e.g., MCP server or agent SDK) that can actually invoke billing operations, and documentation of scoped/least-privilege API credentials for agent access.
- [claimed-docs] “Recurly gives you full control over how and when subscription changes take effect. Upgrade or downgrade a plan, adjust quantity or price, ad…”
- [claimed-docs] “Recurly bills only the difference during an immediate change when the underlying plan is unchanged.”
- [claimed-docs] “Credit invoices give every credit its own distinct invoice — separate from charge invoices.”
- [claimed-docs] “Official Client Libraries ### Ruby ... ### Node.js ... ### Python ... ### Dotnet ... ### Java ... ### PHP ... ### Go”
- [claimed-docs] “Recurly Compass MCP connects your AI development tools directly to Recurly's documentation and API knowledge.”
- [probe] “official MCP server documented at https://docs.recurly.com/recurly-subscriptions/docs/compass-public-mcp-server”
Orb documents a full programmatic surface for the core actions an agent would need: creating subscriptions via API (orb-docs-8), editing subscription pricing/config (orb-docs-10, orb-docs-11), and plan changes with automatic proration credit (orb-docs-16). However, there is no evidence of a scoped-permission or API-key-role model that would let an agent operate with restricted access rather than full account credentials — the error-handling doc (orb-docs-15) only covers error codes, not auth scopes, and the OpenAPI spec probe returned 404s (orb-probe-2), leaving the permission-scoping mechanism unverified. missing for 10: documented scoped API keys/OAuth roles for agent-level permissions, an official agent/MCP-style integration, independent confirmation that these endpoints support least-privilege access.
- [claimed-docs] “You can create a subscription either from the Orb UI or via the subscription creation API endpoint.”
- [claimed-docs] “Subscription Edits let you change a single subscription's pricing configuration ... without changing the plan itself.”
- [claimed-docs] “Preview the billing and invoice impact before you save, so you can confirm how charges and invoices will look.”
- [claimed-docs] “Orb will credit the customer an amount corresponding to the portion of the time period that was left unused using day-based proration.”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.withorb.com/openapi.json, https://docs.withorb.com/swagger.json, https://docs.withorb.c…”
Agent queries
ai-native userAn agent can answer questions about my billing state — MRR, delinquent accounts, upcoming renewals, a customer's invoices — by querying the platform directly
weight 2 · round to RecurlyRecurly exposes a REST API and official client libraries that could let an agent fetch account/invoice/subscription data, and entitlements are described as checkable in real time via the API. However, the only AI-agent integration evidenced (Compass MCP) is explicitly scoped to documentation and API knowledge, not live billing-state queries, and there is no evidence of dedicated endpoints/reports for MRR, delinquent-account lists, or renewal forecasts that an agent could query directly. missing for 10: evidence of an MCP or agent-facing endpoint returning MRR/delinquency/renewal data, documentation of a reporting/analytics API for these metrics, independent confirmation of agent-driven billing queries working in practice.
- [claimed-docs] “Recurly Compass MCP connects your AI development tools directly to Recurly's documentation and API knowledge.”
- [claimed-docs] “Entitlements can be applied across platforms (web, mobile, and app stores) and checked in real time via the Recurly API.”
- [claimed-docs] “Official Client Libraries ### Ruby ... ### Node.js ... ### Python ... ### Dotnet ... ### Java ... ### PHP ... ### Go”
- [claimed-docs] “Modify invoice details post-issuance — including account notes, due dates, PO numbers, customer notes, and terms — to keep records accurate.”
- [probe] “official MCP server documented at https://docs.recurly.com/recurly-subscriptions/docs/compass-public-mcp-server”
Orbnone0/10Evidence covers billing mechanics (dunning, revenue recognition, invoice issuance, exports) but nothing shows an agent-facing query interface for aggregate billing state like MRR, delinquent-account lists, or renewal schedules — no MCP server, natural-language API, or reporting endpoint is documented, and the openapi probe returned 404s, undermining even basic API discoverability for an autonomous agent.
- [claimed-docs] “Orb will automatically retry the charge after 1, 4, and 9 days from the initial payment attempt.”
- [claimed-docs] “Automatic Issuance ... Invoices auto-issue after grace periods and buffers ... Manual Issuance ... All invoices require manual approval befo…”
- [claimed-docs] “Orb automatically calculates recognized revenue in a way that accords with ASC606 guidelines.”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.withorb.com/openapi.json, https://docs.withorb.com/swagger.json, https://docs.withorb.c…”
Billing math — stories about billing math in this arenaBilling math
Stories about billing math in this arena
Billing cycles
ops userControl billing anchors and cycles — calendar vs anniversary billing, alignment across a customer's subscriptions, custom invoice dates
weight 2 · round to OrbEvidence only shows post-issuance invoice detail edits (due dates, PO numbers, terms) via recurly-docs-10, which loosely touches 'custom invoice dates' but doesn't address billing anchor/cycle configuration. No documentation covers calendar-vs-anniversary billing modes or aligning billing cycles across a customer's multiple subscriptions. Missing for 10: explicit billing anchor/cycle-start configuration, calendar vs anniversary billing toggle, cross-subscription date alignment tooling, and any independent/hands-on confirmation of these mechanics.
- [claimed-docs] “Modify invoice details post-issuance — including account notes, due dates, PO numbers, customer notes, and terms — to keep records accurate.”
Docs confirm subscriptions can be aligned to the creation date (anniversary billing) and that billing cycle alignment is configurable, but there's no explicit evidence of calendar-vs-anniversary toggle, cross-subscription cycle alignment for a single customer, or custom invoice date overrides. Missing for 10: explicit calendar billing option, multi-subscription alignment mechanics, custom invoice date configuration.
- [claimed-docs] “Orb also allows you to align the billing cycles with the date that the subscription was created.”
- [claimed-docs] “You can create a subscription either from the Orb UI or via the subscription creation API endpoint.”
- [claimed-docs] “Subscription Edits let you change a single subscription's pricing configuration ... without changing the plan itself.”
Proration
finance leadMid-cycle upgrades and downgrades are prorated with documented, auditable math — credits, partial periods, and invoice line items that reconcile to the penny
weight 3 · round to OrbRecurly's docs confirm mid-cycle changes are billed as prorated differences and that credits get their own separate, dedicated invoices, plus invoices can be amended post-issuance for record accuracy — supporting the core proration/credit reconciliation need. However, there's no documented breakdown of the proration math formula, no explicit audit trail feature, and no evidence (first-party or independent) that invoice reconciliation is verified to the penny in practice. Missing for 10: detailed proration calculation documentation, explicit audit-log/traceability feature, and independent verification of penny-accurate reconciliation.
- [claimed-docs] “Recurly bills only the difference during an immediate change when the underlying plan is unchanged.”
- [claimed-docs] “Credit invoices give every credit its own distinct invoice — separate from charge invoices.”
- [claimed-docs] “Modify invoice details post-issuance — including account notes, due dates, PO numbers, customer notes, and terms — to keep records accurate.”
Orb documents day-based proration on plan changes, credits for unused time, a preview of billing/invoice impact before saving edits, and minimum/maximum charge handling — supporting the core proration and invoice line-item math. However, there is no explicit documentation of penny-level reconciliation guarantees, detailed audit trails, or worked examples showing exact credit/debit math across partial periods. missing for 10: worked/auditable proration examples, explicit 'reconciles to the penny' guarantee, audit-log/traceability documentation for finance review.
- [claimed-docs] “Orb will credit the customer an amount corresponding to the portion of the time period that was left unused using day-based proration.”
- [claimed-docs] “Subscription Edits let you change a single subscription's pricing configuration ... without changing the plan itself.”
- [claimed-docs] “Preview the billing and invoice impact before you save, so you can confirm how charges and invoices will look.”
- [claimed-docs] “Minimums: Ensures a minimum charge is applied. Maximums: Caps the charge at a certain amount.”
Dunning recovery — stories about dunning recovery in this arenaDunning recovery
Stories about dunning recovery in this arena
Card recovery
ops userExpiring or replaced cards update themselves — network account updater, backup payment methods, and hosted update-payment pages
weight 2 · round to RecurlyDocs confirm backup payment methods and general 'tools to proactively catch and resolve' involuntary churn from expired cards/failed payments, but the evidence pack never explicitly names or documents a network account updater feature or hosted update-payment pages. missing for 10: explicit documentation of account updater integration, explicit hosted update-payment page feature, and any hands-on confirmation these self-heal expiring cards.
- [claimed-docs] “Involuntary churn happens when customers are lost due to factors outside their intent — expired cards, bank changes, or failed payments. Rec…”
- [claimed-docs] “Let customers designate a backup payment method so service continues automatically if their primary payment fails.”
Orbnone0/10Evidence covers dunning retry scheduling (orb-docs-19) but there is no mention of network account updater, backup payment methods, or hosted update-payment pages anywhere in the pack, so the specific card-self-update capability is unevidenced.
- [claimed-docs] “Orb will automatically retry the charge after 1, 4, and 9 days from the initial payment attempt.”
Dunning comms
ops userConfigure automated dunning sequences — branded emails, in-app banners, grace periods, and final-state rules like cancel vs unpaid
weight 2 · round to RecurlyDocs confirm dunning campaigns (up to 50 targeted campaigns) and proactive involuntary-churn tools like backup payment methods, and community evidence corroborates automated dunning emails as a differentiator vs competitors. However, there's no documentation of branded email customization, in-app banners, configurable grace periods, or final-state rules distinguishing cancel vs unpaid outcomes. missing for 10: branded email templates/customization, in-app banner support, grace period configuration, and cancel-vs-unpaid final-state rule settings.
- [claimed-docs] “merchants on Professional and Elite plans can create up to 50 targeted campaigns for different customer segments or plans”
- [claimed-docs] “Involuntary churn happens when customers are lost due to factors outside their intent — expired cards, bank changes, or failed payments. Rec…”
- [claimed-docs] “Let customers designate a backup payment method so service continues automatically if their primary payment fails.”
- [community] “Comparing Stripe and Recurly: It's faster to get going with Stripe (no merchant account required) but Recurly appears to offer more automate…”
- [community] “Chargify finally has statements. Recurly has a large emphasis on making sure your customers are retained -- so much so that they added their…”
Orb docs confirm basic dunning mechanics — automatic payment retries at 1/4/9 days (orb-docs-19) and automatic vs manual invoice issuance after grace periods (orb-docs-21) — but there is no evidence of branded/customizable dunning emails, in-app banners, or configurable final-state rules like cancel vs unpaid. missing for 10: branded email templates, in-app banner/notification configuration, customizable grace period settings beyond fixed retry schedule, and explicit cancel-vs-unpaid final state logic.
- [claimed-docs] “Orb will automatically retry the charge after 1, 4, and 9 days from the initial payment attempt.”
- [claimed-docs] “Automatic Issuance ... Invoices auto-issue after grace periods and buffers ... Manual Issuance ... All invoices require manual approval befo…”
Smart retries
finance leadFailed renewal payments are retried automatically on an ML- or schedule-optimized cadence that measurably recovers involuntary churn
weight 3 · round drawnRecurly documents dunning management and involuntary-churn tools like targeted campaigns and backup payment methods, and community commentary notes Recurly's automated retention/dunning emails as a differentiator, but there is no evidence of ML-optimized or schedule-optimized retry cadence logic, nor any quantified recovery-rate/churn-reduction metrics. missing for 10: documentation of the actual retry scheduling/ML logic behind payment retries, evidence of adaptive or optimized cadence (vs simple campaign rules), and third-party or first-party data on measurable involuntary churn recovery.
- [claimed-docs] “merchants on Professional and Elite plans can create up to 50 targeted campaigns for different customer segments or plans”
- [claimed-docs] “Involuntary churn happens when customers are lost due to factors outside their intent — expired cards, bank changes, or failed payments. Rec…”
- [claimed-docs] “Let customers designate a backup payment method so service continues automatically if their primary payment fails.”
- [community] “Comparing Stripe and Recurly: It's faster to get going with Stripe (no merchant account required) but Recurly appears to offer more automate…”
- [community] “Chargify finally has statements. Recurly has a large emphasis on making sure your customers are retained -- so much so that they added their…”
Orb documents automatic dunning retries on a fixed schedule (1, 4, and 9 days after failed payment) via its 'essential-dunning' feature, which addresses the schedule-optimized retry cadence for involuntary churn recovery. However, there is no evidence of ML-based optimization of retry timing, nor any documented metrics or case studies showing measurable churn recovery impact. Missing for 10: ML-driven retry optimization, evidence of measured recovery rates/churn reduction, configurability of retry cadence per customer segment.
- [claimed-docs] “Orb will automatically retry the charge after 1, 4, and 9 days from the initial payment attempt.”
- [claimed-docs] “Automatic Issuance ... Invoices auto-issue after grace periods and buffers ... Manual Issuance ... All invoices require manual approval befo…”
Entitlements — stories about entitlements in this arenaEntitlements
Stories about entitlements in this arena
Entitlement checks
developerFeatures and limits are modeled as entitlements attached to plans, and my application can check a customer's entitlements at runtime via API or SDK
weight 3 · round to RecurlyRecurly's docs explicitly describe an entitlements feature that can be checked in real time via the Recurly API across platforms, matching the core of the story. Missing for 10: detailed documentation of how entitlements are attached to specific plans, explicit SDK-level (not just API) entitlement-check examples, and independent/hands-on confirmation of this specific feature.
- [claimed-docs] “Entitlements can be applied across platforms (web, mobile, and app stores) and checked in real time via the Recurly API.”
- [claimed-docs] “Official Client Libraries ### Ruby ... ### Node.js ... ### Python ... ### Dotnet ... ### Java ... ### PHP ... ### Go”
Orbnone0/10The evidence pack covers subscriptions, plans, usage metrics, invoicing, and revenue recognition, but nowhere mentions entitlements, feature-level access modeling, or an API/SDK for checking a customer's entitlements at runtime. Since billing platforms plausibly could offer such a feature, the axis applies, but there is no supporting evidence.
Plan versioning
ops userChange pricing without breaking existing subscribers — versioned plans, grandfathering, and bulk migration tools for moving cohorts to new prices
weight 2 · round to OrbRecurlynone0/10Docs only describe changing an individual subscription's plan/price/add-ons (recurly-docs-1, recurly-docs-2); there is no mention of versioned plan definitions, grandfathering existing subscribers on old pricing, or bulk/cohort migration tooling for moving groups of subscribers to new pricing.
- [claimed-docs] “Recurly gives you full control over how and when subscription changes take effect. Upgrade or downgrade a plan, adjust quantity or price, ad…”
- [claimed-docs] “Recurly bills only the difference during an immediate change when the underlying plan is unchanged.”
Orb documents subscription-level pricing edits with pre-save impact previews, proration on plan changes, and a Stripe migration path with usage backfill — enabling controlled repricing without breaking existing invoices. But there is no explicit documentation of plan versioning, formal grandfathering mechanisms, or a bulk cohort-migration tool for moving many subscribers to new prices at once. missing for 10: explicit plan-versioning/grandfathering feature, bulk cohort migration tooling beyond one-off Stripe import, evidence of moving existing subscriber cohorts to a new plan version at scale.
- [claimed-docs] “Subscription Edits let you change a single subscription's pricing configuration ... without changing the plan itself.”
- [claimed-docs] “Preview the billing and invoice impact before you save, so you can confirm how charges and invoices will look.”
- [claimed-docs] “Orb will credit the customer an amount corresponding to the portion of the time period that was left unused using day-based proration.”
- [claimed-docs] “Orb supports a seamless migration from Stripe Billing, whether you're using flat-rate subscriptions, usage-based pricing, pricing on seats, …”
- [claimed-docs] “If you want Orb to generate accurate invoices or analytics based on prior usage data, you can backfill events.”
- [claimed-docs] “Test pricing changes against historical usage before deploying.”
Invoicing tax — stories about invoicing tax in this arenaInvoicing tax
Stories about invoicing tax in this arena
Invoices
ops userThe platform generates compliant invoices, credit notes, and receipts — sequential numbering, custom fields, memos — delivered to customers automatically
weight 2 · round drawnDocs confirm invoice modification (notes, due dates, PO numbers, terms), separate credit invoices, and account/record-keeping features, implying automated invoice generation and delivery, but there is no explicit evidence of sequential/compliant numbering rules, custom field configuration on invoices, memo fields, or receipt generation/delivery workflows. missing for 10: explicit documentation of sequential invoice numbering compliance, custom fields on invoices, memo support, automatic receipt generation/delivery, and independent confirmation of these features working as described.
- [claimed-docs] “Credit invoices give every credit its own distinct invoice — separate from charge invoices.”
- [claimed-docs] “Modify invoice details post-issuance — including account notes, due dates, PO numbers, customer notes, and terms — to keep records accurate.”
- [claimed-docs] “Use it to populate your account, analyze performance and customer behavior, and keep records for compliance.”
Orb documents automatic invoice issuance, proration/minimums/maximums calculations, and revenue recognition, indicating invoices are generated and delivered automatically, but there is no explicit evidence of sequential invoice numbering, credit note generation, receipts, or custom fields/memos on invoices. missing for 10: sequential numbering guarantees, credit note issuance, receipt generation, custom fields/memo support on invoices, and independent confirmation of compliance formatting.
- [claimed-docs] “Minimums: Ensures a minimum charge is applied. Maximums: Caps the charge at a certain amount.”
- [claimed-docs] “Automatic Issuance ... Invoices auto-issue after grace periods and buffers ... Manual Issuance ... All invoices require manual approval befo…”
- [claimed-docs] “Orb automatically calculates recognized revenue in a way that accords with ASC606 guidelines.”
- [claimed-docs] “Orb will credit the customer an amount corresponding to the portion of the time period that was left unused using day-based proration.”
Multi currency
finance leadBill customers in their local currency with per-currency price points and consolidated reporting in my home currency
weight 2 · round drawnRecurlynone0/10None of the evidence items mention multi-currency pricing, per-currency price points, or consolidated home-currency reporting — the pack covers subscription changes, usage billing, dunning, entitlements, invoices, and general community commentary, none of which address currency handling. This is a fair capability to expect from a billing platform, so absence of evidence yields 'none' rather than 'na'.
Tax
finance leadSales tax, VAT, and GST are calculated on every invoice — natively or through a first-class tax integration — with the right registrations and rates
weight 2 · round drawnRecurlynone0/10The evidence pack contains no mention of tax calculation, VAT/GST handling, tax registrations, or any first-party or third-party tax integration (e.g., Avalara, TaxJar) — invoicing/credit/dunning features are covered but tax compliance is entirely absent. missing for 10: any documentation of sales tax/VAT/GST calculation, tax registration management, or tax engine integration.
Orbnone0/10The evidence pack covers billing, metering, revenue recognition, dunning, and invoicing calculations but contains no mention of sales tax, VAT, GST calculation, tax registrations, or a first-class tax integration (e.g., Avalara, Stripe Tax). This is a fair axis for a billing/invoicing platform, so absent evidence yields 'none' rather than 'na'.
Migration portability — stories about migration portability in this arenaMigration portability
Stories about migration portability in this arena
Export
finance leadExport every subscription, invoice, and usage record — and port stored payment methods out to another provider — if I choose to leave
weight 2 · round to RecurlyRecurly docs mention a general 'Data Imports and Exports' feature for populating accounts and keeping compliance records, and a community comment specifically praises Recurly's gateway for supporting 'credit card portability' to another provider. However, there is no vendor documentation detailing bulk export of subscriptions, invoices, or usage records, nor an official description of the payment-method portability mechanism (it's only referenced anecdotally by a user, not documented by Recurly). Missing for 10: explicit docs on invoice/usage-record export formats and scope, official documentation of stored payment-method portability/migration tooling, and any independent verification that a full data export actually contains all subscription/invoice/usage history.
- [claimed-docs] “Use it to populate your account, analyze performance and customer behavior, and keep records for compliance.”
- [claimed-docs] “Import customer data to populate accounts and records across your Recurly account.”
- [community] “I am in the process of integrating Recurly into my existing business... I want all the subscription management features Recurly provides. Fu…”
Orb documents managed daily data exports of resource types to any destination (orb-docs-12) and revenue/financial exports (orb-docs-7), which could cover subscriptions/invoices/usage, but there is no evidence describing exporting stored payment methods or any outbound migration path — the only migration content covers importing customers *into* Orb from Stripe (orb-docs-13, orb-docs-14), not leaving Orb for another provider. Missing for 10: explicit payment-method export/portability, confirmation that exports cover all three record types comprehensively, and documentation of an outbound/offboarding migration process.
- [claimed-docs] “Orb supports managed, daily exports of resource types to your data destination of your choice.”
- [claimed-docs] “Track recognized and deferred revenue with exports for financial systems.”
- [claimed-docs] “Orb supports a seamless migration from Stripe Billing, whether you're using flat-rate subscriptions, usage-based pricing, pricing on seats, …”
- [claimed-docs] “If you want Orb to generate accurate invoices or analytics based on prior usage data, you can backfill events.”
Import
ops userMigrate live subscriptions in from a spreadsheet or a competing platform — including PCI-compliant card data transfer — without customers noticing
weight 3 · round to OrbRecurly's docs mention generic data import/export tools (recurly-docs-11, recurly-docs-15) and a community comment notes Recurly's gateway offers 'credit card portability' (recurly-comm-7), suggesting some support for migrating billing data and card credentials. However, there is no documentation of a structured migration workflow, PCI-compliant card tokenization/transfer process, or assurance that live subscriptions migrate without customer disruption. missing for 10: explicit migration/onboarding documentation, PCI-compliant card data transfer mechanics, evidence of seamless customer experience during migration.
- [claimed-docs] “Use it to populate your account, analyze performance and customer behavior, and keep records for compliance.”
- [claimed-docs] “Import customer data to populate accounts and records across your Recurly account.”
- [community] “I am in the process of integrating Recurly into my existing business... I want all the subscription management features Recurly provides. Fu…”
Orb documents migration from Stripe Billing (including flat-rate, usage-based, seat pricing) and backfilling historical usage events for accurate invoicing, which supports competing-platform migration. However there is no evidence of spreadsheet-based import tooling, PCI-compliant card/payment-method data transfer, or explicit assurances that migration is seamless enough that customers won't notice. Missing for 10: spreadsheet import support, PCI-compliant card data transfer mechanism, and documentation/testimony of zero customer disruption during migration.
- [claimed-docs] “Orb supports a seamless migration from Stripe Billing, whether you're using flat-rate subscriptions, usage-based pricing, pricing on seats, …”
- [claimed-docs] “If you want Orb to generate accurate invoices or analytics based on prior usage data, you can backfill events.”
- [claimed-docs] “You can create a subscription either from the Orb UI or via the subscription creation API endpoint.”
Openness — open source, data portability, and self-hosting storiesOpenness
Open source, data portability, and self-hosting stories
ai-native userDo everything through the API that I can do in the UI
weight 2 · round drawnDocs show broad API coverage of core subscription, invoicing, entitlements, and data import/export features (recurly-docs-1,8,10,11) alongside official client libraries (recurly-docs-13), suggesting most UI actions are API-accessible. However, there is no explicit statement of full UI/API parity, and some UI-only features like dunning campaign configuration (recurly-docs-5) aren't confirmed as API-manageable. Missing for 10: an explicit parity guarantee or comprehensive API reference mapping every UI action, and independent confirmation that all UI-configurable settings (e.g., dunning campaigns) are exposed via API.
- [claimed-docs] “Recurly gives you full control over how and when subscription changes take effect. Upgrade or downgrade a plan, adjust quantity or price, ad…”
- [claimed-docs] “Entitlements can be applied across platforms (web, mobile, and app stores) and checked in real time via the Recurly API.”
- [claimed-docs] “Modify invoice details post-issuance — including account notes, due dates, PO numbers, customer notes, and terms — to keep records accurate.”
- [claimed-docs] “Use it to populate your account, analyze performance and customer behavior, and keep records for compliance.”
- [claimed-docs] “Official Client Libraries ### Ruby ... ### Node.js ... ### Python ... ### Dotnet ... ### Java ... ### PHP ... ### Go”
- [claimed-docs] “merchants on Professional and Elite plans can create up to 50 targeted campaigns for different customer segments or plans”
Docs show many UI actions—creating subscriptions, editing subscriptions, configuring billing metrics, exports, webhooks—are explicitly available via API endpoints (orb-docs-8, orb-docs-10, orb-docs-12, orb-doces-18, orb-docs-23), suggesting strong UI/API parity for core billing workflows. However there is no explicit blanket claim that 'everything in the UI is possible via API,' and a probe for a public OpenAPI/swagger spec returned 404s, weakening confidence in full documented API surface coverage. missing for 10: an explicit parity statement, a discoverable full API reference/OpenAPI spec, and confirmation that dashboard-only features (e.g., invoice approval UI, analytics dashboards) have full API equivalents.
- [claimed-docs] “You can create a subscription either from the Orb UI or via the subscription creation API endpoint.”
- [claimed-docs] “Subscription Edits let you change a single subscription's pricing configuration ... without changing the plan itself.”
- [claimed-docs] “Orb supports managed, daily exports of resource types to your data destination of your choice.”
- [claimed-docs] “A billable metric consists of event filters as well as an aggregation clause.”
- [claimed-docs] “Orb provides a powerful, real-time webhooks system that can issue calls to an endpoint of your choosing when certain events take place.”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.withorb.com/openapi.json, https://docs.withorb.com/swagger.json, https://docs.withorb.c…”
ai-native userExport all of my data in open formats and leave
weight 3 · round drawnRecurly documents a data-imports-and-exports feature for populating accounts and keeping records, and community mentions note credit-card portability, suggesting some data can be extracted, but there is no evidence of a comprehensive, open-format (CSV/JSON) full-account export or an official data-portability/exit process. missing for 10: explicit open-format export documentation, full account data export scope, and confirmation of easy migration/exit workflow.
- [claimed-docs] “Use it to populate your account, analyze performance and customer behavior, and keep records for compliance.”
- [claimed-docs] “Import customer data to populate accounts and records across your Recurly account.”
- [community] “I am in the process of integrating Recurly into my existing business... I want all the subscription management features Recurly provides. Fu…”
Orb supports managed daily exports of resource types to a destination of choice and revenue/financial exports, which gives data portability, but this is billing-data export for downstream systems rather than a full account-level open-format export-and-leave capability aimed at an AI-native user. missing for 10: explicit full account/data export (all entities, configs, historical events) in open formats, documented data portability/deletion workflow for leaving the platform, and independent confirmation of export completeness.
- [claimed-docs] “Orb supports managed, daily exports of resource types to your data destination of your choice.”
- [claimed-docs] “Track recognized and deferred revenue with exports for financial systems.”
ai-native userRead the product's source under an open license
weight 2 · round drawnRecurlynone0/10The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)
ai-native userSelf-host the core product
weight 3 · round drawnRecurlynone0/10The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)
Privacy posture — data-handling and privacy storiesPrivacy posture
Data-handling and privacy stories
ai-native userChoose where my data is stored (region/residency)
weight 2 · round drawnRecurlynone0/10No evidence in the pack mentions data residency, regional storage options, or geographic data control for Recurly; this is a fair axis for a SaaS billing platform handling customer/payment data, but absence of evidence means 'none'.
ai-native userControl data retention and deletion
weight 2 · round drawnRecurlynone0/10Evidence covers subscription management, billing, invoicing, imports/exports and an MCP docs connector, but nothing addresses data retention policies, deletion requests, or privacy/GDPR-style controls for AI-native users. missing for 10: data retention policy documentation, data deletion/right-to-erasure mechanisms, privacy compliance controls (GDPR/CCPA), evidence of user-initiated data deletion API or workflow.
Orbnone0/10The evidence pack covers billing, invoicing, revenue recognition, and data exports, but contains no mention of customer data retention policies, deletion APIs, or privacy controls for AI-native users. This is an applicable axis for any SaaS handling customer data, but there is no evidence Orb provides retention/deletion controls.
Revenue recognition — stories about revenue recognition in this arenaRevenue recognition
Stories about revenue recognition in this arena
Analytics
founderI get MRR, churn, cohort retention, and LTV analytics computed from the billing source of truth, not a bolted-on BI export
weight 2 · round to OrbRecurlynone0/10The evidence pack covers subscription changes, dunning, entitlements, invoicing, and data import/export, but contains no mention of MRR, churn, cohort retention, or LTV analytics/reporting dashboards computed from billing data. Missing for 10: any documentation of built-in revenue/retention analytics, cohort or LTV reporting, or dashboards distinct from raw data export.
- [claimed-docs] “Use it to populate your account, analyze performance and customer behavior, and keep records for compliance.”
- [claimed-docs] “Import customer data to populate accounts and records across your Recurly account.”
Orb documents revenue-related metrics computed from billing data itself (accrued revenue, recognized/deferred revenue per ASC606, expansion forecasting) rather than a bolted-on BI export, and supports data exports/SQL for custom analytics. However, there is no explicit mention of MRR, churn, cohort retention, or LTV dashboards as native analytics features — these appear to require building via SQL/export rather than being out-of-the-box computed metrics. missing for 10: explicit MRR/churn/cohort retention/LTV dashboards or reports, evidence these are native computed analytics rather than DIY SQL queries, independent confirmation of accuracy/usability.
- [claimed-docs] “Track accrued revenue before invoices. Forecast and spot expansion.”
- [claimed-docs] “Build complex billing metrics with full SQL control.”
- [claimed-docs] “Track recognized and deferred revenue with exports for financial systems.”
- [claimed-docs] “Orb automatically calculates recognized revenue in a way that accords with ASC606 guidelines.”
- [claimed-docs] “Orb supports managed, daily exports of resource types to your data destination of your choice.”
Gl sync
finance leadInvoices, payments, and recognition entries sync to my general ledger or ERP — QuickBooks, Xero, NetSuite — so the close doesn't need CSV surgery
weight 2 · round to OrbRecurlynone0/10The evidence pack shows generic data import/export capability but no mention of specific GL/ERP integrations (QuickBooks, Xero, NetSuite) or revenue recognition sync. Missing for 10: any documentation of accounting/ERP connectors, revenue recognition automation, or close-process integration.
- [claimed-docs] “Use it to populate your account, analyze performance and customer behavior, and keep records for compliance.”
- [claimed-docs] “Import customer data to populate accounts and records across your Recurly account.”
Orb documents ASC606-compliant revenue recognition tracking and generic data exports 'for financial systems' to 'a data destination of your choice,' which could feed a GL, but no evidence names a native QuickBooks, Xero, or NetSuite connector or a no-CSV sync path — buyers would need to build the export pipeline themselves. Missing for 10: named ERP/accounting-system integrations, automated invoice/payment sync (not just data exports), and confirmation that no manual CSV mapping is required.
- [claimed-docs] “Track recognized and deferred revenue with exports for financial systems.”
- [claimed-docs] “Orb supports managed, daily exports of resource types to your data destination of your choice.”
- [claimed-docs] “Orb automatically calculates recognized revenue in a way that accords with ASC606 guidelines.”
Revrec reports
finance leadRevenue is recognized on an ASC 606 / IFRS 15 basis — deferred revenue schedules and audit-ready recognition reports straight from billing data
weight 3 · round to OrbRecurlynone0/10Evidence covers subscriptions, invoicing, dunning, entitlements, and data import/export for general compliance, but nothing addresses ASC 606/IFRS 15 deferred revenue schedules or audit-ready recognition reporting specifically. Absence of evidence for this applicable billing-platform capability means it can't be credited as delivered.
Orb explicitly automates ASC606-compliant recognized/deferred revenue calculation (orb-docs-22), tracks accrued revenue before invoicing (orb-docs-4), and supports exporting recognized/deferred revenue data for financial systems (orb-docs-7), plus general data exports to downstream destinations (orb-docs-12). This covers the core finance-lead need of ASC606 deferred revenue schedules and exportable recognition data. missing for 10: no explicit mention of IFRS 15 compliance, no detail on audit-ready report formatting/controls, and no independent/hands-on corroboration beyond vendor docs.
- [claimed-docs] “Orb automatically calculates recognized revenue in a way that accords with ASC606 guidelines.”
- [claimed-docs] “Track accrued revenue before invoices. Forecast and spot expansion.”
- [claimed-docs] “Track recognized and deferred revenue with exports for financial systems.”
- [claimed-docs] “Orb supports managed, daily exports of resource types to your data destination of your choice.”
Subscription lifecycle — stories about subscription lifecycle in this arenaSubscription lifecycle
Stories about subscription lifecycle in this arena
Lifecycle api
developerCreate, update, pause, and cancel subscriptions through a documented API — the full lifecycle without touching a dashboard
weight 3 · round drawnDocs confirm a documented API for changing subscriptions (upgrade/downgrade, quantity/price, add-ons) with official client libraries in 7 languages, supporting programmatic subscription management. However, the evidence never explicitly documents 'create' or 'pause'/'cancel' endpoints—only mid-cycle change operations are detailed, so full lifecycle coverage is inferred rather than shown directly. Missing for 10: explicit documentation of subscription creation and cancellation/pause API calls, and any hands-on developer confirmation of the full lifecycle working end-to-end via API alone.
- [claimed-docs] “Recurly gives you full control over how and when subscription changes take effect. Upgrade or downgrade a plan, adjust quantity or price, ad…”
- [claimed-docs] “Recurly bills only the difference during an immediate change when the underlying plan is unchanged.”
- [claimed-docs] “Official Client Libraries ### Ruby ... ### Node.js ... ### Python ... ### Dotnet ... ### Java ... ### PHP ... ### Go”
Docs confirm subscription creation via API (orb-docs-8), billing cycle alignment (orb-docs-9), and editing subscription pricing with preview of impact (orb-docs-10, orb-docs-11), plus plan-change proration logic (orb-docs-16) — covering create/update well. However, no explicit documentation of 'pause' or 'cancel' subscription API endpoints appears in the evidence pack, and no OpenAPI/spec confirmation was found (orb-probe-2 shows 404s for spec files), so full lifecycle coverage is unverified via API alone. Missing for 10: explicit pause/cancel endpoint documentation, confirmed OpenAPI spec listing all lifecycle operations, independent/hands-on verification of API-only workflow.
- [claimed-docs] “You can create a subscription either from the Orb UI or via the subscription creation API endpoint.”
- [claimed-docs] “Orb also allows you to align the billing cycles with the date that the subscription was created.”
- [claimed-docs] “Subscription Edits let you change a single subscription's pricing configuration ... without changing the plan itself.”
- [claimed-docs] “Preview the billing and invoice impact before you save, so you can confirm how charges and invoices will look.”
- [claimed-docs] “Orb will credit the customer an amount corresponding to the portion of the time period that was left unused using day-based proration.”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.withorb.com/openapi.json, https://docs.withorb.com/swagger.json, https://docs.withorb.c…”
Lifecycle events
developerEvery subscription state change — renewal, upgrade, payment failure, cancellation — emits a webhook or event my systems can act on reliably
weight 2 · round to OrbRecurlynone0/10The evidence pack contains no documentation of Recurly's webhook or event system for subscription lifecycle changes (renewal, upgrade, payment failure, cancellation). The only indirect mention is a community comment noting dunning emails work 'without needing to implement webhooks,' which implies webhooks exist as an option but provides no detail on reliability, event types, or delivery guarantees.
- [community] “Comparing Stripe and Recurly: It's faster to get going with Stripe (no merchant account required) but Recurly appears to offer more automate…”
Orb documents a real-time webhooks system for triggering calls on event occurrence, and related lifecycle mechanics (dunning retries, proration on plan change, invoice issuance) exist, implying underlying state changes are trackable. However, the evidence never enumerates which specific lifecycle events (renewal, upgrade, cancellation, payment failure) fire webhooks, nor details delivery guarantees like retries/signing for reliability. Missing for 10: explicit list of webhook event types covering renewal/upgrade/cancellation/payment failure, documentation of delivery reliability guarantees (retries, at-least-once delivery, signature verification), and independent/hands-on confirmation of webhook reliability.
- [claimed-docs] “Orb provides a powerful, real-time webhooks system that can issue calls to an endpoint of your choosing when certain events take place.”
- [claimed-docs] “Orb will automatically retry the charge after 1, 4, and 9 days from the initial payment attempt.”
- [claimed-docs] “Orb will credit the customer an amount corresponding to the portion of the time period that was left unused using day-based proration.”
- [claimed-docs] “Automatic Issuance ... Invoices auto-issue after grace periods and buffers ... Manual Issuance ... All invoices require manual approval befo…”
Self serve
founderMy customers can upgrade, downgrade, or cancel themselves through a hosted portal or embeddable components, without a support ticket
weight 2 · round to RecurlyDocs confirm customers can change plans (upgrade/downgrade, add-ons, proration) via Recurly's subscription APIs, implying self-service capability, but no direct evidence of a hosted customer portal or embeddable UI components for self-cancel/upgrade, nor explicit mention of eliminating support tickets. missing for 10: explicit hosted portal documentation, embeddable component/widget evidence, self-service cancellation flow.
- [claimed-docs] “Recurly gives you full control over how and when subscription changes take effect. Upgrade or downgrade a plan, adjust quantity or price, ad…”
- [claimed-docs] “Recurly bills only the difference during an immediate change when the underlying plan is unchanged.”
Orbnone0/10The evidence pack details subscription creation, edits, plan changes, and proration via the Orb UI or API, but nowhere mentions a hosted customer-facing portal or embeddable self-serve components that end customers could use to upgrade/downgrade/cancel without a support ticket. Missing for 10: any mention of a hosted billing portal, embeddable UI components, or self-serve customer-facing flows.
- [claimed-docs] “You can create a subscription either from the Orb UI or via the subscription creation API endpoint.”
- [claimed-docs] “Orb also allows you to align the billing cycles with the date that the subscription was created.”
- [claimed-docs] “Subscription Edits let you change a single subscription's pricing configuration ... without changing the plan itself.”
- [claimed-docs] “Preview the billing and invoice impact before you save, so you can confirm how charges and invoices will look.”
- [claimed-docs] “Orb will credit the customer an amount corresponding to the portion of the time period that was left unused using day-based proration.”
Trials promos
ops userRun free trials, coupons, and promotion codes with automatic conversion to paid — including trial-end notifications required by card-network rules
weight 2 · round drawnRecurlynone0/10The evidence pack contains no documentation of free trials, coupons/promotion codes, automatic trial-to-paid conversion, or card-network-mandated trial-end notifications. Related dunning/involuntary-churn features are covered, but none address trial or coupon lifecycle specifically. missing for 10: docs on free trial setup, coupon/promo code creation, automatic trial conversion, and trial-end notification compliance.
Orbnone0/10The evidence pack covers subscriptions, proration, dunning, invoicing, and revenue recognition, but contains no mention of free trials, coupon/promotion codes, automatic trial-to-paid conversion, or trial-end notifications required by card-network rules. Since Orb is a subscription billing platform, this axis is a fair question, but there is no supporting evidence for these specific capabilities.
Usage metering — stories about usage metering in this arenaUsage metering
Stories about usage metering in this arena
Event ingestion
developerStream high-volume usage events — API calls, tokens, compute-seconds — into a metering pipeline with idempotent ingestion and backdating support
weight 3 · round to OrbRecurly documents usage-based billing where usage-based add-ons are reported and billed at the end of a cycle (recurly-docs-3, recurly-docs-4), showing basic support for metering usage into billing, but there is no evidence of a high-volume streaming ingestion API, idempotency keys/idempotent ingestion, or explicit backdating support for usage events. Missing for 10: evidence of a streaming/bulk usage ingestion endpoint, documented idempotency guarantees for usage events, and backdating/timestamp-adjustment support for late-arriving usage records.
- [claimed-docs] “Recurly's usage-based billing model lets you charge customers at the end of each billing cycle based on their actual consumption”
- [claimed-docs] “Measured units describe the usage tracked by a usage-based add-on and enable consistent reporting across multiple add-ons and plans.”
Orb's docs directly support high-volume streaming ingestion (1M+ events/sec), per-event idempotency guarantees, and backdating/backfill support for historical usage recalculation and invoice adjustment. missing for 10: independent/hands-on benchmarks corroborating the 1M+ events/sec claim, and more detail on the exact idempotency key mechanism/grace period edge cases.
- [claimed-docs] “Handle 1M+ billing events per second in production deployments.”
- [claimed-docs] “Orb provides per-event idempotency through the API to guarantee that duplicates are never processed within the account grace period.”
- [claimed-docs] “Late data handled safely. Orb recalculates affected invoices automatically.”
- [claimed-docs] “If you want Orb to generate accurate invoices or analytics based on prior usage data, you can backfill events.”
- [claimed-docs] “Test pricing changes against historical usage before deploying.”
Pricing models
finance leadPrice with tiered, volume, graduated, per-unit, and hybrid subscription-plus-usage models — including credits, commitments, and overage — without custom code
weight 3 · round to OrbRecurly's docs confirm usage-based billing with measured units and flexible subscription/add-on changes (recurly-docs-1, recurly-docs-3, recurly-docs-4), which supports basic per-unit and subscription-plus-usage hybrid billing without code. However, the evidence never explicitly documents tiered, volume, or graduated pricing structures, nor named 'commitment' or 'overage' billing mechanics as configurable models. missing for 10: explicit tiered/volume/graduated pricing tier configuration docs, explicit commitment and overage billing mechanics, independent confirmation these complex pricing combos work without custom code.
- [claimed-docs] “Recurly gives you full control over how and when subscription changes take effect. Upgrade or downgrade a plan, adjust quantity or price, ad…”
- [claimed-docs] “Recurly's usage-based billing model lets you charge customers at the end of each billing cycle based on their actual consumption”
- [claimed-docs] “Measured units describe the usage tracked by a usage-based add-on and enable consistent reporting across multiple add-ons and plans.”
- [claimed-docs] “Credit invoices give every credit its own distinct invoice — separate from charge invoices.”
Orb's docs cover billable metrics with flexible aggregation/SQL (orb-docs-5, orb-docs-18), subscription edits and plan changes with proration (orb-docs-10, orb-docs-16), minimums/maximums for commitments and overage (orb-docs-20), and subscription/plan configuration via UI or API — collectively supporting tiered, volume, graduated, per-unit, and hybrid subscription+usage pricing without custom code. missing for 10: explicit named documentation pages for each pricing model (tiered/volume/graduated) and credits/prepaid commitment mechanics, plus independent (non-vendor) confirmation of these pricing structures in production use.
- [claimed-docs] “Build complex billing metrics with full SQL control.”
- [claimed-docs] “A billable metric consists of event filters as well as an aggregation clause.”
- [claimed-docs] “Subscription Edits let you change a single subscription's pricing configuration ... without changing the plan itself.”
- [claimed-docs] “Preview the billing and invoice impact before you save, so you can confirm how charges and invoices will look.”
- [claimed-docs] “Orb will credit the customer an amount corresponding to the portion of the time period that was left unused using day-based proration.”
- [claimed-docs] “Minimums: Ensures a minimum charge is applied. Maximums: Caps the charge at a certain amount.”
- [claimed-docs] “You can create a subscription either from the Orb UI or via the subscription creation API endpoint.”
- [claimed-docs] “Orb also allows you to align the billing cycles with the date that the subscription was created.”
Usage visibility
ops userCustomers and my team can see current-cycle usage and accrued cost in real time, with thresholds and alerts before an invoice surprises anyone
weight 2 · round to OrbRecurlynone0/10Recurly's docs mention usage-based billing that charges at the end of the billing cycle and 'measured units' for reporting, but there is no evidence of real-time current-cycle usage/cost dashboards, customer-facing usage views, or threshold/alert mechanisms to warn before an invoice is generated.
- [claimed-docs] “Recurly's usage-based billing model lets you charge customers at the end of each billing cycle based on their actual consumption”
- [claimed-docs] “Measured units describe the usage tracked by a usage-based add-on and enable consistent reporting across multiple add-ons and plans.”
Orb docs confirm accrued cost/revenue tracking before invoicing (orb-docs-4), configurable usage-threshold alerts for product-led growth (orb-docs-6), and real-time webhooks for event notifications (orb-docs-23), covering the core alerting/real-time-visibility ask. missing for 10: explicit evidence of a customer-facing self-serve usage dashboard and independent/hands-on confirmation that alerts reliably fire before invoice generation.
- [claimed-docs] “Track accrued revenue before invoices. Forecast and spot expansion.”
- [claimed-docs] “Trigger alerts at usage thresholds to power product-led growth.”
- [claimed-docs] “Orb provides a powerful, real-time webhooks system that can issue calls to an endpoint of your choosing when certain events take place.”
Not comparable on these axes
ai-native userPlug MCP servers into this product so it can use their tools
weight 3 · not comparableRecurlynone0/10Evidence only shows Recurly publishing its own MCP server (Compass) so external AI tools can query Recurly's docs/API — this is Recurly acting as an MCP server provider, not as a client that can plug in and use other MCP servers' tools. No evidence Recurly itself can consume/plug in external MCP servers.
- [claimed-docs] “Recurly Compass MCP connects your AI development tools directly to Recurly's documentation and API knowledge.”
- [probe] “official MCP server documented at https://docs.recurly.com/recurly-subscriptions/docs/compass-public-mcp-server”
ai-native userVersion, review, and roll back my automations
weight 1 · not comparableRecurlynone0/10No evidence of version control, review workflows, or rollback for automations/workflows in Recurly; the docs cover subscription/billing config changes, dunning campaigns, and an MCP server for documentation access, but none describe versioning, audit/review, or rollback of automations themselves.
Orbn/aOrb is a usage-based billing and revenue platform, not an automation-building tool; it has no concept of 'automations' to version, review, or roll back. The evidence pack covers billing metrics, subscriptions, invoicing, and revenue recognition, none of which relate to versioning/rollback of automations, so this axis is a category mismatch for Orb's product type.
ai-native userPrevent my data from being used to train AI models
weight 3 · not comparableRecurlyn/aRecurly is a subscription billing/payments platform, not an AI model or AI product that trains on user data; the axis of preventing data use for AI training does not apply to its category.
ai-native userOpt out of telemetry and usage tracking
weight 2 · not comparableRecurlyn/aRecurly is a subscription billing platform, not an AI agent/tool whose telemetry or usage-tracking opt-out would be a meaningful axis for AI-native users; this consumer-privacy-style story doesn't map to its product category.
Orbn/aOrb is a billing/revenue platform, not an AI agent or tool with telemetry collected from AI-native usage; the evidence pack contains no telemetry/analytics-opt-out feature relevant to an 'AI-native user's own tool usage tracking. This axis is a category mismatch for a billing infrastructure product.