Skip to content

How Fly.io’s scores are calculated

The full audit trail, recomputed from the verdict data at build time through the same code that produced the leaderboard: verdict × quality × story weight per cell, cells sum to dimension scores, dimensions blend into the PA Score. Every number on the product page is reproducible from this page alone; for why the formula looks like this, see the methodology.

verdict factors: full ×1.0 · partial ×0.6 · disputed ×0.3 · none ×0.0 · n/a excluded from both sides · cell points = weight × quality × factor · cell max = weight × 10

PA Score20/100

Agent-ready 40.6 × 0.30 = 12.18

API quality 4.2 × 0.20 = 0.84

Openness 7.2 × 0.20 = 1.44

Built-in AI 27.0 × 0.15 = 4.05

Automation 7.5 × 0.15 = 1.13

(12.18 + 0.84 + 1.44 + 4.05 + 1.13) ÷ (0.30 + 0.20 + 0.20 + 0.15 + 0.15) = 19.63 ÷ 1.00 = 19.6

Scores are stored to 1 decimal; the product page’s pills round to whole numbers for display. Each dimension below shows the stories, verdicts, and cited evidence behind its number.

Agent-ready40.6/100×0.30 of the PA blend

Outside-in: can YOUR agent reach and drive this product — API, MCP, CLI, headless runs, agent docs.

Point an agent at llms.txt or agent-oriented docsweight 2

2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max

  • [probe] https://fly.io/llms.txtPROBE llms.txt: HTTP 200 at https://fly.io/llms.txt # Fly.io Documentation > Fly.io is a global public cloud for running full-stack and AI applications. Customer > workloa
  • [claimed-docs] https://fly.ioEvery agent gets its own Linux box. A Sprite checkpoints itself while the agent works, so harnesses like Claude and Codex find everything where they left it.
  • [claimed-docs] https://fly.ioRun every MCP server in its own Sprite, with persistent disk, and egress policy. Pay only for the tool calls you use.

Run the product headlessly / in CI for automationweight 2

2 (weight) × 6 (quality) × 0.6 (partial) = 7.2 of 20 max

  • [claimed-docs] https://fly.io/docs**Step 1:** Install `flyctl` ... **Step 2:** Run `fly launch`
  • [claimed-docs] https://fly.io/docs/getting-started/launch/Run `fly launch` from inside your project source directory to create, configure, and (for most apps) deploy a new application.
  • [claimed-docs] https://fly.ioThe `fly machine run` command is a tool to configure, build, and start a new Machine in one line.
  • [claimed-docs] https://fly.ioThe Fly Machines REST API provides resources to provision and manage Fly Apps, Fly Machines, and Fly Volumes.
  • [claimed-docs] https://fly.io/docs/machines/overview/Fly Machines are fast-launching VMs with a simple REST API.
  • [probe] https://fly.io/docs/flyctl/official CLI documented at https://fly.io/docs/flyctl/

Plug MCP servers into this product so it can use their toolsweight 3

n/a — not applicable to this product: excluded from numerator and denominator

  • [claimed-docs] https://fly.ioRun every MCP server in its own Sprite, with persistent disk, and egress policy. Pay only for the tool calls you use.
  • [claimed-docs] https://fly.io/docs/blueprints/remote-mcp-servers/Fly Machines are tightly isolated VMs that are perfect for running MCP servers.
  • [claimed-docs] https://fly.io/docs/flyctl/mcp/[experimental] Start a flyctl MCP server

Connect an agent via an official MCP serverweight 3

3 (weight) × 5 (quality) × 0.6 (partial) = 9.0 of 30 max

  • [claimed-docs] https://fly.io/docs/flyctl/mcp/[experimental] Start a flyctl MCP server
  • [claimed-docs] https://fly.io/docs/blueprints/remote-mcp-servers/Fly Machines are tightly isolated VMs that are perfect for running MCP servers.

Use an official CLIweight 2

2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max

  • [claimed-docs] https://fly.io/docs**Step 1:** Install `flyctl` ... **Step 2:** Run `fly launch`
  • [claimed-docs] https://fly.io/docs/getting-started/launch/Run `fly launch` from inside your project source directory to create, configure, and (for most apps) deploy a new application.
  • [claimed-docs] https://fly.ioRun diagnostics: `fly doctor` — checks WireGuard, IPs, and Docker.
  • [claimed-docs] https://fly.ioThe `fly machine run` command is a tool to configure, build, and start a new Machine in one line.
  • [claimed-docs] https://fly.io/docs/flyctl/mcp/[experimental] Start a flyctl MCP server
  • [probe] https://fly.io/docs/flyctl/official CLI documented at https://fly.io/docs/flyctl/
  • [claimed-docs] https://fly.ioEvery agent gets its own Linux box. A Sprite checkpoints itself while the agent works, so harnesses like Claude and Codex find everything where they left it.

Drive the product through a documented public APIweight 3

3 (weight) × 6 (quality) × 0.6 (partial) = 10.8 of 30 max

  • [claimed-docs] https://fly.ioThe Fly Machines REST API provides resources to provision and manage Fly Apps, Fly Machines, and Fly Volumes.
  • [claimed-docs] https://fly.io/docs/machines/overview/Fly Machines are fast-launching VMs with a simple REST API.
  • [claimed-docs] https://fly.io/docs/blueprints/remote-mcp-servers/Fly Machines are tightly isolated VMs that are perfect for running MCP servers.
  • [probe] https://fly.io/openapi.jsonPROBE openapi: all candidate paths 404 (https://fly.io/openapi.json, https://fly.io/swagger.json, https://fly.io/api/openapi.json, https://fly.io/.well-known/openapi.json)
  • [claimed-docs] https://fly.io/docs/flyctl/mcp/[experimental] Start a flyctl MCP server

Issue scoped/least-privilege API credentials for an agentweight 2

2 (weight) × 7 (quality) × 1.0 (full) = 14.0 of 20 max

  • [claimed-docs] https://fly.ioConnect a service once for your whole org, and every Sprite can reach it. Nobody pastes a token.
  • [claimed-docs] https://fly.ioGrant access on purpose: by Sprite name, by label, or down to a single endpoint, so a Slack bot can post messages and never touch admin.
  • [claimed-docs] https://fly.ioRotate once and every Sprite has the new credential. Cut off a single Sprite and the rest never notice.
  • [claimed-docs] https://fly.ioSecrets allow sensitive values, such as credentials, to be passed securely to your Fly App. The secret is encrypted and stored in a vault.

Build against official SDKsweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [claimed-docs] https://fly.ioThe Fly Machines REST API provides resources to provision and manage Fly Apps, Fly Machines, and Fly Volumes.
  • [probe] https://fly.io/openapi.jsonPROBE openapi: all candidate paths 404 (https://fly.io/openapi.json, https://fly.io/swagger.json, https://fly.io/api/openapi.json, https://fly.io/.well-known/openapi.json)
  • [probe] https://fly.io/docs/flyctl/official CLI documented at https://fly.io/docs/flyctl/

Subscribe to events via webhooksweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [claimed-docs] https://fly.io/laravel-bytes/never-miss-a-webhook/Fly.io converts Docker images to fast-booting VMs and runs them globally. This is great or ingesting incoming data!

Agent-ready = 73.0 ÷ 180 × 100 = 40.6

API quality4.2/100×0.20 of the PA blend

The programmable surface once an agent is there — machine-readable spec, interactive docs, sandbox, versioning discipline.

Explore an interactive API reference with runnable examplesweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [claimed-docs] https://fly.ioThe Fly Machines REST API provides resources to provision and manage Fly Apps, Fly Machines, and Fly Volumes.
  • [probe] https://fly.io/openapi.jsonPROBE openapi: all candidate paths 404 (https://fly.io/openapi.json, https://fly.io/swagger.json, https://fly.io/api/openapi.json, https://fly.io/.well-known/openapi.json)

Download a machine-readable API spec (OpenAPI or equivalent)weight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [probe] https://fly.io/openapi.jsonPROBE openapi: all candidate paths 404 (https://fly.io/openapi.json, https://fly.io/swagger.json, https://fly.io/api/openapi.json, https://fly.io/.well-known/openapi.json)
  • [claimed-docs] https://fly.ioThe Fly Machines REST API provides resources to provision and manage Fly Apps, Fly Machines, and Fly Volumes.

Test against a sandbox environment without touching production dataweight 1

1 (weight) × 7 (quality) × 0.6 (partial) = 4.2 of 10 max

  • [claimed-docs] https://fly.ioRun it in a hardware-isolated VM. A clean baseline per request, so no two users ever share state. Egress-locked and disposable.
  • [claimed-docs] https://fly.ioRoll back to any checkpoint in seconds
  • [claimed-docs] https://fly.ioEvery agent gets its own Linux box. A Sprite checkpoints itself while the agent works, so harnesses like Claude and Codex find everything where they left it.
  • [claimed-docs] https://fly.ioGive it a Sprite that sleeps until it's needed and wakes up remembering everything.

Rely on versioned APIs with a documented deprecation policyweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [probe] https://fly.io/openapi.jsonPROBE openapi: all candidate paths 404 (https://fly.io/openapi.json, https://fly.io/swagger.json, https://fly.io/api/openapi.json, https://fly.io/.well-known/openapi.json)
  • [probe] https://fly.io/docs.mdPROBE docs-md: HTTP 404 at https://fly.io/docs.md

The documented rate limits and concurrency caps for APIs and functions so I can plan for scaleweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

API quality = 4.2 ÷ 100 × 100 = 4.2

Openness7.2/100×0.20 of the PA blend

Can you leave, inspect, or self-host — data export, open source, portability.

Do everything through the API that I can do in the UIweight 2

2 (weight) × 6 (quality) × 0.6 (partial) = 7.2 of 20 max

  • [claimed-docs] https://fly.ioThe Fly Machines REST API provides resources to provision and manage Fly Apps, Fly Machines, and Fly Volumes.
  • [claimed-docs] https://fly.io/docs/machines/overview/Fly Machines are fast-launching VMs with a simple REST API.
  • [probe] https://fly.io/openapi.jsonPROBE openapi: all candidate paths 404 (https://fly.io/openapi.json, https://fly.io/swagger.json, https://fly.io/api/openapi.json, https://fly.io/.well-known/openapi.json)
  • [probe] https://fly.io/docs/flyctl/official CLI documented at https://fly.io/docs/flyctl/

Export all of my data in open formats and leaveweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Read the product's source under an open licenseweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Self-host the core productweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

  • [claimed-docs] https://fly.io/docs**Step 1:** Install `flyctl` ... **Step 2:** Run `fly launch`
  • [claimed-docs] https://fly.ioFly Machines are hardware-isolated VMs that boot in under a second, run close to your users, and scale out to tens of thousands when the traffic arrives.
  • [probe] https://fly.io/llms.txtPROBE llms.txt: HTTP 200 at https://fly.io/llms.txt # Fly.io Documentation > Fly.io is a global public cloud for running full-stack and AI applications. Customer > workloa
  • [probe] https://fly.io/openapi.jsonPROBE openapi: all candidate paths 404 (https://fly.io/openapi.json, https://fly.io/swagger.json, https://fly.io/api/openapi.json, https://fly.io/.well-known/openapi.json)

Openness = 7.2 ÷ 100 × 100 = 7.2

Built-in AI27.0/100×0.15 of the PA blend

Inside-out: how agentic the product itself is for its users — built-in assistants, autonomous features.

Get AI-generated insights and suggestions from my data inside the productweight 2

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Set up automations that run autonomously in the backgroundweight 2

2 (weight) × 5 (quality) × 0.6 (partial) = 6.0 of 20 max

  • [claimed-docs] https://fly.ioEvery agent gets its own Linux box. A Sprite checkpoints itself while the agent works, so harnesses like Claude and Codex find everything where they left it.
  • [claimed-docs] https://fly.ioGive it a Sprite that sleeps until it's needed and wakes up remembering everything.
  • [claimed-docs] https://fly.ioThe Machine restart policy defines whether and how flyd restarts a Machine after its main process exits.
  • [community] https://news.ycombinator.com/item?id=31390506For me the missing bit in Fly is scheduled tasks. I know how to solve this by spinning up an app that runs permanently as a scheduler, but basic cron-like scheduling should be part of the platform IMO.

Delegate tasks to a built-in AI assistant inside the productweight 3

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Operate the product with natural-language commandsweight 2

2 (weight) × 4 (quality) × 0.6 (partial) = 4.8 of 20 max

  • [claimed-docs] https://fly.io/docs/flyctl/mcp/[experimental] Start a flyctl MCP server
  • [claimed-docs] https://fly.ioGive your agent a real computer and get back to building.
  • [claimed-docs] https://fly.ioEvery agent gets its own Linux box. A Sprite checkpoints itself while the agent works, so harnesses like Claude and Codex find everything where they left it.
  • [claimed-docs] https://fly.io/docs**Step 1:** Install `flyctl` ... **Step 2:** Run `fly launch`

Built-in AI = 10.8 ÷ 40 × 100 = 27.0

Automation7.5/100×0.15 of the PA blend

Depth of automation primitives — rules, scheduling, bulk operations, webhooks.

Perform bulk operations across many items at onceweight 2

2 (weight) × 3 (quality) × 0.6 (partial) = 3.6 of 20 max

  • [claimed-docs] https://fly.ioThe Fly Machines REST API provides resources to provision and manage Fly Apps, Fly Machines, and Fly Volumes.
  • [claimed-docs] https://fly.ioThe `fly machine run` command is a tool to configure, build, and start a new Machine in one line.
  • [claimed-docs] https://fly.iothe `fly scale` command for quick horizontal and vertical scaling

Define rules that trigger actions automatically on eventsweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

  • [claimed-docs] https://fly.ioThe Machine restart policy defines whether and how flyd restarts a Machine after its main process exits.
  • [community] https://news.ycombinator.com/item?id=31390506For me the missing bit in Fly is scheduled tasks. I know how to solve this by spinning up an app that runs permanently as a scheduler, but basic cron-like scheduling should be part of the platform IMO.

Schedule recurring jobs or workflowsweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [community] https://news.ycombinator.com/item?id=31390506For me the missing bit in Fly is scheduled tasks. I know how to solve this by spinning up an app that runs permanently as a scheduler, but basic cron-like scheduling should be part of the platform IMO.

Version, review, and roll back my automationsweight 1

1 (weight) × 4 (quality) × 0.6 (partial) = 2.4 of 10 max

  • [claimed-docs] https://fly.ioRoll back to any checkpoint in seconds
  • [claimed-docs] https://fly.ioEvery agent gets its own Linux box. A Sprite checkpoints itself while the agent works, so harnesses like Claude and Codex find everything where they left it.
  • [claimed-docs] https://fly.ioGive it a Sprite that sleeps until it's needed and wakes up remembering everything.

Automation = 6.0 ÷ 80 × 100 = 7.5