Modal Sandboxes vs Blaxel

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to Blaxel
   Modal Sandboxes✓fullprobed8/10

   Modal has a working llms.txt (HTTP 200) with a documentation summary, and docs pages are also served as markdown (.md suffix) confirming agent-oriented doc access. missing for 10: no independent confirmation that agents actually consume this llms.txt successfully in practice, and no dedicated agent-docs index beyond the generic llms.txt.

   • [probe] “PROBE llms.txt: HTTP 200 at https://modal.com/llms.txt # Modal Documentation > Modal is a platform for running AI workloads in the cloud wi…”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”

   proof ↗⚑ flag

   Blaxel✓fullprobed9/10

   A probe directly confirms llms.txt is live and returns HTTP 200 with a structured documentation summary at docs.blaxel.ai/llms.txt, and Blaxel also ships agent-oriented skills/MCP docs for agents to consume. Missing for 10: no independent third-party confirmation that agents actually consume/parse this llms.txt successfully in practice.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.blaxel.ai/llms.txt # Documentation > Blaxel documentation: cloud sandboxes, AI agent hosting, MCP …”
   • [claimed-docs] “Install the Blaxel skill to give your agent the ability to deploy AI agents, create sandboxes for code execution, host MCP servers, run batc…”
   • [claimed-docs] “The open source Blaxel migration skill gives agents the knowledge they need to migrate sandbox code from other providers to Blaxel.”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to Modal Sandboxes
   Modal Sandboxes✓fullprobed8/10

   Modal Sandboxes is built for programmatic/headless use via Python, JS/Go SDKs and a CLI, with sandboxes created and controlled entirely through code (modal-docs-2, modal-docs-19, modal-docs-20, modal-gh-1). Community evidence confirms real-world CI/automation usage — a developer built GitHub Actions self-hosted runners on Modal Sandboxes explicitly for scalable, pay-per-use automation (modal-comm-1), and another wired gRPC control for an agent runtime (modal-comm-2). Missing for 10: no first-party CI/CD integration guide (e.g., official GitHub Actions/Jenkins docs) or dedicated 'headless CI' documentation section beyond community projects.

   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “Create a Sandbox with the specified Image, and mount a Volume”
   • [github] “The JS and Go SDKs allow you to use Modal Sandboxes, invoke deployed Modal Functions, and interact with some Modal platform resources.”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”
   • [claimed-docs] “Run a command or interactive shell inside a Modal container.”
   • [probe] “official CLI documented at https://modal.com/docs/reference/cli”

   proof ↗⚑ flag

   Blaxel~partialprobed6/10

   Blaxel documents CLI with API keys usable in CI, and explicitly points to an official GitHub Action for CI pipeline integration, plus batch Jobs for automation-style workloads. However, there's no evidence of a full headless CI walkthrough, no independent corroboration, and no explicit CI examples for deploying/testing agents beyond the brief GitHub Action mention. Missing for 10: detailed CI workflow examples/docs, independent hands-on confirmation of CI usage, broader automation guidance beyond batch jobs.

   • [claimed-docs] “API keys can be used in the Blaxel APIs, CLI and SDK.”
   • [claimed-docs] “The easiest way to integrate the Blaxel CLI into a CI pipeline is with the official Blaxel GitHub Action.”
   • [claimed-docs] “Jobs allow you to run many AI tasks in parallel using batch processing.”
   • [probe] “official CLI documented at https://docs.blaxel.ai/cli-reference/introduction”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round to Blaxel
   Modal Sandboxes—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   Blaxel~partialprobed5/10

   Blaxel documents hosting MCP servers (docs-11) and exposing sandboxes as MCP servers (docs-5), and its skill can 'host MCP servers' for agents (docs-1), suggesting agents built on Blaxel can be wired to MCP tool servers. However there is no explicit documentation of connecting arbitrary third-party/external MCP servers into an agent so it can call their tools as a client — the evidence is server-hosting/exposure oriented rather than consumption-oriented. Missing for 10: explicit client-side MCP integration docs, examples of an agent consuming an external MCP server's tools, and independent confirmation this works in practice.

   • [claimed-docs] “Install the Blaxel skill to give your agent the ability to deploy AI agents, create sandboxes for code execution, host MCP servers, run batc…”
   • [claimed-docs] “Every sandbox is exposed via an MCP server that allows agents to operate it using tool calls.”
   • [claimed-docs] “MCP Server Hosting is a serverless computing service that allows you to host remote MCP servers without having to manage infrastructure.”
   • [probe] “official MCP server documented at https://docs.blaxel.ai/skills-mcp”

   proof ↗⚑ flag

4. ai-native userConnect an agent via an official MCP server

   weight 3 · round to Blaxel
   Modal Sandboxes—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   Blaxel✓fullprobed7/10

   Blaxel documents that every sandbox is exposed via an official MCP server enabling agents to operate it via tool calls, and it offers a serverless 'MCP Server Hosting' service for hosting remote MCP servers, with a dedicated docs page confirmed by probe. This is a platform (not itself an agent), so the connect-via-official-MCP-server axis clearly applies and is met by first-party evidence. missing for 10: independent/hands-on confirmation of connecting an external agent to these MCP servers, and more detail on the protocol/setup beyond the docs summary.

   • [claimed-docs] “Every sandbox is exposed via an MCP server that allows agents to operate it using tool calls.”
   • [claimed-docs] “MCP Server Hosting is a serverless computing service that allows you to host remote MCP servers without having to manage infrastructure.”
   • [probe] “official MCP server documented at https://docs.blaxel.ai/skills-mcp”

   proof ↗⚑ flag

5. ai-native userUse an official CLI

   weight 2 · round to Modal Sandboxes
   Modal Sandboxes✓fullprobed8/10

   Modal ships an official CLI (documented at modal.com/docs/reference/cli) supporting sandbox-related operations like running commands/shells and managing/connecting to containers, confirmed by probe access to the CLI reference page. This CLI is core to Modal's agentic/AI-native workflows (creating and interacting with sandboxes for untrusted/agent code). missing for 10: independent hands-on user reports specifically praising/using the CLI (vs. SDK), and more detail on CLI coverage of full sandbox lifecycle (snapshots, networking) beyond the two documented subcommands.

   • [claimed-docs] “Run a command or interactive shell inside a Modal container.”
   • [claimed-docs] “Manage and connect to running containers.”
   • [probe] “official CLI documented at https://modal.com/docs/reference/cli”
   • [claimed-docs] “Running commands”

   proof ↗⚑ flag

   Blaxel✓fullprobed7/10

   Blaxel has an official CLI documented at a dedicated CLI reference, integrable into CI via a GitHub Action, and used for API key auth alongside SDK/API — consistent with AI-native, agentic workflows (deploying agents, sandboxes, MCP servers, jobs). Missing for 10: no independent/hands-on corroboration of CLI usage, no detailed command list or examples in the evidence pack beyond the reference page's existence.

   • [probe] “official CLI documented at https://docs.blaxel.ai/cli-reference/introduction”
   • [claimed-docs] “API keys can be used in the Blaxel APIs, CLI and SDK.”
   • [claimed-docs] “The easiest way to integrate the Blaxel CLI into a CI pipeline is with the official Blaxel GitHub Action.”

   proof ↗⚑ flag

6. ai-native userDrive the product through a documented public API

   weight 3 · round to Modal Sandboxes
   Modal Sandboxes✓fullprobed8/10

   Modal exposes documented Python, JS, and Go SDKs plus a CLI for programmatically creating, controlling, and interacting with Sandboxes (exec, filesystem, snapshots, networking, logs), and community developers confirm driving it via gRPC/SDK for agent runtimes. missing for 10: no public OpenAPI/REST spec (probe found only 404s), so the 'documented public API' is SDK/CLI-based rather than a formal API contract, and there's no independent audit of API stability/versioning.

   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “Create a Sandbox with the specified Image, and mount a Volume”
   • [claimed-docs] “const p = await sb.exec(["cat", "/mnt/volume/message.txt"]);”
   • [github] “The JS and Go SDKs allow you to use Modal Sandboxes, invoke deployed Modal Functions, and interact with some Modal platform resources.”
   • [claimed-docs] “Run a command or interactive shell inside a Modal container.”
   • [claimed-docs] “Manage and connect to running containers.”
   • [claimed-docs] “We've added a modal.Sandbox.logs API, allowing you to fetch() logs from a specific date/time range or to tail() the most recent logs.”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://modal.com/openapi.json, https://modal.com/swagger.json, https://modal.com/api/openapi.json, …”

   proof ↗⚑ flag

   Blaxel~partialprobed5/10

   Docs confirm a public API exists and that API keys can drive it (used across CLI, SDK, and API) and CLI reference documentation is available, but a dedicated public API reference/OpenAPI spec could not be located (all openapi.json/swagger.json paths returned 404), so the story is only partially evidenced. missing for 10: a discoverable OpenAPI/API reference doc, hands-on example of calling the API directly, independent confirmation of API completeness.

   • [claimed-docs] “API keys can be used in the Blaxel APIs, CLI and SDK.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.blaxel.ai/openapi.json, https://docs.blaxel.ai/swagger.json, https://docs.blaxel.ai/api…”
   • [probe] “official CLI documented at https://docs.blaxel.ai/cli-reference/introduction”

   proof ↗⚑ flag

7. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round drawn
   Modal Sandboxes—none0/10

   The evidence describes secure sandboxed execution environments and general platform features, but there is no concrete evidence of a mechanism for issuing scoped/least-privilege API credentials specifically to agents—only a vague marketing line about 'credentials already in place' (modal-docs-3) with no documentation of credential scoping, token permissions, or per-agent access control. Missing for 10: any documented API/credential-scoping mechanism, permission granularity docs, or examples of issuing least-privilege tokens to agents.

   • [claimed-docs] “Autonomous agents with the right tools, context, and credentials already in place — running securely in a full, isolated dev environment.”

   proof ↗⚑ flag

   Blaxel—none0/10

   Evidence shows Blaxel issues generic API keys usable across its APIs, CLI, and SDK (blaxel-docs-12), and preview URLs can require an access token (blaxel-docs-8), but there is no documentation of scoping, role-based permissions, or least-privilege credential configuration for agents specifically.

   • [claimed-docs] “API keys can be used in the Blaxel APIs, CLI and SDK.”
   • [claimed-docs] “When you create a private preview URL a token is required to access the URL.”

   proof ↗⚑ flag

8. ai-native userBuild against official SDKs

   weight 2 · round to Modal Sandboxes
   Modal Sandboxes✓fullcommunity8/10

   Modal ships official Python SDK plus JS/Go SDKs for Sandboxes with documented APIs (exec, filesystem, snapshots, networking, logs) and community developers confirm building against them (GitHub Actions runners, gRPC-based JS agent runtime). missing for 10: independent third-party SDK reference/benchmarks beyond Modal's own docs and one community mention, and no evidence of broader language SDK coverage (e.g. Rust, Java).

   • [github] “The JS and Go SDKs allow you to use Modal Sandboxes, invoke deployed Modal Functions, and interact with some Modal platform resources.”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “Create a Sandbox with the specified Image, and mount a Volume”
   • [claimed-docs] “const p = await sb.exec(["cat", "/mnt/volume/message.txt"]);”
   • [claimed-docs] “We've added a modal.Sandbox.logs API, allowing you to fetch() logs from a specific date/time range or to tail() the most recent logs.”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”

   proof ↗⚑ flag

   Blaxel~partialclaimed3/10

   Only indirect evidence exists that an SDK exists (API keys usable in "Blaxel APIs, CLI and SDK"), but there is no dedicated SDK documentation, language coverage, install instructions, or examples in the pack. missing for 10: dedicated SDK docs/reference, supported languages, code examples, independent corroboration of SDK usage.

   • [claimed-docs] “API keys can be used in the Blaxel APIs, CLI and SDK.”

   proof ↗⚑ flag

9. ai-native userSubscribe to events via webhooks

   weight 2 · round drawn
   Modal Sandboxes—none0/10

   No evidence anywhere in the pack of a webhook subscription mechanism or event-driven notification system for Modal Sandboxes; the docs focus on programmatic spin-up, execution, logging, and filesystem access, not outbound event/webhook subscriptions.

   ⚑ flag

   Blaxel—none0/10

   No evidence in the pack mentions webhooks, event subscriptions, or push notifications from Blaxel; documentation covers sandboxes, MCP hosting, jobs, and CLI/API but nothing about webhook subscription mechanisms.

   ⚑ flag

Agentic features

1. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Modal Sandboxes
   Modal Sandboxes~partialcommunity6/10

   Modal Sandboxes explicitly supports running autonomous agent code in isolated environments at scale (docs-3, docs-4, docs-22), and community evidence shows real-world use for background automation (self-hosted CI runners on modal-comm-1, agent runtimes on modal-comm-2). However, the evidence pack lacks explicit documentation of scheduling/cron triggers or persistent background job orchestration—sandboxes are framed as on-demand execution environments rather than a native 'set it and forget it' automation scheduler. Missing for 10: native scheduling/cron trigger docs, first-party persistent orchestration for long-running background automations.

   • [claimed-docs] “Autonomous agents with the right tools, context, and credentials already in place — running securely in a full, isolated dev environment.”
   • [claimed-docs] “Spin up hundreds of thousands of concurrent rollout environments in seconds.”
   • [claimed-docs] “We execute the agent's code in a Modal Sandbox, which allows us to run arbitrary code in a safe environment.”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”

   proof ↗⚑ flag

   Blaxel~partialclaimed5/10

   Blaxel supports background-running infrastructure like Jobs for batch processing, scale-to-zero agents/sandboxes, and MCP-server hosting, which can enable autonomous background workloads, but there's no explicit documentation of a scheduling/trigger/automation-workflow feature (e.g., cron-like triggers or event-based automation) for setting up self-running automations. missing for 10: explicit scheduler/trigger mechanism for autonomous automations, independent/hands-on evidence of autonomous background execution, and clear docs describing 'automation' as a first-class concept rather than inferred from batch jobs and hosting.

   • [claimed-docs] “Jobs allow you to run many AI tasks in parallel using batch processing.”
   • [claimed-docs] “MCP Server Hosting is a serverless computing service that allows you to host remote MCP servers without having to manage infrastructure.”
   • [claimed-docs] “Automatic scale-to-zero after 5s inactivity, resume from standby under 25ms even after weeks.”
   • [claimed-docs] “Agent Drive is a distributed filesystem that can be mounted to multiple sandboxes or agents at any time, including while they are already ru…”

   proof ↗⚑ flag

2. ai-native userOperate the product with natural-language commands

   weight 2 · round to Blaxel
   Modal Sandboxes—none0/10

   Modal Sandboxes is controlled exclusively via SDK (Python/JS/Go) calls and a traditional CLI (modal run, modal shell, etc.); there is no evidence of a natural-language command interface or chat-driven control layer for operating sandboxes. Evidence shows programmatic/API-first design (modal-docs-2, modal-docs-15, modal-docs-19) rather than any NL-command capability.

   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Run a command or interactive shell inside a Modal container.”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [probe] “official CLI documented at https://modal.com/docs/reference/cli”

   proof ↗⚑ flag

   Blaxel~partialprobed6/10

   Blaxel offers an installable 'skill' that lets an agent deploy agents, create sandboxes, host MCP servers, and run jobs via simple prompts with zero code, directly supporting natural-language operation. However, this is scoped to agent-orchestration skills rather than a full natural-language interface across the entire product surface (e.g., CLI/dashboard operations still require explicit commands). Missing for 10: independent/hands-on verification of the skill's natural-language reliability, and evidence that all major product functions (not just deploy/sandbox/mcp/jobs) are controllable via natural language.

   • [claimed-docs] “Install the Blaxel skill to give your agent the ability to deploy AI agents, create sandboxes for code execution, host MCP servers, run batc…”
   • [claimed-docs] “The open source Blaxel migration skill gives agents the knowledge they need to migrate sandbox code from other providers to Blaxel.”
   • [probe] “official MCP server documented at https://docs.blaxel.ai/skills-mcp”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   Modal Sandboxes—none0/10

   Evidence shows standard docs pages, CLI reference, and SDK code snippets, but no interactive API reference or runnable-example playground; the openapi probe returned 404s and no docs mention an in-browser code runner or live API explorer.

   • [probe] “PROBE openapi: all candidate paths 404 (https://modal.com/openapi.json, https://modal.com/swagger.json, https://modal.com/api/openapi.json, …”
   • [claimed-docs] “Run a command or interactive shell inside a Modal container.”
   • [claimed-docs] “const p = await sb.exec(["cat", "/mnt/volume/message.txt"]);”

   proof ↗⚑ flag

   Blaxel—none0/10

   The probe explicitly found no OpenAPI/Swagger spec at any candidate path (404s), and no evidence exists of an interactive API reference with runnable examples—only static markdown docs and CLI/skills references are documented.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.blaxel.ai/openapi.json, https://docs.blaxel.ai/swagger.json, https://docs.blaxel.ai/api…”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   Modal Sandboxes—none0/10

   The probe explicitly found all candidate OpenAPI/swagger endpoints return 404, and no other evidence pack item mentions a machine-readable API spec being published; Modal's public interface is SDK/CLI-based rather than a REST API with a spec.

   • [probe] “PROBE openapi: all candidate paths 404 (https://modal.com/openapi.json, https://modal.com/swagger.json, https://modal.com/api/openapi.json, …”

   proof ↗⚑ flag

   Blaxel—none0/10

   The probe explicitly checked common OpenAPI spec locations and found all returned 404, and no documentation item references a downloadable machine-readable API spec; only an llms.txt for documentation is present.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.blaxel.ai/openapi.json, https://docs.blaxel.ai/swagger.json, https://docs.blaxel.ai/api…”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.blaxel.ai/llms.txt # Documentation > Blaxel documentation: cloud sandboxes, AI agent hosting, MCP …”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round drawn
   Modal Sandboxes✓fullprobed7/10

   Modal Sandboxes are explicitly designed as ephemeral, isolated containers for running untrusted or agent code, with filesystem, networking/security, and snapshot controls documented (modal-docs-2, modal-docs-8, modal-docs-9, modal-docs-10, modal-probe-2), and community examples show real usage isolating workloads from production infra (modal-comm-1, modal-comm-2). This satisfies the 'test without touching production' need since each sandbox is a fresh, isolated environment rather than shared prod state. Missing for 10: explicit documentation or case study framing isolation specifically in terms of 'production data' separation (e.g., no prod DB/staging parity guidance), and no independent audit confirming true isolation guarantees.

   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Filesystem Access”
   • [claimed-docs] “Snapshots”
   • [claimed-docs] “Networking and security”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “We execute the agent's code in a Modal Sandbox, which allows us to run arbitrary code in a safe environment.”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”

   proof ↗⚑ flag

   Blaxel✓fullclaimed7/10

   Blaxel's core product is isolated sandbox environments for code execution, separate from production infra, with dedicated MCP-controlled sandbox instances, custom templates, and preview URLs, clearly enabling safe testing without touching production data. missing for 10: no explicit documentation contrasting sandbox vs production data isolation guarantees, and no independent/hands-on verification of data isolation beyond vendor docs.

   • [claimed-docs] “Every sandbox is exposed via an MCP server that allows agents to operate it using tool calls.”
   • [claimed-docs] “Sandbox images allow you to create customized & reusable sandbox environments.”
   • [claimed-docs] “Agent Drive is a distributed filesystem that can be mounted to multiple sandboxes or agents at any time, including while they are already ru…”
   • [claimed-docs] “When you create a private preview URL a token is required to access the URL.”
   • [claimed-docs] “Install the Blaxel skill to give your agent the ability to deploy AI agents, create sandboxes for code execution, host MCP servers, run batc…”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Modal Sandboxes—none0/10

   Evidence shows release notes and beta/alpha labels for some sandbox features, but no documentation of API versioning scheme or a deprecation policy; OpenAPI spec probes also 404. missing for 10: explicit versioning scheme documentation, deprecation policy page, changelog with deprecation notices, stability guarantees for GA vs beta APIs.

   • [claimed-docs] “VM Sandboxes Beta”
   • [claimed-docs] “Sidecars Alpha”
   • [claimed-docs] “We've added a modal.Sandbox.logs API, allowing you to fetch() logs from a specific date/time range or to tail() the most recent logs.”
   • [claimed-docs] “it's now possible to opt into a more performant Sandbox backend, which affords substantially higher creation rates and concurrency”
   • [probe] “PROBE openapi: all candidate paths 404 (https://modal.com/openapi.json, https://modal.com/swagger.json, https://modal.com/api/openapi.json, …”

   proof ↗⚑ flag

   Blaxel—none0/10

   No evidence of API versioning scheme or a documented deprecation policy; the OpenAPI spec probe even returned 404s, and no docs mention version history or deprecation timelines.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.blaxel.ai/openapi.json, https://docs.blaxel.ai/swagger.json, https://docs.blaxel.ai/api…”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round drawn
   Modal Sandboxes~partialcommunity6/10

   Modal supports spinning up hundreds of thousands of concurrent sandboxes/rollout environments programmatically, which is strong evidence of bulk/parallel scaling (modal-docs-4, modal-docs-14, modal-docs-5), and this is corroborated by community builds using it for scaled runner fleets (modal-comm-1). However, this is infrastructure-level bulk provisioning, not a documented 'bulk operations' API/pattern for operating across many existing items (e.g., batch file ops, batch snapshot management, or a dedicated batch API) — missing for 10: a first-party batch/bulk-operation API or workflow example (vs just scaling raw sandbox count), independent benchmarks confirming claimed concurrency at scale, and documentation of bulk management operations (e.g., bulk terminate/list/update) rather than just creation.

   • [claimed-docs] “Spin up hundreds of thousands of concurrent rollout environments in seconds.”
   • [claimed-docs] “Spin up hundreds of thousands of concurrent rollout environments in seconds. Fast enough to keep your GPU inference resources saturated acro…”
   • [claimed-docs] “H100s, A100s, A10Gs available on demand. Attach to any sandbox, scale to thousands of concurrent runs, pay by the second with no reserved ca…”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [claimed-docs] “Integrated logging and full visibility into every function, sandbox, and container.”

   proof ↗⚑ flag

   Blaxel~partialclaimed6/10

   Blaxel's Jobs feature explicitly supports running many AI tasks in parallel via batch processing, and the Blaxel skill can invoke batch jobs via simple prompts, showing bulk-operation capability for AI-native users. However, the evidence gives no detail on limits, monitoring, or how a user would trigger operations across a large item set (e.g., bulk API endpoints), nor any independent/hands-on validation of batch performance. Missing for 10: concrete batch API/CLI examples, scale limits, and independent corroboration of bulk job execution.

   • [claimed-docs] “Jobs allow you to run many AI tasks in parallel using batch processing.”
   • [claimed-docs] “Install the Blaxel skill to give your agent the ability to deploy AI agents, create sandboxes for code execution, host MCP servers, run batc…”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round drawn
   Modal Sandboxes—none0/10

   The evidence describes Modal Sandboxes as a programmatic, on-demand execution environment for untrusted/agent code, but nothing in the pack shows a rules/trigger engine that fires actions automatically on external events (e.g., webhooks, event-driven policies). Absent such evidence, this applicable automation-depth axis is unmet.

   ⚑ flag

   Blaxel—none0/10

   Blaxel's evidence covers deployment, sandboxes, MCP hosting, batch jobs, and scale-to-zero infrastructure, but nothing describes a rules/trigger engine where users define conditions that automatically fire actions on events (e.g., webhooks, event-based automation rules). Since this is a plausible capability for an agent-hosting platform, absence of evidence means 'none' rather than 'na'.

   ⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round drawn
   Modal Sandboxes—none0/10

   Evidence covers ephemeral sandbox creation, execution, networking, snapshots, and SDKs, but there is no mention of cron-like scheduling, recurring triggers, or persistent workflow orchestration for Modal Sandboxes. missing for 10: any scheduling/cron API, recurring job trigger mechanism, or workflow orchestration documentation tied to Sandboxes.

   • [claimed-docs] “VM Sandboxes Beta”
   • [claimed-docs] “Running commands”
   • [claimed-docs] “Snapshots”

   proof ↗⚑ flag

   Blaxel—none0/10

   Blaxel documents one-off batch 'Jobs' for parallel AI task processing (blaxel-docs-10) but no evidence shows support for recurring/scheduled triggers, cron-like scheduling, or workflow orchestration over time. missing for 10: scheduling/cron mechanism, recurring trigger configuration, workflow automation docs.

   • [claimed-docs] “Jobs allow you to run many AI tasks in parallel using batch processing.”

   proof ↗⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round drawn
   Modal Sandboxes—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   Blaxel—none0/10

   No evidence of versioning, review workflows, or rollback capabilities for automations/agents/jobs; documentation covers deployment, sandboxes, scaling, and infra regions but nothing about tracking automation history or reverting changes.

   ⚑ flag

Agent workloads

1. ai-native userRun coding agents like Claude Code or Codex inside the sandbox following the vendor's own recipe

   weight 2 · round drawn
   Modal Sandboxes—none0/10

   Modal's docs show general support for running 'agents' and 'untrusted code' in Sandboxes (modal-docs-3, modal-docs-22, modal-docs-19), but no evidence pack item mentions a vendor recipe, tutorial, or example specifically for running Claude Code or Codex inside a Sandbox.

   ⚑ flag

   Blaxel—none0/10

   The evidence describes Blaxel sandboxes, MCP exposure, skills, and job/batch features, but nowhere mentions a documented recipe or guide for running Claude Code or Codex specifically inside a Blaxel sandbox. Missing for 10: any vendor recipe/tutorial naming Claude Code or Codex, or a hands-on report of doing so.

   • [claimed-docs] “Every sandbox is exposed via an MCP server that allows agents to operate it using tool calls.”
   • [claimed-docs] “Sandbox images allow you to create customized & reusable sandbox environments.”
   • [claimed-docs] “Install the Blaxel skill to give your agent the ability to deploy AI agents, create sandboxes for code execution, host MCP servers, run batc…”

   proof ↗⚑ flag

Workloads

1. developerRun a headless browser or full desktop environment inside the sandbox

   weight 2 · round drawn
   Modal Sandboxes—none0/10

   Evidence describes Modal Sandboxes as isolated containers/VMs for running arbitrary code, agents, filesystem access, networking, and command execution, but nowhere mentions headless browser or full desktop/GUI environment support (e.g., Xvfb, VNC, browser automation tooling). No docs or community reports reference browser or desktop environments running inside sandboxes.

   ⚑ flag

   Blaxel—none0/10

   Evidence describes sandboxes for code execution, MCP server exposure, filesystem mounting, and batch jobs, but no mention of headless browser support or full desktop/GUI environment capability inside a sandbox.

   ⚑ flag

2. developerAttach GPUs to sandboxed workloads

   weight 2 · round to Modal Sandboxes
   Modal Sandboxes✓fullcommunity8/10

   Modal docs explicitly state GPUs (H100s, A100s, A10Gs) can be attached to any sandbox on demand with per-second billing, and community evidence (HN runner project) confirms real-world GPU-backed sandbox usage. missing for 10: independent hands-on benchmark of GPU attach performance/reliability, and explicit code example showing GPU parameter in sandbox creation API.

   • [claimed-docs] “H100s, A100s, A10Gs available on demand. Attach to any sandbox, scale to thousands of concurrent runs, pay by the second with no reserved ca…”
   • [claimed-docs] “Attach to any sandbox, scale to thousands of concurrent runs, pay by the second with no reserved capacity.”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”

   proof ↗⚑ flag

   Blaxel—none0/10

   No evidence pack item mentions GPU support, GPU instance types, or hardware acceleration for sandboxes; documentation covers scaling, preview URLs, filesystems, jobs, and MCP hosting but never GPU attachment. missing for 10: any mention of GPU-enabled sandbox instances, GPU pricing/config, or GPU-specific workload examples.

   ⚑ flag

Files

1. developerRead, write, upload, and download files in the sandbox filesystem via the SDK

   weight 2 · round to Modal Sandboxes
   Modal Sandboxes✓fullprobed8/10

   Modal explicitly documents a dedicated 'Filesystem Access' guide for sandboxes (modal-docs-8) alongside SDK examples showing file read/write via mounted volumes and exec commands (modal-docs-20, modal-docs-21), confirming read/write/upload/download-style operations through the SDK. missing for 10: no independent/hands-on community confirmation specifically of file upload/download methods (only exec/volume examples), and no explicit code snippet showing a dedicated upload/download API method distinct from volume mounting.

   • [claimed-docs] “Filesystem Access”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “Create a Sandbox with the specified Image, and mount a Volume”
   • [claimed-docs] “const p = await sb.exec(["cat", "/mnt/volume/message.txt"]);”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”

   proof ↗⚑ flag

   Blaxel—none0/10

   The evidence describes sandboxes, MCP-based tool operation, and a separate 'Agent Drive' distributed filesystem, but nothing documents SDK methods for reading, writing, uploading, or downloading files within a sandbox. Missing for 10: SDK file I/O API reference, upload/download examples, hands-on confirmation of file operations in sandbox.

   • [claimed-docs] “Every sandbox is exposed via an MCP server that allows agents to operate it using tool calls.”
   • [claimed-docs] “Agent Drive is a distributed filesystem that can be mounted to multiple sandboxes or agents at any time, including while they are already ru…”
   • [claimed-docs] “Sandbox images allow you to create customized & reusable sandbox environments.”

   proof ↗⚑ flag

Runtimes

1. developerDefine custom sandbox templates or bring my own container image

   weight 2 · round to Modal Sandboxes
   Modal Sandboxes✓fullcommunity8/10

   Modal explicitly supports creating Sandboxes with custom Images ('Custom images, any dependency' and 'Create a Sandbox with the specified Image, and mount a Volume'), and community projects (GitHub Actions runners, MCP-JS) confirm developers configuring custom sandbox environments in practice. Missing for 10: explicit documentation of importing arbitrary Docker/OCI registry images or a dedicated 'bring your own container' guide beyond the Image API reference.

   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Create a Sandbox with the specified Image, and mount a Volume”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”

   proof ↗⚑ flag

   Blaxel~partialclaimed6/10

   Docs explicitly reference 'Sandbox images' allowing creation of customized & reusable sandbox environments (blaxel-docs-7), which is direct evidence of custom template/image support. However, no evidence details how to bring an arbitrary Dockerfile/OCI image, base image customization workflow, or CLI/SDK commands for building/pushing custom images. missing for 10: concrete docs/examples on bring-your-own-container-image workflow, supported base image formats, and CLI commands for building/publishing custom sandbox templates.

   • [claimed-docs] “Sandbox images allow you to create customized & reusable sandbox environments.”

   proof ↗⚑ flag

2. developerExecute code in multiple language runtimes (Python, JavaScript, and more) and get rich results back

   weight 2 · round to Modal Sandboxes
   Modal Sandboxes~partialclaimed6/10

   Modal Sandboxes let you run arbitrary commands/code in isolated containers with custom images (any dependency, any language stack), including SDK examples for JS/Go exec and Python examples of running agent code (modal-docs-2, modal-docs-13, modal-docs-19-21). However, evidence never shows explicit multi-language runtime presets (Python vs JS vs others) or structured/rich result payloads (e.g., images, dataframes, mime-typed outputs) — outputs shown are plain stdout/exec results. missing for 10: explicit built-in multi-language runtime templates, documented rich/structured result formats beyond raw stdout.

   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Running commands”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “Create a Sandbox with the specified Image, and mount a Volume”
   • [claimed-docs] “const p = await sb.exec(["cat", "/mnt/volume/message.txt"]);”
   • [claimed-docs] “We execute the agent's code in a Modal Sandbox, which allows us to run arbitrary code in a safe environment.”

   proof ↗⚑ flag

   Blaxel~partialclaimed3/10

   Blaxel's docs confirm sandboxes exist for code execution (blaxel-docs-1, blaxel-docs-7) and can be customized via templates, but there is no explicit evidence of specific language runtime support (Python, JavaScript, etc.) or of structured/rich result output formats from executed code. Missing for 10: explicit list of supported language runtimes, documentation of output/result format (e.g., stdout, images, rich objects), and any hands-on demonstration of multi-language execution.

   • [claimed-docs] “Install the Blaxel skill to give your agent the ability to deploy AI agents, create sandboxes for code execution, host MCP servers, run batc…”
   • [claimed-docs] “Every sandbox is exposed via an MCP server that allows agents to operate it using tool calls.”
   • [claimed-docs] “Sandbox images allow you to create customized & reusable sandbox environments.”

   proof ↗⚑ flag

Untrusted code

1. developerExecute untrusted, AI-generated code without risking my own infrastructure

   weight 3 · round to Modal Sandboxes
   Modal Sandboxes✓fullprobed8/10

   Modal Sandboxes is explicitly built for running untrusted/AI-generated code in isolated, ephemeral containers with networking/filesystem controls, resource limits, and logging, and community examples show real-world use (self-hosted CI runners, agent runtimes) confirming the isolation model works in practice. Missing for 10: independent security audit or adversarial escape-testing evidence, and clearer documentation of resource/network isolation guarantees beyond marketing copy.

   • [claimed-docs] “Programmatically scale secure, ephemeral environments for running untrusted code.”
   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Networking and security”
   • [claimed-docs] “VM Sandboxes Beta”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “We execute the agent's code in a Modal Sandbox, which allows us to run arbitrary code in a safe environment.”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”

   proof ↗⚑ flag

   Blaxel✓fullclaimed7/10

   Blaxel provides sandboxes explicitly for isolated code execution, exposed via MCP tool calls, with scale-to-zero/resume infrastructure meaning execution happens off the developer's own infra, plus custom sandbox images and private preview URLs with tokens for security. Missing for 10: independent/hands-on evidence of isolation guarantees (e.g., security audits, escape testing) and explicit detail on resource/network sandboxing limits beyond docs claims.

   • [claimed-docs] “Every sandbox is exposed via an MCP server that allows agents to operate it using tool calls.”
   • [claimed-docs] “Sandbox images allow you to create customized & reusable sandbox environments.”
   • [claimed-docs] “Automatic scale-to-zero after 5s inactivity, resume from standby under 25ms even after weeks.”
   • [claimed-docs] “When you create a private preview URL a token is required to access the URL.”
   • [claimed-docs] “Install the Blaxel skill to give your agent the ability to deploy AI agents, create sandboxes for code execution, host MCP servers, run batc…”

   proof ↗⚑ flag

2. developerRun arbitrary shell commands and install packages inside the sandbox

   weight 2 · round to Modal Sandboxes
   Modal Sandboxes✓fullprobed9/10

   Modal Sandboxes documentation explicitly covers running arbitrary shell commands (sb.exec, CLI 'run a command or interactive shell inside a Modal container'), custom images with any dependency installation, filesystem access, and community reports (HN self-hosted runners, JS agent runtime) corroborate real-world use for arbitrary command execution and untrusted code. Missing for 10: no independent benchmark of package installation limits or explicit documented apt/pip install examples beyond generic 'any dependency' claim.

   • [claimed-docs] “Running commands”
   • [claimed-docs] “Run a command or interactive shell inside a Modal container.”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “const p = await sb.exec(["cat", "/mnt/volume/message.txt"]);”
   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”

   proof ↗⚑ flag

   Blaxel~partialclaimed5/10

   Blaxel's sandboxes are documented as code-execution environments exposed via MCP tool calls, and customizable sandbox images imply support for installing packages/dependencies, but no evidence explicitly shows arbitrary shell command execution or package installation examples/APIs. missing for 10: explicit shell/exec command documentation, package manager install examples, hands-on demonstration of arbitrary command execution.

   • [claimed-docs] “Install the Blaxel skill to give your agent the ability to deploy AI agents, create sandboxes for code execution, host MCP servers, run batc…”
   • [claimed-docs] “Every sandbox is exposed via an MCP server that allows agents to operate it using tool calls.”
   • [claimed-docs] “Sandbox images allow you to create customized & reusable sandbox environments.”

   proof ↗⚑ flag

Agent lifecycle

1. ai-native userMy agent can provision its own sandbox, execute code, read the results, and tear it down — end to end without a human

   weight 3 · round drawn
   Modal Sandboxes✓fullprobed8/10

   Modal Sandboxes are explicitly designed for programmatic, API-driven provisioning, code execution, log/result retrieval, and teardown, with SDK examples (Python/JS/Go) showing sandbox creation, exec, file access, and logs, plus a documented agent-code-execution example. missing for 10: independent hands-on report showing a fully autonomous agent loop (provision->execute->read->teardown) with no human intervention, and clearer documentation of teardown/termination API specifics.

   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Autonomous agents with the right tools, context, and credentials already in place — running securely in a full, isolated dev environment.”
   • [claimed-docs] “Running commands”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “const p = await sb.exec(["cat", "/mnt/volume/message.txt"]);”
   • [claimed-docs] “We execute the agent's code in a Modal Sandbox, which allows us to run arbitrary code in a safe environment.”
   • [claimed-docs] “We've added a modal.Sandbox.logs API, allowing you to fetch() logs from a specific date/time range or to tail() the most recent logs.”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”

   proof ↗⚑ flag

   Blaxel✓fullclaimed8/10

   Blaxel docs describe agents provisioning sandboxes via a skill/API with zero code, exposing each sandbox as an MCP server so agents can execute code via tool calls, and scale-to-zero/teardown behavior after inactivity — covering provision, execute, read results, and teardown end-to-end without human intervention. missing for 10: independent hands-on verification of a full agentic loop and explicit documentation of an agent reading back execution results programmatically.

   • [claimed-docs] “Install the Blaxel skill to give your agent the ability to deploy AI agents, create sandboxes for code execution, host MCP servers, run batc…”
   • [claimed-docs] “Every sandbox is exposed via an MCP server that allows agents to operate it using tool calls.”
   • [claimed-docs] “Automatic scale-to-zero after 5s inactivity, resume from standby under 25ms even after weeks.”
   • [claimed-docs] “Sandbox images allow you to create customized & reusable sandbox environments.”

   proof ↗⚑ flag

Isolation

1. platform-engineerRely on a documented hard isolation boundary (microVM or equivalent) between sandboxes and my systems

   weight 3 · round to Modal Sandboxes
   Modal Sandboxes~partialprobed5/10

   Modal repeatedly describes Sandboxes as 'secure,' 'isolated' containers for untrusted code, and references a 'VM Sandboxes Beta' page implying microVM-based isolation, but no evidence pack item actually documents the underlying isolation technology (e.g., Firecracker/gVisor specifics, threat model, or security boundary guarantees) that a platform engineer would need to rely on. Missing for 10: explicit technical documentation of the isolation mechanism (hypervisor/kernel boundary), any third-party security audit or independent verification of the boundary's robustness.

   • [claimed-docs] “VM Sandboxes Beta”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “We execute the agent's code in a Modal Sandbox, which allows us to run arbitrary code in a safe environment.”

   proof ↗⚑ flag

   Blaxel—none0/10

   The evidence pack describes Blaxel's sandbox features (scale-to-zero, MCP exposure, preview URLs, templates) but never documents the underlying isolation mechanism (e.g., microVM, gVisor, container hardening) or any explicit security boundary guarantee between sandboxes and host systems. Missing for 10: any mention of microVM/hypervisor-based isolation, a security/architecture whitepaper describing the isolation boundary, or third-party/independent verification of sandbox isolation.

   • [claimed-docs] “Every sandbox is exposed via an MCP server that allows agents to operate it using tool calls.”
   • [claimed-docs] “Sandbox images allow you to create customized & reusable sandbox environments.”
   • [claimed-docs] “When you create a private preview URL a token is required to access the URL.”

   proof ↗⚑ flag

2. ai-native userGive an agent a sandbox where host secrets and credentials are unreachable by the code it runs

   weight 2 · round drawn
   Modal Sandboxes~partialprobed5/10

   Modal Sandboxes are documented as isolated, ephemeral containers for running untrusted/agent code with dedicated networking-and-security guides, supporting the general isolation story (modal-docs-1, modal-docs-2, modal-docs-10, modal-probe-2, modal-docs-19, modal-docs-22). However, no evidence explicitly states that host secrets/credentials are unreachable by default or describes the secrets-injection model that would confirm this — one doc even implies credentials are provisioned into the agent environment (modal-docs-3), which cuts against a strong 'unreachable unless granted' guarantee. Missing for 10: explicit secrets-isolation documentation, details of the opt-in secrets mechanism, and independent/hands-on confirmation that host credentials cannot leak into sandboxed code.

   • [claimed-docs] “Programmatically scale secure, ephemeral environments for running untrusted code.”
   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Autonomous agents with the right tools, context, and credentials already in place — running securely in a full, isolated dev environment.”
   • [claimed-docs] “Networking and security”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “We execute the agent's code in a Modal Sandbox, which allows us to run arbitrary code in a safe environment.”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”

   proof ↗⚑ flag

   Blaxel~partialclaimed5/10

   Blaxel provides isolated sandboxes for code execution with MCP-based control and access tokens for API/CLI/SDK auth, implying separation between host credentials and sandbox execution, but there's no explicit documentation stating host secrets/credentials are unreachable by sandbox code or describing secret-injection isolation guarantees. missing for 10: explicit secret-isolation architecture docs, third-party security audit or penetration-test evidence, and confirmation that host credentials cannot leak into sandbox runtime.

   • [claimed-docs] “Install the Blaxel skill to give your agent the ability to deploy AI agents, create sandboxes for code execution, host MCP servers, run batc…”
   • [claimed-docs] “Every sandbox is exposed via an MCP server that allows agents to operate it using tool calls.”
   • [claimed-docs] “Sandbox images allow you to create customized & reusable sandbox environments.”
   • [claimed-docs] “API keys can be used in the Blaxel APIs, CLI and SDK.”

   proof ↗⚑ flag

Network policy

1. platform-engineerRestrict or allow the sandbox's network egress with explicit policy

   weight 3 · round to Modal Sandboxes
   Modal Sandboxes~partialprobed5/10

   Modal has a dedicated 'Networking and security' guide for Sandboxes, confirming the platform documents network controls for sandboxes, but the evidence pack only shows the page title without excerpts detailing how egress allow/deny policies are configured. missing for 10: actual policy syntax/examples (e.g. allowlist/blocklist configuration), confirmation of default-deny behavior, independent/hands-on verification that egress restrictions work as documented.

   • [claimed-docs] “Networking and security”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”

   proof ↗⚑ flag

   Blaxel—none0/10

   No evidence describes any network egress control, firewall, or allow/deny-list policy for sandboxes—docs cover scaling, preview URLs, filesystem mounting, and regions but nothing about network access restriction. missing for 10: any mention of egress policy configuration, allow/deny lists, network isolation controls, or firewall rules for sandboxes.

   ⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Modal Sandboxes
   Modal Sandboxes~partialprobed6/10

   Modal Sandboxes is fundamentally API/SDK-driven (Python, JS/Go) with a full CLI for running commands and managing containers, and docs describe programmatic control over image building, execution, filesystem, networking, snapshots, and logs — suggesting the API is the primary and most complete interface. However, there's no explicit documentation stating full UI/API parity or listing any UI-exclusive features, and no OpenAPI spec is discoverable (probe found 404s), making it hard to confirm completeness of a public REST API surface beyond the SDKs/CLI. Missing for 10: explicit parity statement between dashboard UI and API/CLI capabilities, and a public API reference/spec confirming no UI-only actions.

   • [claimed-docs] “Filesystem Access”
   • [claimed-docs] “Snapshots”
   • [claimed-docs] “Networking and security”
   • [claimed-docs] “Running commands”
   • [claimed-docs] “Run a command or interactive shell inside a Modal container.”
   • [claimed-docs] “Manage and connect to running containers.”
   • [github] “The JS and Go SDKs allow you to use Modal Sandboxes, invoke deployed Modal Functions, and interact with some Modal platform resources.”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://modal.com/openapi.json, https://modal.com/swagger.json, https://modal.com/api/openapi.json, …”
   • [probe] “official CLI documented at https://modal.com/docs/reference/cli”

   proof ↗⚑ flag

   Blaxel~partialprobed5/10

   Blaxel offers API keys usable across API, CLI, and SDK, and documents CLI/SDK for core resources (sandboxes, agents, jobs, MCP servers), suggesting broad programmatic control. However, no OpenAPI/spec was found (all candidate paths 404), and there is no explicit documentation claiming full UI/API feature parity. Missing for 10: a public OpenAPI reference, an explicit parity statement, and independent confirmation that all UI actions are exposed via API.

   • [claimed-docs] “API keys can be used in the Blaxel APIs, CLI and SDK.”
   • [probe] “official CLI documented at https://docs.blaxel.ai/cli-reference/introduction”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.blaxel.ai/openapi.json, https://docs.blaxel.ai/swagger.json, https://docs.blaxel.ai/api…”
   • [claimed-docs] “The easiest way to integrate the Blaxel CLI into a CI pipeline is with the official Blaxel GitHub Action.”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round drawn
   Modal Sandboxes—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   Blaxel—none0/10

   No evidence pack items address data export or portability in open formats, or any means to leave the platform with one's data; the closest is a migration skill for importing sandbox code from other providers, not exporting data out of Blaxel.

   ⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round drawn
   Modal Sandboxes—none0/10

   Evidence only shows a GitHub repo for the client SDK (modal-client) used to invoke Modal Sandboxes, not the underlying sandbox/runtime platform itself, and no license terms are mentioned. There is no evidence that Modal Sandboxes' core product source is published under an open license.

   • [github] “The JS and Go SDKs allow you to use Modal Sandboxes, invoke deployed Modal Functions, and interact with some Modal platform resources.”

   proof ↗⚑ flag

   Blaxel—none0/10

   No evidence indicates Blaxel's core source code is published under an open license; only one skill (migration skill) is mentioned as open source, but the product itself (platform/CLI/SDK) has no cited open-license repository or licensing statement.

   • [claimed-docs] “The open source Blaxel migration skill gives agents the knowledge they need to migrate sandbox code from other providers to Blaxel.”

   proof ↗⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round drawn
   Modal Sandboxes—none0/10

   Modal is a fully-managed cloud platform for running Sandboxes; there is no evidence of any self-hostable/on-prem version of the core product, and everything described (pay-per-second billing, GPU access, hosted CLI) implies a proprietary SaaS deployment model only.

   ⚑ flag

   Blaxel—none0/10

   Blaxel is presented entirely as a serverless/managed cloud platform (scale-to-zero, regions, hosted MCP servers, hosted sandboxes) with no mention of a self-hostable core product, on-prem deployment, or open-source server component that a user could run themselves; only a migration skill and CLI/SDK for interacting with the hosted service are documented.

   • [claimed-docs] “Automatic scale-to-zero after 5s inactivity, resume from standby under 25ms even after weeks.”
   • [claimed-docs] “you can optionally pin an agent or MCP server to a specific region by setting the region field in your blaxel.toml”
   • [claimed-docs] “MCP Server Hosting is a serverless computing service that allows you to host remote MCP servers without having to manage infrastructure.”
   • [claimed-docs] “When deploying a resource on Blaxel, you sometimes have the option to choose the deployment region.”

   proof ↗⚑ flag

Latency

1. developerStart sandboxes with documented sub-second-to-few-second cold starts

   weight 3 · round to Blaxel
   Modal Sandboxes~partialclaimed4/10

   Evidence shows general speed/scale claims ("spin up hundreds of thousands of concurrent rollout environments in seconds," faster backend for higher creation rates) but no explicit documented cold-start latency benchmark (e.g., sub-second or few-second startup time) for sandboxes specifically. Missing for 10: a concrete cold-start time benchmark/number, independent performance verification, and comparison across sandbox sizes/images.

   • [claimed-docs] “Spin up hundreds of thousands of concurrent rollout environments in seconds.”
   • [claimed-docs] “Spin up hundreds of thousands of concurrent rollout environments in seconds. Fast enough to keep your GPU inference resources saturated acro…”
   • [claimed-docs] “it's now possible to opt into a more performant Sandbox backend, which affords substantially higher creation rates and concurrency”
   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”

   proof ↗⚑ flag

   Blaxel~partialclaimed6/10

   Docs claim resume from standby under 25ms even after weeks of inactivity, implying sub-second warm start, and scale-to-zero after 5s inactivity, which is documented performance behavior specific to sandboxes. However, this is vendor-claimed only with no independent benchmark or hands-on cold-start timing (first-boot from image) reported, and no numbers for true 'cold start' (first creation) vs resume-from-standby. Missing for 10: independent benchmarks/hands-on cold-start measurements, explicit distinction between first-time cold start and standby resume times.

   • [claimed-docs] “Automatic scale-to-zero after 5s inactivity, resume from standby under 25ms even after weeks.”
   • [claimed-docs] “Sandbox images allow you to create customized & reusable sandbox environments.”

   proof ↗⚑ flag

Scale

1. platform-engineerRun large concurrent fleets of sandboxes with documented concurrency limits

   weight 3 · round to Modal Sandboxes
   Modal Sandboxes~partialcommunity5/10

   Modal markets very high concurrency ('hundreds of thousands of concurrent rollout environments', 'scale to thousands of concurrent runs') and even documents a backend upgrade for higher creation rates/concurrency, plus community reports of building scalable runner fleets on Sandboxes, showing the capability is real and used at scale. However there is no evidence of documented hard limits, quotas, or per-account/per-plan concurrency ceilings that a platform engineer could plan capacity against. Missing for 10: explicit documented concurrency/rate limits or quotas, benchmark/load-test data, and any per-tier scaling caps.

   • [claimed-docs] “Spin up hundreds of thousands of concurrent rollout environments in seconds.”
   • [claimed-docs] “H100s, A100s, A10Gs available on demand. Attach to any sandbox, scale to thousands of concurrent runs, pay by the second with no reserved ca…”
   • [claimed-docs] “Spin up hundreds of thousands of concurrent rollout environments in seconds. Fast enough to keep your GPU inference resources saturated acro…”
   • [claimed-docs] “Attach to any sandbox, scale to thousands of concurrent runs, pay by the second with no reserved capacity.”
   • [claimed-docs] “it's now possible to opt into a more performant Sandbox backend, which affords substantially higher creation rates and concurrency”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”

   proof ↗⚑ flag

   Blaxel~partialclaimed3/10

   Docs show sandboxes can run in parallel (Jobs batch processing, scale-to-zero/resume, regional pinning) suggesting fleet-scale operation, but no evidence anywhere states explicit concurrency limits, quotas, or maximum fleet sizes a platform-engineer could plan against. missing for 10: documented per-account/per-region concurrency limits, quota API or dashboard, load-test or benchmark data showing fleet scale in practice.

   • [claimed-docs] “Automatic scale-to-zero after 5s inactivity, resume from standby under 25ms even after weeks.”
   • [claimed-docs] “Jobs allow you to run many AI tasks in parallel using batch processing.”
   • [claimed-docs] “you can optionally pin an agent or MCP server to a specific region by setting the region field in your blaxel.toml”
   • [claimed-docs] “Agent Drive is a distributed filesystem that can be mounted to multiple sandboxes or agents at any time, including while they are already ru…”

   proof ↗⚑ flag

2. developerKeep a sandbox session running for hours or days for long agent tasks

   weight 2 · round drawn
   Modal Sandboxes~partialcommunity5/10

   Modal docs describe sandboxes as isolated, scalable execution environments with snapshotting, filesystem, and networking support, and a community project (self-hosted GitHub Actions runners) demonstrates sandboxes being kept alive for extended, real workloads. However, no evidence pack item states explicit maximum session duration, timeout configuration, or guarantees for multi-day persistence, so long-running-session support is inferred rather than directly documented. Missing for 10: explicit docs on configurable/no timeout limits, guidance for multi-day sandbox lifecycles, and independent confirmation of hours/days-long sessions beyond one community anecdote.

   • [claimed-docs] “Snapshots”
   • [claimed-docs] “VM Sandboxes Beta”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [claimed-docs] “Integrated logging and full visibility into every function, sandbox, and container.”
   • [claimed-docs] “We've added a modal.Sandbox.logs API, allowing you to fetch() logs from a specific date/time range or to tail() the most recent logs.”

   proof ↗⚑ flag

   Blaxel~partialclaimed5/10

   Docs describe scale-to-zero after 5s inactivity with sub-25ms resume even after weeks, implying persistent state across long periods, but this describes a serverless suspend/resume model rather than an explicit 'keep running continuously for hours/days' capability, and there is no documentation of session duration limits or long-running execution during active agent work. missing for 10: explicit session/timeout limits, evidence of continuous multi-hour/day execution without suspension, independent confirmation of long-running task support.

   • [claimed-docs] “Automatic scale-to-zero after 5s inactivity, resume from standby under 25ms even after weeks.”

   proof ↗⚑ flag

Pricing

1. developerStart building with a free tier or included credits without talking to sales

   weight 1 · round drawn
   Modal Sandboxes—none0/10

   The evidence pack includes pricing-related docs (pay-per-use, burst capacity) but no mention of a free tier, free credits, or self-serve signup without contacting sales; nothing confirms developers can start building without sales contact.

   ⚑ flag

   Blaxel—none0/10

   No evidence pack items mention pricing, free tier, included credits, or self-service signup without sales contact; all evidence covers technical features (sandboxes, MCP, jobs, regions).

   ⚑ flag

2. platform-engineerPay per second only for the compute a sandbox actually uses

   weight 3 · round to Modal Sandboxes
   Modal Sandboxes✓fullcommunity8/10

   Modal's pricing docs and marketing explicitly state per-second billing with no reserved capacity ('pay by the second with no reserved capacity', 'only pay for what you use... burst up to what you need without over-allocating'), directly matching the story, and a community user corroborates paying only for resources actually consumed when building GPU runners on Sandboxes. Missing for 10: independent verification of exact billing granularity/metering mechanics and no first-party pricing breakdown showing per-second line items in the evidence pack.

   • [claimed-docs] “Only pay for what you use. Burst up to what you need without over-allocating CPU or memory in advance.”
   • [claimed-docs] “Attach to any sandbox, scale to thousands of concurrent runs, pay by the second with no reserved capacity.”
   • [claimed-docs] “Burst up to what you need without over-allocating CPU or memory in advance.”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”

   proof ↗⚑ flag

   Blaxel~partialclaimed3/10

   Docs describe scale-to-zero after 5s of inactivity and fast resume (blaxel-docs-2), implying sandboxes only consume compute while active, which is suggestive of fine-grained usage-based billing, but no evidence explicitly states a per-second pricing/billing model or shows pricing pages confirming this granularity. missing for 10: explicit pricing/billing documentation stating per-second metering, evidence of actual invoicing granularity, independent confirmation of cost savings from scale-to-zero.

   • [claimed-docs] “Automatic scale-to-zero after 5s inactivity, resume from standby under 25ms even after weeks.”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round to Blaxel
   Modal Sandboxes—none0/10

   No evidence pack item mentions data residency, region selection, or geographic data storage controls for Modal Sandboxes; the pack covers execution, scaling, GPUs, filesystem, and networking but nothing about choosing storage region/residency.

   ⚑ flag

   Blaxel~partialclaimed6/10

   Docs confirm users can pin agents/MCP servers to a specific region via blaxel.toml and that deployment region choice is sometimes available, which directly supports data residency control. Missing for 10: a full list of available regions, compliance/certification details (e.g., GDPR, SOC2 residency guarantees), and independent confirmation that this pinning is enforced for all data at rest (not just compute location).

   • [claimed-docs] “you can optionally pin an agent or MCP server to a specific region by setting the region field in your blaxel.toml”
   • [claimed-docs] “When deploying a resource on Blaxel, you sometimes have the option to choose the deployment region.”

   proof ↗⚑ flag

2. ai-native userControl data retention and deletion

   weight 2 · round drawn
   Modal Sandboxes—none0/10

   Evidence covers sandbox ephemerality, filesystem access, snapshots, and networking/security, but nothing describes data retention policies, deletion controls, or compliance/data-handling settings that a user could configure. missing for 10: explicit data retention policy documentation, user-controllable deletion/purge mechanisms, compliance certifications (e.g., SOC2/GDPR data handling), retention period configuration.

   ⚑ flag

   Blaxel—none0/10

   No evidence in the pack addresses data retention policies, deletion controls, or data lifecycle management for user data/sandboxes; the docs focus on infrastructure, scaling, and access tokens rather than privacy/retention controls.

   ⚑ flag

3. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Modal Sandboxes—none0/10

   No evidence in the pack mentions telemetry, usage tracking, analytics collection, or an opt-out mechanism for Modal Sandboxes; the docs cover sandboxing, GPUs, filesystem, and logging but not data-collection/privacy controls.

   ⚑ flag

   Blaxel—none0/10

   No evidence in the pack addresses telemetry, usage tracking, or any opt-out mechanism; the docs cover deployment, sandboxes, MCP hosting, and security tokens but nothing about data collection settings. missing for 10: any documentation or setting related to telemetry collection or opt-out, privacy policy mention, or usage-tracking configuration.

   ⚑ flag

Lifecycle

1. developerSet timeouts so sandboxes shut down automatically and stop billing when idle or done

   weight 2 · round to Blaxel
   Modal Sandboxes—none0/10

   The evidence pack only mentions general pay-per-second billing and bursty scaling, but never documents an explicit sandbox timeout or idle-shutdown parameter/config. No citation confirms configurable timeouts that auto-terminate sandboxes when idle or complete.

   ⚑ flag

   Blaxel~partialclaimed6/10

   Docs confirm automatic scale-to-zero after 5s of inactivity with fast resume, which directly supports idle shutdown and implies billing stops when inactive (blaxel-docs-2). However, there is no evidence of a developer-configurable timeout parameter (e.g., custom idle duration) or explicit confirmation that billing halts during scale-to-zero, and no independent/hands-on corroboration of this behavior. Missing for 10: explicit configurable timeout setting, billing-stop confirmation, independent verification.

   • [claimed-docs] “Automatic scale-to-zero after 5s inactivity, resume from standby under 25ms even after weeks.”

   proof ↗⚑ flag

2. developerSpin up an isolated sandbox with one API/SDK call and get a live environment in seconds

   weight 3 · round to Modal Sandboxes
   Modal Sandboxes✓fullprobed8/10

   Docs and SDK references (Python, JS/Go) show a single call (e.g. modal.Sandbox.create) spins up an isolated, ephemeral container/VM in seconds, with fast creation rates highlighted (hundreds of thousands of concurrent environments in seconds, faster backend for higher creation rates). Community builders (HN posts) independently confirm using Modal Sandboxes programmatically for real workloads like CI runners and agent runtimes. Missing for 10: independent benchmark of actual cold-start latency and a public OpenAPI spec (probe found 404s), so exact 'seconds' claim isn't third-party verified.

   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Spin up hundreds of thousands of concurrent rollout environments in seconds.”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “Create a Sandbox with the specified Image, and mount a Volume”
   • [claimed-docs] “it's now possible to opt into a more performant Sandbox backend, which affords substantially higher creation rates and concurrency”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”

   proof ↗⚑ flag

   Blaxel~partialprobed6/10

   Docs confirm sandboxes are a core primitive with fast resume (<25ms from standby), customizable/reusable images, API/CLI/SDK access via API keys, and MCP-server exposure for tool control, all pointing to programmatic creation. However, there's no explicit example or evidence of a single one-call SDK/API invocation creating a sandbox with a concrete latency claim for cold creation (only resume-from-standby timing is given), and no independent/hands-on confirmation of 'seconds to live environment' for a fresh (non-cached) sandbox. missing for 10: explicit one-call code sample creating a sandbox, documented cold-start provisioning latency, independent verification of speed claims.

   • [claimed-docs] “Automatic scale-to-zero after 5s inactivity, resume from standby under 25ms even after weeks.”
   • [claimed-docs] “Sandbox images allow you to create customized & reusable sandbox environments.”
   • [claimed-docs] “API keys can be used in the Blaxel APIs, CLI and SDK.”
   • [claimed-docs] “Every sandbox is exposed via an MCP server that allows agents to operate it using tool calls.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.blaxel.ai/llms.txt # Documentation > Blaxel documentation: cloud sandboxes, AI agent hosting, MCP …”

   proof ↗⚑ flag

Preview access

1. developerExpose a port from the sandbox on a public preview URL to reach services running inside

   weight 2 · round to Blaxel
   Modal Sandboxes~partialprobed3/10

   The docs pack includes a 'Networking and security' guide for sandboxes (modal-docs-10) which is the likely location for tunnel/port-exposure features, but no evidence text actually describes exposing a port to a public preview URL, tunnel URLs, or reachability details. missing for 10: explicit documentation or example showing sandbox.tunnels()/port forwarding, a public URL format, and independent confirmation of accessing services inside a sandbox via that URL.

   • [claimed-docs] “Networking and security”
   • [claimed-docs] “VM Sandboxes Beta”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”

   proof ↗⚑ flag

   Blaxel✓fullclaimed8/10

   Docs confirm sandboxes support preview URLs, including custom domains and private/token-protected URLs, directly enabling exposing a port to reach services running inside the sandbox via a public URL. Missing for 10: no explicit hands-on example showing the port-to-URL mapping mechanics or independent third-party confirmation.

   • [claimed-docs] “Preview URLs with your own custom domain.”
   • [claimed-docs] “When you create a private preview URL a token is required to access the URL.”

   proof ↗⚑ flag

Snapshots

1. developerPause a running sandbox and resume it later with filesystem and memory state intact

   weight 2 · round to Blaxel
   Modal Sandboxes~partialclaimed4/10

   Modal has a dedicated 'Snapshots' doc for sandboxes suggesting some pause/resume-with-state capability, but the evidence pack only gives the page title with no detail on what state (filesystem/memory) is preserved or how resume works. Missing for 10: documented mechanics of snapshot/resume (filesystem+memory fidelity), explicit confirmation memory state is preserved, independent/hands-on validation of the feature working as described.

   • [claimed-docs] “Snapshots”
   • [claimed-docs] “Filesystem Access”

   proof ↗⚑ flag

   Blaxel~partialclaimed5/10

   Docs mention automatic scale-to-zero after inactivity with resume from standby in under 25ms 'even after weeks' (blaxel-docs-2), implying state is preserved across pause/resume cycles, and Agent Drive provides a persistent distributed filesystem mountable to sandboxes (blaxel-docs-9). However, there is no explicit developer-triggered pause/resume API, and no confirmation that in-memory process state (not just filesystem) is preserved. missing for 10: explicit manual pause/resume command, explicit statement that memory/process state (not just filesystem) survives resume, independent verification of the claim.

   • [claimed-docs] “Automatic scale-to-zero after 5s inactivity, resume from standby under 25ms even after weeks.”
   • [claimed-docs] “Agent Drive is a distributed filesystem that can be mounted to multiple sandboxes or agents at any time, including while they are already ru…”

   proof ↗⚑ flag

2. developerSnapshot a sandbox and later restore or fork new sandboxes from that snapshot

   weight 3 · round to Modal Sandboxes
   Modal Sandboxes~partialclaimed5/10

   Modal explicitly documents a 'Sandbox Snapshots' feature (modal-docs-9), confirming snapshot capability exists, but the evidence pack gives no detail on the actual snapshot/restore/fork API mechanics, no confirmation that new independent sandboxes can be forked from a single snapshot, and no community/hands-on corroboration of this specific workflow. missing for 10: documented API details on restoring/forking multiple sandboxes from one snapshot, independent/hands-on confirmation of snapshot-restore-fork workflow, and clarity on any beta/experimental limitations.

   • [claimed-docs] “Snapshots”
   • [claimed-docs] “VM Sandboxes Beta”

   proof ↗⚑ flag

   Blaxel—none0/10

   The evidence covers sandbox creation, scale-to-zero/resume, preview URLs, templates, and filesystem mounting, but nowhere mentions snapshotting a sandbox's state or forking new sandboxes from a saved snapshot. Sandbox images/templates ([blaxel-docs-7]) are pre-built environments, not runtime snapshots of a live sandbox's state.

   • [claimed-docs] “Automatic scale-to-zero after 5s inactivity, resume from standby under 25ms even after weeks.”
   • [claimed-docs] “Sandbox images allow you to create customized & reusable sandbox environments.”
   • [claimed-docs] “Agent Drive is a distributed filesystem that can be mounted to multiple sandboxes or agents at any time, including while they are already ru…”

   proof ↗⚑ flag