E2B vs Modal Sandboxes

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to E2B
   E2B✓fullprobed9/10

   E2B serves a live llms.txt index (HTTP 200) and .md-suffixed doc pages explicitly designed for agent consumption, with the quickstart page itself pointing agents to the llms.txt index. missing for 10: no independent/community confirmation that agents actually consume llms.txt successfully in practice.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.e2b.dev/llms.txt # E2B Docs - [E2B Documentation](https://docs.e2b.dev/index.md) - [Running your f…”
   • [probe] “PROBE docs-md: HTTP 200 at https://docs.e2b.dev/quickstart.md > ## Documentation Index > Fetch the complete documentation index at: https://…”

   proof ↗⚑ flag

   Modal Sandboxes✓fullprobed8/10

   Modal has a working llms.txt (HTTP 200) with a documentation summary, and docs pages are also served as markdown (.md suffix) confirming agent-oriented doc access. missing for 10: no independent confirmation that agents actually consume this llms.txt successfully in practice, and no dedicated agent-docs index beyond the generic llms.txt.

   • [probe] “PROBE llms.txt: HTTP 200 at https://modal.com/llms.txt # Modal Documentation > Modal is a platform for running AI workloads in the cloud wi…”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round drawn
   E2B✓fullclaimed8/10

   E2B has a documented CLI, SDK, and non-interactive auth mode explicitly recommended for CI/CD pipelines, servers, and containers, plus scriptable sandbox lifecycle (create, exec with --background, shutdown, snapshot) and webhooks for lifecycle events — all consistent with headless/CI automation. missing for 10: no independent third-party report of a real CI pipeline using E2B, and no explicit CI provider integration examples (e.g., GitHub Actions workflow snippet).

   • [claimed-docs] “This is the recommended option for non-interactive environments like CI/CD pipelines, servers, and containers. Set a single environment vari…”
   • [claimed-docs] “E2B CLI is a command line tool that allows you to manage and interact with sandboxes and templates.”
   • [claimed-docs] “Use the `--background` flag to run a command in the background and return immediately.”
   • [claimed-docs] “You can shutdown single or all running sandboxes with the E2B CLI.”
   • [claimed-docs] “Webhooks provide a way for notifications to be delivered to an external web server whenever certain sandbox lifecycle events occur.”
   • [claimed-docs] “If you need an explicit configuration — for example several API keys, teams, or BYOC or self-hosted deployments — create an `E2B` client and…”
   • [claimed-docs] “If you need an explicit configuration — for example several API keys, teams, or [BYOC or self-hosted deployments](/byoc) in a single process…”

   proof ↗⚑ flag

   Modal Sandboxes✓fullprobed8/10

   Modal Sandboxes is built for programmatic/headless use via Python, JS/Go SDKs and a CLI, with sandboxes created and controlled entirely through code (modal-docs-2, modal-docs-19, modal-docs-20, modal-gh-1). Community evidence confirms real-world CI/automation usage — a developer built GitHub Actions self-hosted runners on Modal Sandboxes explicitly for scalable, pay-per-use automation (modal-comm-1), and another wired gRPC control for an agent runtime (modal-comm-2). Missing for 10: no first-party CI/CD integration guide (e.g., official GitHub Actions/Jenkins docs) or dedicated 'headless CI' documentation section beyond community projects.

   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “Create a Sandbox with the specified Image, and mount a Volume”
   • [github] “The JS and Go SDKs allow you to use Modal Sandboxes, invoke deployed Modal Functions, and interact with some Modal platform resources.”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”
   • [claimed-docs] “Run a command or interactive shell inside a Modal container.”
   • [probe] “official CLI documented at https://modal.com/docs/reference/cli”

   proof ↗⚑ flag

3. ai-native userConnect an agent via an official MCP server

   weight 3 · round drawn
   E2B—none0/10

   E2B is a sandbox/code-execution infrastructure product (not itself an agent), so an official MCP server exposing its sandbox tools to agents is a fair, applicable axis. The evidence pack documents SDKs, CLI, templates, and LLM tool-use integration (e2b-docs-32) but never mentions an MCP server, official MCP integration, or MCP protocol support anywhere.

   • [claimed-docs] “E2B can work with any LLM and AI framework. The easiest way to connect an LLM to E2B is to use the tool use capabilities of the LLM”
   • [claimed-docs] “E2B CLI is a command line tool that allows you to manage and interact with sandboxes and templates.”
   • [github] “E2B is an open-source infrastructure that allows you to run AI-generated code in secure isolated sandboxes in the cloud.”

   proof ↗⚑ flag

   Modal Sandboxes—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

4. ai-native userUse an official CLI

   weight 2 · round drawn
   E2B✓fullprobed8/10

   E2B ships an official CLI documented extensively (create/connect/exec/list/shutdown/snapshot/auth/metrics commands) covering sandbox lifecycle management, well-suited for AI-native agentic workflows. missing for 10: independent/hands-on community validation of the CLI itself (community evidence only covers SDK usage, not CLI) and no evidence of CLI-specific agent integration examples.

   • [claimed-docs] “E2B CLI is a command line tool that allows you to manage and interact with sandboxes and templates.”
   • [claimed-docs] “You can create a sandbox and connect an interactive terminal to it.”
   • [claimed-docs] “This will return running sandboxes, you can specify `--state` to get paused or both.”
   • [claimed-docs] “You can shutdown single or all running sandboxes with the E2B CLI.”
   • [claimed-docs] “You can manage sandbox snapshots with the `e2b sandbox snapshot` (alias `snap`) commands.”
   • [claimed-docs] “If your E2B account has access to more than one project, you can choose which one the CLI uses”
   • [claimed-docs] “This is the recommended option for non-interactive environments like CI/CD pipelines, servers, and containers. Set a single environment vari…”
   • [claimed-docs] “e2b sandbox create <template>”
   • [claimed-docs] “Unlike the `create` command, `connect` does not kill the sandbox when you disconnect.”
   • [claimed-docs] “Use the `--background` flag to run a command in the background and return immediately.”
   • [claimed-docs] “To shutdown all running sandboxes, run the following command:”
   • [claimed-docs] “You can name the snapshot template with the `--name` flag. If a template with that name already exists, it is reused.”
   • [probe] “official CLI documented at https://docs.e2b.dev/cli”

   proof ↗⚑ flag

   Modal Sandboxes✓fullprobed8/10

   Modal ships an official CLI (documented at modal.com/docs/reference/cli) supporting sandbox-related operations like running commands/shells and managing/connecting to containers, confirmed by probe access to the CLI reference page. This CLI is core to Modal's agentic/AI-native workflows (creating and interacting with sandboxes for untrusted/agent code). missing for 10: independent hands-on user reports specifically praising/using the CLI (vs. SDK), and more detail on CLI coverage of full sandbox lifecycle (snapshots, networking) beyond the two documented subcommands.

   • [claimed-docs] “Run a command or interactive shell inside a Modal container.”
   • [claimed-docs] “Manage and connect to running containers.”
   • [probe] “official CLI documented at https://modal.com/docs/reference/cli”
   • [claimed-docs] “Running commands”

   proof ↗⚑ flag

5. ai-native userDrive the product through a documented public API

   weight 3 · round drawn
   E2B✓fullprobed8/10

   E2B ships extensive documented SDK/CLI APIs (Sandbox.create, files.read/write, commands.run, timeouts, metrics, snapshots, webhooks) with quickstart and reference docs, and explicit LLM/agent integration guides (connect-llms, coding-agents), demonstrating a fully documented public API surface for AI-native use. Missing for 10: no discoverable formal OpenAPI/swagger spec (probe found 404s across candidate paths) and no independent third-party audit of API completeness beyond vendor docs/community anecdotes.

   • [claimed-docs] “const sbx = await Sandbox.create() // Creates a persistent sandbox session const execution = await sbx.runCode('print("hello world")')”
   • [claimed-docs] “You can write single files to the sandbox filesystem using the `files.write()` method.”
   • [claimed-docs] “You can run terminal commands inside the sandbox using the `commands.run()` method.”
   • [claimed-docs] “E2B CLI is a command line tool that allows you to manage and interact with sandboxes and templates.”
   • [claimed-docs] “If you need an explicit configuration — for example several API keys, teams, or BYOC or self-hosted deployments — create an `E2B` client and…”
   • [claimed-docs] “E2B can work with any LLM and AI framework. The easiest way to connect an LLM to E2B is to use the tool use capabilities of the LLM”
   • [claimed-docs] “Run AI coding agents like Claude Code, Codex, and Amp in secure E2B sandboxes with full terminal, filesystem, and git access.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.e2b.dev/openapi.json, https://docs.e2b.dev/swagger.json, https://docs.e2b.dev/api/opena…”

   proof ↗⚑ flag

   Modal Sandboxes✓fullprobed8/10

   Modal exposes documented Python, JS, and Go SDKs plus a CLI for programmatically creating, controlling, and interacting with Sandboxes (exec, filesystem, snapshots, networking, logs), and community developers confirm driving it via gRPC/SDK for agent runtimes. missing for 10: no public OpenAPI/REST spec (probe found only 404s), so the 'documented public API' is SDK/CLI-based rather than a formal API contract, and there's no independent audit of API stability/versioning.

   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “Create a Sandbox with the specified Image, and mount a Volume”
   • [claimed-docs] “const p = await sb.exec(["cat", "/mnt/volume/message.txt"]);”
   • [github] “The JS and Go SDKs allow you to use Modal Sandboxes, invoke deployed Modal Functions, and interact with some Modal platform resources.”
   • [claimed-docs] “Run a command or interactive shell inside a Modal container.”
   • [claimed-docs] “Manage and connect to running containers.”
   • [claimed-docs] “We've added a modal.Sandbox.logs API, allowing you to fetch() logs from a specific date/time range or to tail() the most recent logs.”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://modal.com/openapi.json, https://modal.com/swagger.json, https://modal.com/api/openapi.json, …”

   proof ↗⚑ flag

6. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round drawn
   E2B—none0/10

   Evidence shows E2B supports multiple API keys per team/project (e2b-docs-30, e2b-docs-31, e2b-docs-34, e2b-docs-35) but no documentation of scoped or least-privilege permissions (e.g., read-only, sandbox-restricted, rate-limited keys) that would let an AI-native user issue a restricted credential to an agent.

   • [claimed-docs] “If you need an explicit configuration — for example several API keys, teams, or BYOC or self-hosted deployments — create an `E2B` client and…”
   • [claimed-docs] “If your E2B account has access to more than one project, you can choose which one the CLI uses”
   • [claimed-docs] “If you need an explicit configuration — for example several API keys, teams, or [BYOC or self-hosted deployments](/byoc) in a single process…”
   • [claimed-docs] “This is the recommended option for non-interactive environments like CI/CD pipelines, servers, and containers. Set a single environment vari…”

   proof ↗⚑ flag

   Modal Sandboxes—none0/10

   The evidence describes secure sandboxed execution environments and general platform features, but there is no concrete evidence of a mechanism for issuing scoped/least-privilege API credentials specifically to agents—only a vague marketing line about 'credentials already in place' (modal-docs-3) with no documentation of credential scoping, token permissions, or per-agent access control. Missing for 10: any documented API/credential-scoping mechanism, permission granularity docs, or examples of issuing least-privilege tokens to agents.

   • [claimed-docs] “Autonomous agents with the right tools, context, and credentials already in place — running securely in a full, isolated dev environment.”

   proof ↗⚑ flag

7. ai-native userBuild against official SDKs

   weight 2 · round drawn
   E2B✓fullprobed8/10

   E2B ships official SDKs (JS/Python) with extensive first-party docs covering sandbox creation, code execution, filesystem, commands, metrics, persistence, and explicit guidance for connecting LLMs/agent frameworks (e2b-docs-1,7,8,32,42-44), plus a CLI (e2b-docs-9, e2b-probe-4) and community confirmation of Python/TS SDK availability (e2b-comm-1). missing for 10: broader independent/hands-on validation beyond one HN thread, and no discoverable OpenAPI spec for programmatic SDK generation (e2b-probe-3).

   • [claimed-docs] “const sbx = await Sandbox.create() // Creates a persistent sandbox session const execution = await sbx.runCode('print("hello world")')”
   • [claimed-docs] “You can write single files to the sandbox filesystem using the `files.write()` method.”
   • [claimed-docs] “You can run terminal commands inside the sandbox using the `commands.run()` method.”
   • [claimed-docs] “E2B CLI is a command line tool that allows you to manage and interact with sandboxes and templates.”
   • [claimed-docs] “E2B can work with any LLM and AI framework. The easiest way to connect an LLM to E2B is to use the tool use capabilities of the LLM”
   • [claimed-docs] “E2B provides a pre-built `claude` template with Claude Code already installed.”
   • [claimed-docs] “With the E2B backend, the agent's built-in tools all execute inside an isolated sandbox instead of your machine”
   • [claimed-docs] “your agent's code runs in an isolated Firecracker microVM instead of your server, with a real Debian toolchain, pause/resume persistence, an…”
   • [community] “E2B does support Python, AgentKit is only available in TypeScript for now.”
   • [probe] “official CLI documented at https://docs.e2b.dev/cli”

   proof ↗⚑ flag

   Modal Sandboxes✓fullcommunity8/10

   Modal ships official Python SDK plus JS/Go SDKs for Sandboxes with documented APIs (exec, filesystem, snapshots, networking, logs) and community developers confirm building against them (GitHub Actions runners, gRPC-based JS agent runtime). missing for 10: independent third-party SDK reference/benchmarks beyond Modal's own docs and one community mention, and no evidence of broader language SDK coverage (e.g. Rust, Java).

   • [github] “The JS and Go SDKs allow you to use Modal Sandboxes, invoke deployed Modal Functions, and interact with some Modal platform resources.”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “Create a Sandbox with the specified Image, and mount a Volume”
   • [claimed-docs] “const p = await sb.exec(["cat", "/mnt/volume/message.txt"]);”
   • [claimed-docs] “We've added a modal.Sandbox.logs API, allowing you to fetch() logs from a specific date/time range or to tail() the most recent logs.”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”

   proof ↗⚑ flag

8. ai-native userSubscribe to events via webhooks

   weight 2 · round to E2B
   E2B✓fullclaimed6/10

   E2B docs explicitly describe webhooks that deliver notifications to an external server on sandbox lifecycle events, directly matching the story. Missing for 10: detailed docs on event types/payload schema, setup/configuration steps, and independent/community corroboration of webhook usage.

   • [claimed-docs] “Webhooks provide a way for notifications to be delivered to an external web server whenever certain sandbox lifecycle events occur.”

   proof ↗⚑ flag

   Modal Sandboxes—none0/10

   No evidence anywhere in the pack of a webhook subscription mechanism or event-driven notification system for Modal Sandboxes; the docs focus on programmatic spin-up, execution, logging, and filesystem access, not outbound event/webhook subscriptions.

   ⚑ flag

Agentic features

1. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Modal Sandboxes
   E2B~partialclaimed5/10

   E2B provides persistent, resumable sandboxes, background command execution (--background flag), webhooks for lifecycle events, and long-running session support (up to 24h, pause/resume), which are building blocks for autonomous background automation. However, there's no evidence of a native scheduler/cron, no first-party 'automation' orchestration layer, and no documented trigger system beyond webhooks — the user must build the autonomy logic themselves using an external agent/LLM. missing for 10: native scheduling/cron or trigger orchestration, evidence of fully autonomous multi-step agent loops running unattended, independent case studies of long-running unattended automations.

   • [claimed-docs] “Pause a sandbox and resume it later from the exact state it was in, including the filesystem and the memory, so running processes and loaded…”
   • [claimed-docs] “A paused sandbox is kept indefinitely with no automatic deletion.”
   • [claimed-docs] “Sandboxes can run continuously for up to 24 hours (Pro) or 1 hour (Base). For longer workloads, use pause and resume”
   • [claimed-docs] “Webhooks provide a way for notifications to be delivered to an external web server whenever certain sandbox lifecycle events occur.”
   • [claimed-docs] “Use the `--background` flag to run a command in the background and return immediately.”
   • [claimed-docs] “Run AI coding agents like Claude Code, Codex, and Amp in secure E2B sandboxes with full terminal, filesystem, and git access.”

   proof ↗⚑ flag

   Modal Sandboxes~partialcommunity6/10

   Modal Sandboxes explicitly supports running autonomous agent code in isolated environments at scale (docs-3, docs-4, docs-22), and community evidence shows real-world use for background automation (self-hosted CI runners on modal-comm-1, agent runtimes on modal-comm-2). However, the evidence pack lacks explicit documentation of scheduling/cron triggers or persistent background job orchestration—sandboxes are framed as on-demand execution environments rather than a native 'set it and forget it' automation scheduler. Missing for 10: native scheduling/cron trigger docs, first-party persistent orchestration for long-running background automations.

   • [claimed-docs] “Autonomous agents with the right tools, context, and credentials already in place — running securely in a full, isolated dev environment.”
   • [claimed-docs] “Spin up hundreds of thousands of concurrent rollout environments in seconds.”
   • [claimed-docs] “We execute the agent's code in a Modal Sandbox, which allows us to run arbitrary code in a safe environment.”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”

   proof ↗⚑ flag

2. ai-native userOperate the product with natural-language commands

   weight 2 · round to E2B
   E2B~partialclaimed5/10

   E2B's own interface is programmatic (SDK calls like Sandbox.create(), files.read(), commands.run()) and a CLI (e2b-docs-9), not a natural-language command surface; however docs explicitly describe connecting LLMs via tool-use so an AI agent can drive E2B through natural-language-triggered function calls (e2b-docs-32, e2b-docs-28, e2b-docs-43). This shows AI-native operation is possible only indirectly through an LLM/agent layer, not as a built-in NL command mode of the product itself. missing for 10: no first-party NL chat/command interface for E2B itself, no evidence of a natural-language CLI or prompt-driven control plane, only indirect LLM tool-use integration documented.

   • [claimed-docs] “E2B can work with any LLM and AI framework. The easiest way to connect an LLM to E2B is to use the tool use capabilities of the LLM”
   • [claimed-docs] “Run AI coding agents like Claude Code, Codex, and Amp in secure E2B sandboxes with full terminal, filesystem, and git access.”
   • [claimed-docs] “With the E2B backend, the agent's built-in tools all execute inside an isolated sandbox instead of your machine”
   • [claimed-docs] “E2B CLI is a command line tool that allows you to manage and interact with sandboxes and templates.”

   proof ↗⚑ flag

   Modal Sandboxes—none0/10

   Modal Sandboxes is controlled exclusively via SDK (Python/JS/Go) calls and a traditional CLI (modal run, modal shell, etc.); there is no evidence of a natural-language command interface or chat-driven control layer for operating sandboxes. Evidence shows programmatic/API-first design (modal-docs-2, modal-docs-15, modal-docs-19) rather than any NL-command capability.

   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Run a command or interactive shell inside a Modal container.”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [probe] “official CLI documented at https://modal.com/docs/reference/cli”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   E2B—none0/10

   Evidence shows extensive static documentation with code snippets (e2b-docs-1, e2b-docs-2, etc.) but no evidence of an interactive, runnable API reference — the probe explicitly found no OpenAPI/Swagger spec (e2b-probe-3), and no playground or 'try it' console is mentioned anywhere in the pack.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.e2b.dev/openapi.json, https://docs.e2b.dev/swagger.json, https://docs.e2b.dev/api/opena…”
   • [claimed-docs] “const sbx = await Sandbox.create() // Creates a persistent sandbox session const execution = await sbx.runCode('print("hello world")')”
   • [claimed-docs] “We’ll write the minimal code for starting Sandbox, executing Python inside it and listing all files inside the root directory.”

   proof ↗⚑ flag

   Modal Sandboxes—none0/10

   Evidence shows standard docs pages, CLI reference, and SDK code snippets, but no interactive API reference or runnable-example playground; the openapi probe returned 404s and no docs mention an in-browser code runner or live API explorer.

   • [probe] “PROBE openapi: all candidate paths 404 (https://modal.com/openapi.json, https://modal.com/swagger.json, https://modal.com/api/openapi.json, …”
   • [claimed-docs] “Run a command or interactive shell inside a Modal container.”
   • [claimed-docs] “const p = await sb.exec(["cat", "/mnt/volume/message.txt"]);”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   E2B—none0/10

   E2B exposes SDKs and a CLI, but active probing found no OpenAPI/swagger spec at any standard location (all candidate paths returned 404), and no documentation page references a machine-readable API spec.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.e2b.dev/openapi.json, https://docs.e2b.dev/swagger.json, https://docs.e2b.dev/api/opena…”

   proof ↗⚑ flag

   Modal Sandboxes—none0/10

   The probe explicitly found all candidate OpenAPI/swagger endpoints return 404, and no other evidence pack item mentions a machine-readable API spec being published; Modal's public interface is SDK/CLI-based rather than a REST API with a spec.

   • [probe] “PROBE openapi: all candidate paths 404 (https://modal.com/openapi.json, https://modal.com/swagger.json, https://modal.com/api/openapi.json, …”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round to E2B
   E2B✓fullclaimed8/10

   E2B provides fully isolated, ephemeral cloud sandboxes (Firecracker microVMs) for running AI-generated code, with no connection to production systems — sandboxes are created fresh from templates, have their own filesystem/network, and can be killed or auto-expire, ensuring test isolation. Persistence/pause-resume and metrics further support safe iterative testing without touching real data. Missing for 10: explicit documentation or case study contrasting sandbox vs production data separation, and independent/hands-on verification beyond vendor docs.

   • [claimed-docs] “const sbx = await Sandbox.create() // Creates a persistent sandbox session const execution = await sbx.runCode('print("hello world")')”
   • [claimed-docs] “Pause a sandbox and resume it later from the exact state it was in, including the filesystem and the memory, so running processes and loaded…”
   • [claimed-docs] “You can shutdown the sandbox any time even before the timeout is up by calling the `kill` method.”
   • [claimed-docs] “Every sandbox has outbound internet access by default. Control and restrict it with a simple on/off switch or fine-grained allow and deny li…”
   • [claimed-docs] “E2B templates allow you to define custom sandboxes. You can define the base image, environment variables, files to copy, commands to run, an…”
   • [github] “E2B is an open-source infrastructure that allows you to run AI-generated code in secure isolated sandboxes in the cloud.”
   • [claimed-docs] “your agent's code runs in an isolated Firecracker microVM instead of your server, with a real Debian toolchain, pause/resume persistence, an…”

   proof ↗⚑ flag

   Modal Sandboxes✓fullprobed7/10

   Modal Sandboxes are explicitly designed as ephemeral, isolated containers for running untrusted or agent code, with filesystem, networking/security, and snapshot controls documented (modal-docs-2, modal-docs-8, modal-docs-9, modal-docs-10, modal-probe-2), and community examples show real usage isolating workloads from production infra (modal-comm-1, modal-comm-2). This satisfies the 'test without touching production' need since each sandbox is a fresh, isolated environment rather than shared prod state. Missing for 10: explicit documentation or case study framing isolation specifically in terms of 'production data' separation (e.g., no prod DB/staging parity guidance), and no independent audit confirming true isolation guarantees.

   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Filesystem Access”
   • [claimed-docs] “Snapshots”
   • [claimed-docs] “Networking and security”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “We execute the agent's code in a Modal Sandbox, which allows us to run arbitrary code in a safe environment.”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   E2B—none0/10

   The evidence pack covers SDK usage, CLI commands, sandbox lifecycle, and self-hosting, but contains no mention of API versioning scheme, changelog, or a documented deprecation policy; the openapi probe even returned 404s. This is a fair axis for an API-first product, but no evidence supports it.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.e2b.dev/openapi.json, https://docs.e2b.dev/swagger.json, https://docs.e2b.dev/api/opena…”

   proof ↗⚑ flag

   Modal Sandboxes—none0/10

   Evidence shows release notes and beta/alpha labels for some sandbox features, but no documentation of API versioning scheme or a deprecation policy; OpenAPI spec probes also 404. missing for 10: explicit versioning scheme documentation, deprecation policy page, changelog with deprecation notices, stability guarantees for GA vs beta APIs.

   • [claimed-docs] “VM Sandboxes Beta”
   • [claimed-docs] “Sidecars Alpha”
   • [claimed-docs] “We've added a modal.Sandbox.logs API, allowing you to fetch() logs from a specific date/time range or to tail() the most recent logs.”
   • [claimed-docs] “it's now possible to opt into a more performant Sandbox backend, which affords substantially higher creation rates and concurrency”
   • [probe] “PROBE openapi: all candidate paths 404 (https://modal.com/openapi.json, https://modal.com/swagger.json, https://modal.com/api/openapi.json, …”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Modal Sandboxes
   E2B~partialclaimed5/10

   E2B supports several bulk-adjacent capabilities: up to 100 concurrent sandboxes, writing multiple files at once, listing all sandboxes, and shutting down single or all running sandboxes via CLI. However there is no unified batch/bulk API (e.g., no evidence of submitting a list of code executions or files to process as one bulk call) — bulk behavior is assembled from separate CLI/SDK primitives rather than a first-class bulk-operations feature. Missing for 10: a documented batch execution or bulk-processing API, evidence of bulk operations on non-sandbox resources (e.g., bulk template management), and independent/hands-on validation of bulk workflows at scale.

   • [claimed-docs] “Up to 24-hour sandbox session length Up to 100 concurrently running sandboxes”
   • [claimed-docs] “You can also write multiple files to the sandbox.”
   • [claimed-docs] “This will return running sandboxes, you can specify `--state` to get paused or both.”
   • [claimed-docs] “You can shutdown single or all running sandboxes with the E2B CLI.”
   • [claimed-docs] “To shutdown all running sandboxes, run the following command:”
   • [claimed-docs] “Use the `--background` flag to run a command in the background and return immediately.”

   proof ↗⚑ flag

   Modal Sandboxes~partialcommunity6/10

   Modal supports spinning up hundreds of thousands of concurrent sandboxes/rollout environments programmatically, which is strong evidence of bulk/parallel scaling (modal-docs-4, modal-docs-14, modal-docs-5), and this is corroborated by community builds using it for scaled runner fleets (modal-comm-1). However, this is infrastructure-level bulk provisioning, not a documented 'bulk operations' API/pattern for operating across many existing items (e.g., batch file ops, batch snapshot management, or a dedicated batch API) — missing for 10: a first-party batch/bulk-operation API or workflow example (vs just scaling raw sandbox count), independent benchmarks confirming claimed concurrency at scale, and documentation of bulk management operations (e.g., bulk terminate/list/update) rather than just creation.

   • [claimed-docs] “Spin up hundreds of thousands of concurrent rollout environments in seconds.”
   • [claimed-docs] “Spin up hundreds of thousands of concurrent rollout environments in seconds. Fast enough to keep your GPU inference resources saturated acro…”
   • [claimed-docs] “H100s, A100s, A10Gs available on demand. Attach to any sandbox, scale to thousands of concurrent runs, pay by the second with no reserved ca…”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [claimed-docs] “Integrated logging and full visibility into every function, sandbox, and container.”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to E2B
   E2B~partialclaimed3/10

   E2B offers webhooks that deliver notifications for sandbox lifecycle events, which is a minimal building block for event-triggered automation, but there is no evidence of an actual rule-engine (defining conditional actions that execute automatically) beyond forwarding notifications to an external server. missing for 10: built-in conditional rule definitions, in-product automatic action execution, and any hands-on evidence of the webhook mechanism working end-to-end.

   • [claimed-docs] “Webhooks provide a way for notifications to be delivered to an external web server whenever certain sandbox lifecycle events occur.”

   proof ↗⚑ flag

   Modal Sandboxes—none0/10

   The evidence describes Modal Sandboxes as a programmatic, on-demand execution environment for untrusted/agent code, but nothing in the pack shows a rules/trigger engine that fires actions automatically on external events (e.g., webhooks, event-driven policies). Absent such evidence, this applicable automation-depth axis is unmet.

   ⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round drawn
   E2B—none0/10

   E2B provides sandbox creation, execution, persistence (pause/resume), and lifecycle webhooks, but nothing in the evidence pack describes a native scheduler, cron-like trigger, or recurring workflow mechanism — scheduling would have to be built by the user via external tooling. This axis is fair for an automation-focused sandbox platform, so absence of evidence means 'none' rather than 'na'.

   • [claimed-docs] “Pause a sandbox and resume it later from the exact state it was in, including the filesystem and the memory, so running processes and loaded…”
   • [claimed-docs] “A paused sandbox is kept indefinitely with no automatic deletion.”
   • [claimed-docs] “Webhooks provide a way for notifications to be delivered to an external web server whenever certain sandbox lifecycle events occur.”
   • [claimed-docs] “When you call the set timeout method, the sandbox timeout will be reset to the new value that you specified.”

   proof ↗⚑ flag

   Modal Sandboxes—none0/10

   Evidence covers ephemeral sandbox creation, execution, networking, snapshots, and SDKs, but there is no mention of cron-like scheduling, recurring triggers, or persistent workflow orchestration for Modal Sandboxes. missing for 10: any scheduling/cron API, recurring job trigger mechanism, or workflow orchestration documentation tied to Sandboxes.

   • [claimed-docs] “VM Sandboxes Beta”
   • [claimed-docs] “Running commands”
   • [claimed-docs] “Snapshots”

   proof ↗⚑ flag

Agent workloads

1. ai-native userRun coding agents like Claude Code or Codex inside the sandbox following the vendor's own recipe

   weight 2 · round to E2B
   E2B✓fullclaimed8/10

   E2B provides a dedicated vendor recipe ('Run AI coding agents like Claude Code, Codex, and Amp in secure E2B sandboxes with full terminal, filesystem, and git access') and a pre-built `claude` template with Claude Code already installed, plus similar guides for other agent frameworks. This is a first-party documented recipe with concrete setup instructions. Missing for 10: independent/hands-on confirmation that Codex specifically runs smoothly (community evidence only discusses a different agent framework's rough edges, not Claude Code/Codex directly), and no third-party benchmark of the recipe's reliability.

   • [claimed-docs] “Run AI coding agents like Claude Code, Codex, and Amp in secure E2B sandboxes with full terminal, filesystem, and git access.”
   • [claimed-docs] “E2B provides a pre-built `claude` template with Claude Code already installed.”
   • [claimed-docs] “With the E2B backend, the agent's built-in tools all execute inside an isolated sandbox instead of your machine”
   • [claimed-docs] “your agent's code runs in an isolated Firecracker microVM instead of your server, with a real Debian toolchain, pause/resume persistence, an…”

   proof ↗⚑ flag

   Modal Sandboxes—none0/10

   Modal's docs show general support for running 'agents' and 'untrusted code' in Sandboxes (modal-docs-3, modal-docs-22, modal-docs-19), but no evidence pack item mentions a vendor recipe, tutorial, or example specifically for running Claude Code or Codex inside a Sandbox.

   ⚑ flag

Workloads

1. developerRun a headless browser or full desktop environment inside the sandbox

   weight 2 · round to E2B
   E2B~partialclaimed6/10

   E2B's GitHub README confirms a separate 'Desktop SDK' providing mouse, keyboard, screenshot, application, and desktop streaming APIs for full desktop environments, but this is an add-on SDK rather than a built-in feature of the core Sandbox class documented elsewhere. There's no first-party doc page (e.g., under docs.e2b.dev) detailing headless browser support or code examples for it. missing for 10: dedicated docs/quickstart for the Desktop SDK, headless browser (e.g., Chromium/Playwright) usage examples, independent hands-on confirmation of desktop streaming reliability.

   • [github] “For mouse, keyboard, screenshot, application, and desktop streaming APIs, install the Desktop SDK”

   proof ↗⚑ flag

   Modal Sandboxes—none0/10

   Evidence describes Modal Sandboxes as isolated containers/VMs for running arbitrary code, agents, filesystem access, networking, and command execution, but nowhere mentions headless browser or full desktop/GUI environment support (e.g., Xvfb, VNC, browser automation tooling). No docs or community reports reference browser or desktop environments running inside sandboxes.

   ⚑ flag

2. developerAttach GPUs to sandboxed workloads

   weight 2 · round to Modal Sandboxes
   E2B—none0/10

   No evidence anywhere in the pack mentions GPU support, GPU attachment, or GPU-enabled sandboxes; docs only reference customizable CPU & RAM, disk, and memory metrics. This is a fair question for a sandbox/compute infrastructure product, but nothing indicates GPU capability is offered.

   • [claimed-docs] “Customize your Sandbox CPU & RAM”
   • [claimed-docs] “Up to 24-hour sandbox session length Up to 100 concurrently running sandboxes”
   • [claimed-docs] “The sandbox metrics allows you to get information about the sandbox's CPU, memory and disk usage.”

   proof ↗⚑ flag

   Modal Sandboxes✓fullcommunity8/10

   Modal docs explicitly state GPUs (H100s, A100s, A10Gs) can be attached to any sandbox on demand with per-second billing, and community evidence (HN runner project) confirms real-world GPU-backed sandbox usage. missing for 10: independent hands-on benchmark of GPU attach performance/reliability, and explicit code example showing GPU parameter in sandbox creation API.

   • [claimed-docs] “H100s, A100s, A10Gs available on demand. Attach to any sandbox, scale to thousands of concurrent runs, pay by the second with no reserved ca…”
   • [claimed-docs] “Attach to any sandbox, scale to thousands of concurrent runs, pay by the second with no reserved capacity.”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”

   proof ↗⚑ flag

Files

1. developerRead, write, upload, and download files in the sandbox filesystem via the SDK

   weight 2 · round drawn
   E2B✓fullclaimed8/10

   E2B SDK docs clearly document files.write() and files.read() for single and multiple files, plus files.list() for directory listing, covering read/write/list operations via the SDK. Upload/download is implicit in write/read of file content but not explicitly named as 'upload/download' with byte-stream examples. missing for 10: explicit upload/download terminology or examples (e.g., binary file transfer, streaming), independent/hands-on corroboration beyond docs.

   • [claimed-docs] “You can write single files to the sandbox filesystem using the `files.write()` method.”
   • [claimed-docs] “You can read files from the sandbox filesystem using the `files.read()` method.”
   • [claimed-docs] “You can also write multiple files to the sandbox.”
   • [claimed-docs] “const files = await sbx.files.list('/')”

   proof ↗⚑ flag

   Modal Sandboxes✓fullprobed8/10

   Modal explicitly documents a dedicated 'Filesystem Access' guide for sandboxes (modal-docs-8) alongside SDK examples showing file read/write via mounted volumes and exec commands (modal-docs-20, modal-docs-21), confirming read/write/upload/download-style operations through the SDK. missing for 10: no independent/hands-on community confirmation specifically of file upload/download methods (only exec/volume examples), and no explicit code snippet showing a dedicated upload/download API method distinct from volume mounting.

   • [claimed-docs] “Filesystem Access”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “Create a Sandbox with the specified Image, and mount a Volume”
   • [claimed-docs] “const p = await sb.exec(["cat", "/mnt/volume/message.txt"]);”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”

   proof ↗⚑ flag

Runtimes

1. developerDefine custom sandbox templates or bring my own container image

   weight 2 · round to E2B
   E2B✓fullclaimed9/10

   E2B templates explicitly let developers define a custom base image, environment variables, files, commands, and start command, and the CLI supports building/naming/snapshotting templates for reuse; this is well-documented first-party functionality with CLI tooling support. Missing for 10: no independent/hands-on third-party confirmation of custom template builds succeeding in practice.

   • [claimed-docs] “E2B templates allow you to define custom sandboxes. You can define the base image, environment variables, files to copy, commands to run, an…”
   • [claimed-docs] “This gives you fully configured sandboxes with running processes ready to use with zero wait time for your users.”
   • [claimed-docs] “You can name the snapshot template with the `--name` flag. If a template with that name already exists, it is reused.”
   • [claimed-docs] “You can manage sandbox snapshots with the `e2b sandbox snapshot` (alias `snap`) commands.”
   • [claimed-docs] “e2b sandbox create <template>”
   • [claimed-docs] “E2B CLI is a command line tool that allows you to manage and interact with sandboxes and templates.”

   proof ↗⚑ flag

   Modal Sandboxes✓fullcommunity8/10

   Modal explicitly supports creating Sandboxes with custom Images ('Custom images, any dependency' and 'Create a Sandbox with the specified Image, and mount a Volume'), and community projects (GitHub Actions runners, MCP-JS) confirm developers configuring custom sandbox environments in practice. Missing for 10: explicit documentation of importing arbitrary Docker/OCI registry images or a dedicated 'bring your own container' guide beyond the Image API reference.

   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Create a Sandbox with the specified Image, and mount a Volume”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”

   proof ↗⚑ flag

2. developerExecute code in multiple language runtimes (Python, JavaScript, and more) and get rich results back

   weight 2 · round to E2B
   E2B✓fullclaimed8/10

   E2B's docs show runCode execution (e2b-docs-1, e2b-docs-21) with sandboxes supporting Python and JavaScript SDKs, plus file I/O and terminal command execution for rich interaction with results (e2b-docs-7, e2b-docs-8, e2b-docs-19). It's explicitly positioned for running AI-generated code across languages with tool-use integration (e2b-docs-32, e2b-docs-33). missing for 10: explicit enumeration of all supported language runtimes beyond Python/JS/CLI, and independent hands-on confirmation of 'rich results' (e.g., charts/images) beyond docs claims.

   • [claimed-docs] “const sbx = await Sandbox.create() // Creates a persistent sandbox session const execution = await sbx.runCode('print("hello world")')”
   • [claimed-docs] “We’ll write the minimal code for starting Sandbox, executing Python inside it and listing all files inside the root directory.”
   • [claimed-docs] “You can write single files to the sandbox filesystem using the `files.write()` method.”
   • [claimed-docs] “You can run terminal commands inside the sandbox using the `commands.run()` method.”
   • [claimed-docs] “You can read files from the sandbox filesystem using the `files.read()` method.”
   • [claimed-docs] “E2B can work with any LLM and AI framework. The easiest way to connect an LLM to E2B is to use the tool use capabilities of the LLM”
   • [claimed-docs] “You can use E2B Sandbox to run AI-generated code to analyze data.”

   proof ↗⚑ flag

   Modal Sandboxes~partialclaimed6/10

   Modal Sandboxes let you run arbitrary commands/code in isolated containers with custom images (any dependency, any language stack), including SDK examples for JS/Go exec and Python examples of running agent code (modal-docs-2, modal-docs-13, modal-docs-19-21). However, evidence never shows explicit multi-language runtime presets (Python vs JS vs others) or structured/rich result payloads (e.g., images, dataframes, mime-typed outputs) — outputs shown are plain stdout/exec results. missing for 10: explicit built-in multi-language runtime templates, documented rich/structured result formats beyond raw stdout.

   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Running commands”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “Create a Sandbox with the specified Image, and mount a Volume”
   • [claimed-docs] “const p = await sb.exec(["cat", "/mnt/volume/message.txt"]);”
   • [claimed-docs] “We execute the agent's code in a Modal Sandbox, which allows us to run arbitrary code in a safe environment.”

   proof ↗⚑ flag

Untrusted code

1. developerExecute untrusted, AI-generated code without risking my own infrastructure

   weight 3 · round to E2B
   E2B✓fullclaimed9/10

   E2B is explicitly designed as isolated sandbox infrastructure (Firecracker microVMs) to run AI-generated/untrusted code without touching the developer's own machine, with persistence, network controls, resource limits, and self-hosting/BYOC options for stricter isolation needs (e2b-gh-3, e2b-docs-44, e2b-docs-10, e2b-gh-2). Docs also show direct integration for coding agents (Claude Code, Codex, Amp) executing in these sandboxes with full filesystem/terminal access (e2b-docs-28, e2b-docs-42). Missing for 10: independent third-party security audit or hands-on penetration-test evidence beyond vendor docs/GitHub description.

   • [github] “E2B is an open-source infrastructure that allows you to run AI-generated code in secure isolated sandboxes in the cloud.”
   • [claimed-docs] “your agent's code runs in an isolated Firecracker microVM instead of your server, with a real Debian toolchain, pause/resume persistence, an…”
   • [claimed-docs] “Every sandbox has outbound internet access by default. Control and restrict it with a simple on/off switch or fine-grained allow and deny li…”
   • [github] “Read the self-hosting guide to learn how to set up the E2B infrastructure on your own. The infrastructure is deployed using Terraform.”
   • [claimed-docs] “Run AI coding agents like Claude Code, Codex, and Amp in secure E2B sandboxes with full terminal, filesystem, and git access.”
   • [claimed-docs] “E2B provides a pre-built `claude` template with Claude Code already installed.”
   • [claimed-docs] “Pause a sandbox and resume it later from the exact state it was in, including the filesystem and the memory, so running processes and loaded…”

   proof ↗⚑ flag

   Modal Sandboxes✓fullprobed8/10

   Modal Sandboxes is explicitly built for running untrusted/AI-generated code in isolated, ephemeral containers with networking/filesystem controls, resource limits, and logging, and community examples show real-world use (self-hosted CI runners, agent runtimes) confirming the isolation model works in practice. Missing for 10: independent security audit or adversarial escape-testing evidence, and clearer documentation of resource/network isolation guarantees beyond marketing copy.

   • [claimed-docs] “Programmatically scale secure, ephemeral environments for running untrusted code.”
   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Networking and security”
   • [claimed-docs] “VM Sandboxes Beta”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “We execute the agent's code in a Modal Sandbox, which allows us to run arbitrary code in a safe environment.”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”

   proof ↗⚑ flag

2. developerRun arbitrary shell commands and install packages inside the sandbox

   weight 2 · round drawn
   E2B✓fullclaimed9/10

   E2B docs explicitly document commands.run() for executing arbitrary shell commands inside sandboxes, plus CLI exec-command support with background flag, and templates allow custom package installation via build commands. Combined with file read/write and persistent sandbox sessions, this directly delivers arbitrary shell execution and package installation. Missing for 10: independent hands-on benchmark specifically verifying package installation success/failure rates.

   • [claimed-docs] “You can run terminal commands inside the sandbox using the `commands.run()` method.”
   • [claimed-docs] “Use the `--background` flag to run a command in the background and return immediately.”
   • [claimed-docs] “E2B templates allow you to define custom sandboxes. You can define the base image, environment variables, files to copy, commands to run, an…”
   • [claimed-docs] “You can create a sandbox and connect an interactive terminal to it.”
   • [claimed-docs] “e2b sandbox create <template>”

   proof ↗⚑ flag

   Modal Sandboxes✓fullprobed9/10

   Modal Sandboxes documentation explicitly covers running arbitrary shell commands (sb.exec, CLI 'run a command or interactive shell inside a Modal container'), custom images with any dependency installation, filesystem access, and community reports (HN self-hosted runners, JS agent runtime) corroborate real-world use for arbitrary command execution and untrusted code. Missing for 10: no independent benchmark of package installation limits or explicit documented apt/pip install examples beyond generic 'any dependency' claim.

   • [claimed-docs] “Running commands”
   • [claimed-docs] “Run a command or interactive shell inside a Modal container.”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “const p = await sb.exec(["cat", "/mnt/volume/message.txt"]);”
   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”

   proof ↗⚑ flag

Agent lifecycle

1. ai-native userMy agent can provision its own sandbox, execute code, read the results, and tear it down — end to end without a human

   weight 3 · round to E2B
   E2B✓fullclaimed9/10

   E2B's SDK API directly supports programmatic sandbox creation, code execution, filesystem read/write, and teardown/kill — all callable by an agent without human intervention (e2b-docs-1, e2b-docs-6, e2b-docs-19, e2b-docs-8). Docs explicitly describe running AI coding agents like Claude Code/Codex end-to-end in sandboxes with full terminal/filesystem access (e2b-docs-28, e2b-docs-42, e2b-docs-43), and timeouts/kill provide automated teardown. Missing for 10: no independent hands-on benchmark of a fully autonomous end-to-end agent loop (only vendor docs and one community note about agent friction, not a failure of the core provision/execute/teardown flow).

   • [claimed-docs] “const sbx = await Sandbox.create() // Creates a persistent sandbox session const execution = await sbx.runCode('print("hello world")')”
   • [claimed-docs] “You can shutdown the sandbox any time even before the timeout is up by calling the `kill` method.”
   • [claimed-docs] “You can read files from the sandbox filesystem using the `files.read()` method.”
   • [claimed-docs] “You can run terminal commands inside the sandbox using the `commands.run()` method.”
   • [claimed-docs] “Run AI coding agents like Claude Code, Codex, and Amp in secure E2B sandboxes with full terminal, filesystem, and git access.”
   • [claimed-docs] “E2B provides a pre-built `claude` template with Claude Code already installed.”
   • [claimed-docs] “With the E2B backend, the agent's built-in tools all execute inside an isolated sandbox instead of your machine”
   • [claimed-docs] “When you call the set timeout method, the sandbox timeout will be reset to the new value that you specified.”

   proof ↗⚑ flag

   Modal Sandboxes✓fullprobed8/10

   Modal Sandboxes are explicitly designed for programmatic, API-driven provisioning, code execution, log/result retrieval, and teardown, with SDK examples (Python/JS/Go) showing sandbox creation, exec, file access, and logs, plus a documented agent-code-execution example. missing for 10: independent hands-on report showing a fully autonomous agent loop (provision->execute->read->teardown) with no human intervention, and clearer documentation of teardown/termination API specifics.

   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Autonomous agents with the right tools, context, and credentials already in place — running securely in a full, isolated dev environment.”
   • [claimed-docs] “Running commands”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “const p = await sb.exec(["cat", "/mnt/volume/message.txt"]);”
   • [claimed-docs] “We execute the agent's code in a Modal Sandbox, which allows us to run arbitrary code in a safe environment.”
   • [claimed-docs] “We've added a modal.Sandbox.logs API, allowing you to fetch() logs from a specific date/time range or to tail() the most recent logs.”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”

   proof ↗⚑ flag

Isolation

1. platform-engineerRely on a documented hard isolation boundary (microVM or equivalent) between sandboxes and my systems

   weight 3 · round to E2B
   E2B✓fullclaimed8/10

   E2B docs explicitly state that code runs 'in an isolated Firecracker microVM instead of your server' with network controls and pause/resume, directly documenting the hard isolation boundary platform engineers need. Missing for 10: independent third-party security audit or pentest report, and more detail on the isolation boundary's guarantees beyond a single doc line.

   • [claimed-docs] “your agent's code runs in an isolated Firecracker microVM instead of your server, with a real Debian toolchain, pause/resume persistence, an…”
   • [claimed-docs] “With the E2B backend, the agent's built-in tools all execute inside an isolated sandbox instead of your machine”
   • [claimed-docs] “Every sandbox has outbound internet access by default. Control and restrict it with a simple on/off switch or fine-grained allow and deny li…”
   • [github] “E2B is an open-source infrastructure that allows you to run AI-generated code in secure isolated sandboxes in the cloud.”

   proof ↗⚑ flag

   Modal Sandboxes~partialprobed5/10

   Modal repeatedly describes Sandboxes as 'secure,' 'isolated' containers for untrusted code, and references a 'VM Sandboxes Beta' page implying microVM-based isolation, but no evidence pack item actually documents the underlying isolation technology (e.g., Firecracker/gVisor specifics, threat model, or security boundary guarantees) that a platform engineer would need to rely on. Missing for 10: explicit technical documentation of the isolation mechanism (hypervisor/kernel boundary), any third-party security audit or independent verification of the boundary's robustness.

   • [claimed-docs] “VM Sandboxes Beta”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “We execute the agent's code in a Modal Sandbox, which allows us to run arbitrary code in a safe environment.”

   proof ↗⚑ flag

2. ai-native userGive an agent a sandbox where host secrets and credentials are unreachable by the code it runs

   weight 2 · round to E2B
   E2B✓fullclaimed7/10

   E2B's docs and repo repeatedly describe sandboxes as isolated Firecracker microVMs separate from the host/server ('your agent's code runs in an isolated Firecracker microVM instead of your server', 'isolated sandbox instead of your machine'), and network access to the sandbox can be restricted via allow/deny lists, all supporting the claim that a sandboxed agent cannot reach host resources. However, there is no explicit documentation or hands-on test specifically confirming host secrets/credentials (e.g., env vars, cloud IAM creds) are unreachable from sandboxed code, nor independent security audit evidence. Missing for 10: explicit secret/credential-isolation guarantees, third-party security audit or hands-on penetration test confirming host secrets are unreachable.

   • [github] “E2B is an open-source infrastructure that allows you to run AI-generated code in secure isolated sandboxes in the cloud.”
   • [claimed-docs] “your agent's code runs in an isolated Firecracker microVM instead of your server, with a real Debian toolchain, pause/resume persistence, an…”
   • [claimed-docs] “With the E2B backend, the agent's built-in tools all execute inside an isolated sandbox instead of your machine”
   • [claimed-docs] “Every sandbox has outbound internet access by default. Control and restrict it with a simple on/off switch or fine-grained allow and deny li…”

   proof ↗⚑ flag

   Modal Sandboxes~partialprobed5/10

   Modal Sandboxes are documented as isolated, ephemeral containers for running untrusted/agent code with dedicated networking-and-security guides, supporting the general isolation story (modal-docs-1, modal-docs-2, modal-docs-10, modal-probe-2, modal-docs-19, modal-docs-22). However, no evidence explicitly states that host secrets/credentials are unreachable by default or describes the secrets-injection model that would confirm this — one doc even implies credentials are provisioned into the agent environment (modal-docs-3), which cuts against a strong 'unreachable unless granted' guarantee. Missing for 10: explicit secrets-isolation documentation, details of the opt-in secrets mechanism, and independent/hands-on confirmation that host credentials cannot leak into sandboxed code.

   • [claimed-docs] “Programmatically scale secure, ephemeral environments for running untrusted code.”
   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Autonomous agents with the right tools, context, and credentials already in place — running securely in a full, isolated dev environment.”
   • [claimed-docs] “Networking and security”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “We execute the agent's code in a Modal Sandbox, which allows us to run arbitrary code in a safe environment.”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”

   proof ↗⚑ flag

Network policy

1. platform-engineerRestrict or allow the sandbox's network egress with explicit policy

   weight 3 · round to E2B
   E2B✓fullclaimed8/10

   E2B docs explicitly document network egress control: outbound internet access is on by default and can be restricted via 'a simple on/off switch or fine-grained allow and deny lists,' directly matching the platform-engineer's need for explicit egress policy. missing for 10: independent/hands-on verification of allow/deny list behavior, and detailed policy syntax/API reference beyond the summary.

   • [claimed-docs] “Every sandbox has outbound internet access by default. Control and restrict it with a simple on/off switch or fine-grained allow and deny li…”
   • [claimed-docs] “your agent's code runs in an isolated Firecracker microVM instead of your server, with a real Debian toolchain, pause/resume persistence, an…”

   proof ↗⚑ flag

   Modal Sandboxes~partialprobed5/10

   Modal has a dedicated 'Networking and security' guide for Sandboxes, confirming the platform documents network controls for sandboxes, but the evidence pack only shows the page title without excerpts detailing how egress allow/deny policies are configured. missing for 10: actual policy syntax/examples (e.g. allowlist/blocklist configuration), confirmation of default-deny behavior, independent/hands-on verification that egress restrictions work as documented.

   • [claimed-docs] “Networking and security”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round drawn
   E2B~partialprobed6/10

   E2B is API/SDK-first, so most core operations (sandbox create, exec, filesystem, commands, metrics, templates, snapshots, webhooks) are documented as API/CLI actions rather than gated behind a UI, and the CLI mirrors sandbox lifecycle management. However, some UI-surfaced features like Desktop streaming (mouse/keyboard/screenshot) require a separate Desktop SDK, and no public OpenAPI spec was found, making full parity hard to verify. Missing for 10: a confirmed OpenAPI/full REST spec, explicit confirmation that dashboard-only features (e.g., billing, team management) are also API-accessible, independent verification of full UI/API parity.

   • [claimed-docs] “const sbx = await Sandbox.create() // Creates a persistent sandbox session const execution = await sbx.runCode('print("hello world")')”
   • [claimed-docs] “E2B CLI is a command line tool that allows you to manage and interact with sandboxes and templates.”
   • [claimed-docs] “E2B templates allow you to define custom sandboxes. You can define the base image, environment variables, files to copy, commands to run, an…”
   • [claimed-docs] “You can manage sandbox snapshots with the `e2b sandbox snapshot` (alias `snap`) commands.”
   • [claimed-docs] “Webhooks provide a way for notifications to be delivered to an external web server whenever certain sandbox lifecycle events occur.”
   • [github] “For mouse, keyboard, screenshot, application, and desktop streaming APIs, install the Desktop SDK”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.e2b.dev/openapi.json, https://docs.e2b.dev/swagger.json, https://docs.e2b.dev/api/opena…”
   • [claimed-docs] “If you need an explicit configuration — for example several API keys, teams, or BYOC or self-hosted deployments — create an `E2B` client and…”

   proof ↗⚑ flag

   Modal Sandboxes~partialprobed6/10

   Modal Sandboxes is fundamentally API/SDK-driven (Python, JS/Go) with a full CLI for running commands and managing containers, and docs describe programmatic control over image building, execution, filesystem, networking, snapshots, and logs — suggesting the API is the primary and most complete interface. However, there's no explicit documentation stating full UI/API parity or listing any UI-exclusive features, and no OpenAPI spec is discoverable (probe found 404s), making it hard to confirm completeness of a public REST API surface beyond the SDKs/CLI. Missing for 10: explicit parity statement between dashboard UI and API/CLI capabilities, and a public API reference/spec confirming no UI-only actions.

   • [claimed-docs] “Filesystem Access”
   • [claimed-docs] “Snapshots”
   • [claimed-docs] “Networking and security”
   • [claimed-docs] “Running commands”
   • [claimed-docs] “Run a command or interactive shell inside a Modal container.”
   • [claimed-docs] “Manage and connect to running containers.”
   • [github] “The JS and Go SDKs allow you to use Modal Sandboxes, invoke deployed Modal Functions, and interact with some Modal platform resources.”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://modal.com/openapi.json, https://modal.com/swagger.json, https://modal.com/api/openapi.json, …”
   • [probe] “official CLI documented at https://modal.com/docs/reference/cli”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round to E2B
   E2B~partialclaimed4/10

   E2B is open-source and self-hostable (Terraform-based), and sandbox files can be read/downloaded via files.read()/CLI, giving a technical path to extract data and avoid lock-in, but there is no documented 'export all account data' feature or guarantee of open data formats for account-level artifacts like snapshots/templates. missing for 10: explicit data-export tooling, documented open format guarantees for exported data, account-level export across all sandboxes/snapshots.

   • [github] “Read the self-hosting guide to learn how to set up the E2B infrastructure on your own. The infrastructure is deployed using Terraform.”
   • [github] “E2B is an open-source infrastructure that allows you to run AI-generated code in secure isolated sandboxes in the cloud.”
   • [claimed-docs] “You can read files from the sandbox filesystem using the `files.read()` method.”
   • [claimed-docs] “You can manage sandbox snapshots with the `e2b sandbox snapshot` (alias `snap`) commands.”

   proof ↗⚑ flag

   Modal Sandboxes—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round to E2B
   E2B✓fullclaimed7/10

   GitHub repo description explicitly states 'E2B is an open-source infrastructure' and provides a self-hosting guide with Terraform, corroborating that the source is publicly available and usable, not just marketing docs. Missing for 10: explicit citation of the license name (e.g., Apache/MIT) and independent confirmation of license terms.

   • [github] “E2B is an open-source infrastructure that allows you to run AI-generated code in secure isolated sandboxes in the cloud.”
   • [github] “Read the self-hosting guide to learn how to set up the E2B infrastructure on your own. The infrastructure is deployed using Terraform.”

   proof ↗⚑ flag

   Modal Sandboxes—none0/10

   Evidence only shows a GitHub repo for the client SDK (modal-client) used to invoke Modal Sandboxes, not the underlying sandbox/runtime platform itself, and no license terms are mentioned. There is no evidence that Modal Sandboxes' core product source is published under an open license.

   • [github] “The JS and Go SDKs allow you to use Modal Sandboxes, invoke deployed Modal Functions, and interact with some Modal platform resources.”

   proof ↗⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round to E2B
   E2B~partialclaimed6/10

   E2B is explicitly open-source and provides a documented self-hosting guide using Terraform to deploy the infrastructure on your own, plus BYOC/self-hosted deployment options referenced in client docs. However, the evidence lacks depth on self-hosting: no details on required components, hardware/scale limits, maintenance burden, or independent reports of someone successfully self-hosting. missing for 10: detailed self-hosting setup docs/tutorial, independent verification of a working self-hosted deployment, and clarity on feature parity between self-hosted and cloud versions.

   • [github] “Read the self-hosting guide to learn how to set up the E2B infrastructure on your own. The infrastructure is deployed using Terraform.”
   • [github] “E2B is an open-source infrastructure that allows you to run AI-generated code in secure isolated sandboxes in the cloud.”
   • [claimed-docs] “If you need an explicit configuration — for example several API keys, teams, or BYOC or self-hosted deployments — create an `E2B` client and…”
   • [claimed-docs] “If you need an explicit configuration — for example several API keys, teams, or [BYOC or self-hosted deployments](/byoc) in a single process…”

   proof ↗⚑ flag

   Modal Sandboxes—none0/10

   Modal is a fully-managed cloud platform for running Sandboxes; there is no evidence of any self-hostable/on-prem version of the core product, and everything described (pay-per-second billing, GPU access, hosted CLI) implies a proprietary SaaS deployment model only.

   ⚑ flag

Latency

1. developerStart sandboxes with documented sub-second-to-few-second cold starts

   weight 3 · round to Modal Sandboxes
   E2B—none0/10

   The evidence pack describes sandbox creation, persistence, pausing/resuming, and templates with 'zero wait time,' but contains no documented benchmark or specific claim of sub-second-to-few-second cold start times. Missing for 10: any explicit startup latency figures, benchmark data, or documentation stating cold-start speed in seconds.

   • [claimed-docs] “const sbx = await Sandbox.create() // Creates a persistent sandbox session const execution = await sbx.runCode('print("hello world")')”
   • [claimed-docs] “This gives you fully configured sandboxes with running processes ready to use with zero wait time for your users.”
   • [claimed-docs] “E2B templates allow you to define custom sandboxes. You can define the base image, environment variables, files to copy, commands to run, an…”

   proof ↗⚑ flag

   Modal Sandboxes~partialclaimed4/10

   Evidence shows general speed/scale claims ("spin up hundreds of thousands of concurrent rollout environments in seconds," faster backend for higher creation rates) but no explicit documented cold-start latency benchmark (e.g., sub-second or few-second startup time) for sandboxes specifically. Missing for 10: a concrete cold-start time benchmark/number, independent performance verification, and comparison across sandbox sizes/images.

   • [claimed-docs] “Spin up hundreds of thousands of concurrent rollout environments in seconds.”
   • [claimed-docs] “Spin up hundreds of thousands of concurrent rollout environments in seconds. Fast enough to keep your GPU inference resources saturated acro…”
   • [claimed-docs] “it's now possible to opt into a more performant Sandbox backend, which affords substantially higher creation rates and concurrency”
   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”

   proof ↗⚑ flag

Scale

1. platform-engineerRun large concurrent fleets of sandboxes with documented concurrency limits

   weight 3 · round to E2B
   E2B~partialclaimed6/10

   E2B publishes concrete documented concurrency limits (up to 100 concurrently running sandboxes, up to 24-hour session length on Pro plans) and metrics/CLI tooling to manage fleets, giving platform engineers real numbers to plan around. However missing for 10: no documentation of how limits scale for enterprise/self-hosted deployments, no guidance on requesting higher concurrency, no independent/hands-on benchmarks validating fleet behavior at scale, and no discussion of failure modes or queuing when limits are hit.

   • [claimed-docs] “Up to 24-hour sandbox session length Up to 100 concurrently running sandboxes”
   • [claimed-docs] “Sandboxes can run continuously for up to 24 hours (Pro) or 1 hour (Base). For longer workloads, use pause and resume”
   • [claimed-docs] “The sandbox metrics allows you to get information about the sandbox's CPU, memory and disk usage.”
   • [claimed-docs] “Getting the metrics of a sandbox returns an array of timestamped metrics containing CPU, memory and disk usage information.”
   • [github] “Read the self-hosting guide to learn how to set up the E2B infrastructure on your own. The infrastructure is deployed using Terraform.”

   proof ↗⚑ flag

   Modal Sandboxes~partialcommunity5/10

   Modal markets very high concurrency ('hundreds of thousands of concurrent rollout environments', 'scale to thousands of concurrent runs') and even documents a backend upgrade for higher creation rates/concurrency, plus community reports of building scalable runner fleets on Sandboxes, showing the capability is real and used at scale. However there is no evidence of documented hard limits, quotas, or per-account/per-plan concurrency ceilings that a platform engineer could plan capacity against. Missing for 10: explicit documented concurrency/rate limits or quotas, benchmark/load-test data, and any per-tier scaling caps.

   • [claimed-docs] “Spin up hundreds of thousands of concurrent rollout environments in seconds.”
   • [claimed-docs] “H100s, A100s, A10Gs available on demand. Attach to any sandbox, scale to thousands of concurrent runs, pay by the second with no reserved ca…”
   • [claimed-docs] “Spin up hundreds of thousands of concurrent rollout environments in seconds. Fast enough to keep your GPU inference resources saturated acro…”
   • [claimed-docs] “Attach to any sandbox, scale to thousands of concurrent runs, pay by the second with no reserved capacity.”
   • [claimed-docs] “it's now possible to opt into a more performant Sandbox backend, which affords substantially higher creation rates and concurrency”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”

   proof ↗⚑ flag

2. developerKeep a sandbox session running for hours or days for long agent tasks

   weight 2 · round to E2B
   E2B~partialclaimed7/10

   E2B sandboxes can run continuously for hours (up to 24h on Pro plans, with configurable/resettable timeouts via setTimeout), covering the 'hours' part of the story fully, and for multi-day tasks E2B explicitly recommends pause/resume to preserve full memory and filesystem state indefinitely rather than true continuous execution. This satisfies long-running agent tasks but with a documented architectural limitation (base tier capped at 1h, Pro at 24h continuous) requiring the pause/resume workaround for day-scale persistence. Missing for 10: evidence of truly continuous multi-day execution without pause/resume, and independent/hands-on confirmation of stability over such long sessions.

   • [claimed-docs] “Sandboxes can run continuously for up to 24 hours (Pro) or 1 hour (Base). For longer workloads, use pause and resume”
   • [claimed-docs] “Pause a sandbox and resume it later from the exact state it was in, including the filesystem and the memory, so running processes and loaded…”
   • [claimed-docs] “A paused sandbox is kept indefinitely with no automatic deletion.”
   • [claimed-docs] “When you call the set timeout method, the sandbox timeout will be reset to the new value that you specified.”
   • [claimed-docs] “You can change the sandbox timeout when it's running by calling the `setTimeout` method in JavaScript or `set_timeout` method in Python.”
   • [claimed-docs] “Up to 24-hour sandbox session length Up to 100 concurrently running sandboxes”

   proof ↗⚑ flag

   Modal Sandboxes~partialcommunity5/10

   Modal docs describe sandboxes as isolated, scalable execution environments with snapshotting, filesystem, and networking support, and a community project (self-hosted GitHub Actions runners) demonstrates sandboxes being kept alive for extended, real workloads. However, no evidence pack item states explicit maximum session duration, timeout configuration, or guarantees for multi-day persistence, so long-running-session support is inferred rather than directly documented. Missing for 10: explicit docs on configurable/no timeout limits, guidance for multi-day sandbox lifecycles, and independent confirmation of hours/days-long sessions beyond one community anecdote.

   • [claimed-docs] “Snapshots”
   • [claimed-docs] “VM Sandboxes Beta”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [claimed-docs] “Integrated logging and full visibility into every function, sandbox, and container.”
   • [claimed-docs] “We've added a modal.Sandbox.logs API, allowing you to fetch() logs from a specific date/time range or to tail() the most recent logs.”

   proof ↗⚑ flag

Pricing

1. developerStart building with a free tier or included credits without talking to sales

   weight 1 · round to E2B
   E2B✓fullclaimed8/10

   E2B explicitly gives every new account $100 in credits with no sales contact required (e2b-docs-41), and the pricing page details a self-serve tier with concrete limits (24-hour sessions, 100 concurrent sandboxes, e2b-docs-14) alongside quickstart docs showing immediate SDK usage without any account-approval step. missing for 10: no independent/community confirmation of the signup flow or credit amount, and no explicit mention of a free tier distinct from trial credits.

   • [claimed-docs] “Every new E2B account get $100 in credits.”
   • [claimed-docs] “Up to 24-hour sandbox session length Up to 100 concurrently running sandboxes”
   • [claimed-docs] “Customize your Sandbox CPU & RAM”
   • [claimed-docs] “const sbx = await Sandbox.create() // Creates a persistent sandbox session const execution = await sbx.runCode('print("hello world")')”

   proof ↗⚑ flag

   Modal Sandboxes—none0/10

   The evidence pack includes pricing-related docs (pay-per-use, burst capacity) but no mention of a free tier, free credits, or self-serve signup without contacting sales; nothing confirms developers can start building without sales contact.

   ⚑ flag

2. platform-engineerPay per second only for the compute a sandbox actually uses

   weight 3 · round to Modal Sandboxes
   E2B~partialclaimed4/10

   The pricing page confirms usage-based configuration (customizable CPU/RAM, session-length/concurrency tiers) and the SDK/CLI let you kill a sandbox or reset its timeout at any moment, plus real-time CPU/memory/disk metrics — all consistent with paying only for compute actually consumed. However, no evidence pack item explicitly states a per-second billing granularity or shows a billing/invoice example. Missing for 10: explicit per-second billing rate documentation, an invoice/billing dashboard example, or a stated billing unit (e.g., 'billed per second of sandbox uptime').

   • [claimed-docs] “Customize your Sandbox CPU & RAM”
   • [claimed-docs] “Up to 24-hour sandbox session length Up to 100 concurrently running sandboxes”
   • [claimed-docs] “When you call the set timeout method, the sandbox timeout will be reset to the new value that you specified.”
   • [claimed-docs] “You can shutdown the sandbox any time even before the timeout is up by calling the `kill` method.”
   • [claimed-docs] “The sandbox metrics allows you to get information about the sandbox's CPU, memory and disk usage.”
   • [claimed-docs] “Getting the metrics of a sandbox returns an array of timestamped metrics containing CPU, memory and disk usage information.”

   proof ↗⚑ flag

   Modal Sandboxes✓fullcommunity8/10

   Modal's pricing docs and marketing explicitly state per-second billing with no reserved capacity ('pay by the second with no reserved capacity', 'only pay for what you use... burst up to what you need without over-allocating'), directly matching the story, and a community user corroborates paying only for resources actually consumed when building GPU runners on Sandboxes. Missing for 10: independent verification of exact billing granularity/metering mechanics and no first-party pricing breakdown showing per-second line items in the evidence pack.

   • [claimed-docs] “Only pay for what you use. Burst up to what you need without over-allocating CPU or memory in advance.”
   • [claimed-docs] “Attach to any sandbox, scale to thousands of concurrent runs, pay by the second with no reserved capacity.”
   • [claimed-docs] “Burst up to what you need without over-allocating CPU or memory in advance.”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round to E2B
   E2B~partialclaimed4/10

   E2B docs mention BYOC and self-hosted deployment options (Terraform-based) which would let an organization control where infrastructure/data physically resides, but there is no evidence of an explicit region/residency selector or documented list of hosting regions for the standard managed service. missing for 10: explicit region-selection UI/API, documented list of available data-residency regions, compliance certifications tied to specific regions.

   • [claimed-docs] “If you need an explicit configuration — for example several API keys, teams, or BYOC or self-hosted deployments — create an `E2B` client and…”
   • [claimed-docs] “If you need an explicit configuration — for example several API keys, teams, or [BYOC or self-hosted deployments](/byoc) in a single process…”
   • [github] “Read the self-hosting guide to learn how to set up the E2B infrastructure on your own. The infrastructure is deployed using Terraform.”

   proof ↗⚑ flag

   Modal Sandboxes—none0/10

   No evidence pack item mentions data residency, region selection, or geographic data storage controls for Modal Sandboxes; the pack covers execution, scaling, GPUs, filesystem, and networking but nothing about choosing storage region/residency.

   ⚑ flag

2. ai-native userControl data retention and deletion

   weight 2 · round to E2B
   E2B~partialclaimed6/10

   E2B gives concrete lifecycle controls over sandbox data: sandboxes can be killed/shutdown anytime, timeouts set/reset, and paused sandboxes are 'kept indefinitely with no automatic deletion' until explicitly resumed or removed via CLI shutdown/kill commands, giving users direct control over when sandbox state (filesystem, memory) persists or is destroyed. However, there's no evidence of broader data-retention policy documentation (e.g., logs, telemetry, backups) or explicit deletion/export guarantees at the account or compliance level (GDPR/SOC2-style retention statements). Missing for 10: explicit data-retention/deletion policy documentation beyond sandbox kill/pause mechanics, account-level data deletion guarantees, and compliance/regulatory retention statements.

   • [claimed-docs] “A paused sandbox is kept indefinitely with no automatic deletion.”
   • [claimed-docs] “You can shutdown the sandbox any time even before the timeout is up by calling the `kill` method.”
   • [claimed-docs] “You can change the sandbox timeout when it's running by calling the `setTimeout` method in JavaScript or `set_timeout` method in Python.”
   • [claimed-docs] “You can shutdown single or all running sandboxes with the E2B CLI.”
   • [claimed-docs] “To shutdown all running sandboxes, run the following command:”
   • [claimed-docs] “Pause a sandbox and resume it later from the exact state it was in, including the filesystem and the memory, so running processes and loaded…”

   proof ↗⚑ flag

   Modal Sandboxes—none0/10

   Evidence covers sandbox ephemerality, filesystem access, snapshots, and networking/security, but nothing describes data retention policies, deletion controls, or compliance/data-handling settings that a user could configure. missing for 10: explicit data retention policy documentation, user-controllable deletion/purge mechanisms, compliance certifications (e.g., SOC2/GDPR data handling), retention period configuration.

   ⚑ flag

3. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   E2B—none0/10

   No evidence pack item mentions telemetry, analytics collection, or an opt-out/opt-in setting for usage tracking; E2B's self-hosting option (e2b-gh-2) implies infrastructure control but does not document any telemetry toggle for the hosted product.

   ⚑ flag

   Modal Sandboxes—none0/10

   No evidence in the pack mentions telemetry, usage tracking, analytics collection, or an opt-out mechanism for Modal Sandboxes; the docs cover sandboxing, GPUs, filesystem, and logging but not data-collection/privacy controls.

   ⚑ flag

Lifecycle

1. developerSet timeouts so sandboxes shut down automatically and stop billing when idle or done

   weight 2 · round to E2B
   E2B✓fullclaimed8/10

   E2B documents explicit timeout controls (setTimeout/set_timeout), default and configurable session lengths (24h Pro/1h Base), automatic kill on timeout, and manual kill/shutdown via SDK and CLI, directly enabling idle sandboxes to auto-terminate. missing for 10: explicit first-party confirmation that billing stops precisely at timeout/kill (only inferred from lifecycle docs), and independent/hands-on verification of billing behavior.

   • [claimed-docs] “When you call the set timeout method, the sandbox timeout will be reset to the new value that you specified.”
   • [claimed-docs] “You can shutdown the sandbox any time even before the timeout is up by calling the `kill` method.”
   • [claimed-docs] “You can change the sandbox timeout when it's running by calling the `setTimeout` method in JavaScript or `set_timeout` method in Python.”
   • [claimed-docs] “Sandboxes can run continuously for up to 24 hours (Pro) or 1 hour (Base). For longer workloads, use pause and resume”
   • [claimed-docs] “You can shutdown single or all running sandboxes with the E2B CLI.”
   • [claimed-docs] “To shutdown all running sandboxes, run the following command:”

   proof ↗⚑ flag

   Modal Sandboxes—none0/10

   The evidence pack only mentions general pay-per-second billing and bursty scaling, but never documents an explicit sandbox timeout or idle-shutdown parameter/config. No citation confirms configurable timeouts that auto-terminate sandboxes when idle or complete.

   ⚑ flag

2. developerSpin up an isolated sandbox with one API/SDK call and get a live environment in seconds

   weight 3 · round to E2B
   E2B✓fullclaimed9/10

   Docs clearly show a single SDK call (Sandbox.create()) provisions an isolated sandbox with near-instant readiness, plus templates offering 'zero wait time' startup, CLI equivalents, and evidence of the core create/run/kill/timeout lifecycle. missing for 10: independent hands-on benchmarking of actual boot latency beyond vendor claims.

   • [claimed-docs] “const sbx = await Sandbox.create() // Creates a persistent sandbox session const execution = await sbx.runCode('print("hello world")')”
   • [claimed-docs] “This gives you fully configured sandboxes with running processes ready to use with zero wait time for your users.”
   • [claimed-docs] “const sbx = await Sandbox.create() // Creates a persistent sandbox session”
   • [claimed-docs] “We’ll write the minimal code for starting Sandbox, executing Python inside it and listing all files inside the root directory.”
   • [claimed-docs] “e2b sandbox create <template>”
   • [github] “E2B is an open-source infrastructure that allows you to run AI-generated code in secure isolated sandboxes in the cloud.”

   proof ↗⚑ flag

   Modal Sandboxes✓fullprobed8/10

   Docs and SDK references (Python, JS/Go) show a single call (e.g. modal.Sandbox.create) spins up an isolated, ephemeral container/VM in seconds, with fast creation rates highlighted (hundreds of thousands of concurrent environments in seconds, faster backend for higher creation rates). Community builders (HN posts) independently confirm using Modal Sandboxes programmatically for real workloads like CI runners and agent runtimes. Missing for 10: independent benchmark of actual cold-start latency and a public OpenAPI spec (probe found 404s), so exact 'seconds' claim isn't third-party verified.

   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Spin up hundreds of thousands of concurrent rollout environments in seconds.”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “Create a Sandbox with the specified Image, and mount a Volume”
   • [claimed-docs] “it's now possible to opt into a more performant Sandbox backend, which affords substantially higher creation rates and concurrency”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”

   proof ↗⚑ flag

Preview access

1. developerExpose a port from the sandbox on a public preview URL to reach services running inside

   weight 2 · round to Modal Sandboxes
   E2B—none0/10

   The evidence pack covers sandbox lifecycle (create, pause/resume, timeouts, files, commands, metrics, CLI) but contains no mention of exposing ports or generating public preview URLs to reach services running inside a sandbox. This is a fair capability question for a sandbox execution platform, so the axis applies, but nothing in the evidence supports it.

   ⚑ flag

   Modal Sandboxes~partialprobed3/10

   The docs pack includes a 'Networking and security' guide for sandboxes (modal-docs-10) which is the likely location for tunnel/port-exposure features, but no evidence text actually describes exposing a port to a public preview URL, tunnel URLs, or reachability details. missing for 10: explicit documentation or example showing sandbox.tunnels()/port forwarding, a public URL format, and independent confirmation of accessing services inside a sandbox via that URL.

   • [claimed-docs] “Networking and security”
   • [claimed-docs] “VM Sandboxes Beta”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”

   proof ↗⚑ flag

Snapshots

1. developerPause a running sandbox and resume it later with filesystem and memory state intact

   weight 2 · round to E2B
   E2B✓fullclaimed8/10

   E2B's docs explicitly document pausing a sandbox and resuming it later 'from the exact state it was in, including the filesystem and the memory, so running processes and loaded variables survive,' with paused sandboxes kept indefinitely, plus CLI support for snapshot/pause management (e2b sandbox snapshot, list --state paused). This directly matches the story's core claim. missing for 10: independent/hands-on third-party verification of state fidelity after resume, and more detail on any limitations (e.g., GPU/network state, long-running process edge cases) beyond first-party docs.

   • [claimed-docs] “Pause a sandbox and resume it later from the exact state it was in, including the filesystem and the memory, so running processes and loaded…”
   • [claimed-docs] “A paused sandbox is kept indefinitely with no automatic deletion.”
   • [claimed-docs] “Sandboxes can run continuously for up to 24 hours (Pro) or 1 hour (Base). For longer workloads, use pause and resume”
   • [claimed-docs] “You can manage sandbox snapshots with the `e2b sandbox snapshot` (alias `snap`) commands.”
   • [claimed-docs] “This will return running sandboxes, you can specify `--state` to get paused or both.”
   • [claimed-docs] “your agent's code runs in an isolated Firecracker microVM instead of your server, with a real Debian toolchain, pause/resume persistence, an…”

   proof ↗⚑ flag

   Modal Sandboxes~partialclaimed4/10

   Modal has a dedicated 'Snapshots' doc for sandboxes suggesting some pause/resume-with-state capability, but the evidence pack only gives the page title with no detail on what state (filesystem/memory) is preserved or how resume works. Missing for 10: documented mechanics of snapshot/resume (filesystem+memory fidelity), explicit confirmation memory state is preserved, independent/hands-on validation of the feature working as described.

   • [claimed-docs] “Snapshots”
   • [claimed-docs] “Filesystem Access”

   proof ↗⚑ flag

2. developerSnapshot a sandbox and later restore or fork new sandboxes from that snapshot

   weight 3 · round to E2B
   E2B✓fullclaimed8/10

   E2B explicitly supports pause/resume of a sandbox preserving filesystem and memory, and exposes CLI/SDK snapshot management (`e2b sandbox snapshot`), naming snapshot templates that can be reused to create new sandboxes, effectively enabling fork-like behavior from a saved state. missing for 10: explicit documentation/example of forking multiple independent sandboxes from a single snapshot simultaneously, and independent hands-on verification of restore/fork fidelity beyond vendor docs.

   • [claimed-docs] “Pause a sandbox and resume it later from the exact state it was in, including the filesystem and the memory, so running processes and loaded…”
   • [claimed-docs] “A paused sandbox is kept indefinitely with no automatic deletion.”
   • [claimed-docs] “Sandboxes can run continuously for up to 24 hours (Pro) or 1 hour (Base). For longer workloads, use pause and resume”
   • [claimed-docs] “You can manage sandbox snapshots with the `e2b sandbox snapshot` (alias `snap`) commands.”
   • [claimed-docs] “You can name the snapshot template with the `--name` flag. If a template with that name already exists, it is reused.”

   proof ↗⚑ flag

   Modal Sandboxes~partialclaimed5/10

   Modal explicitly documents a 'Sandbox Snapshots' feature (modal-docs-9), confirming snapshot capability exists, but the evidence pack gives no detail on the actual snapshot/restore/fork API mechanics, no confirmation that new independent sandboxes can be forked from a single snapshot, and no community/hands-on corroboration of this specific workflow. missing for 10: documented API details on restoring/forking multiple sandboxes from one snapshot, independent/hands-on confirmation of snapshot-restore-fork workflow, and clarity on any beta/experimental limitations.

   • [claimed-docs] “Snapshots”
   • [claimed-docs] “VM Sandboxes Beta”

   proof ↗⚑ flag