Modal Sandboxes vs Maritime

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to Maritime
   Modal Sandboxes✓fullprobed8/10

   Modal has a working llms.txt (HTTP 200) with a documentation summary, and docs pages are also served as markdown (.md suffix) confirming agent-oriented doc access. missing for 10: no independent confirmation that agents actually consume this llms.txt successfully in practice, and no dedicated agent-docs index beyond the generic llms.txt.

   • [probe] “PROBE llms.txt: HTTP 200 at https://modal.com/llms.txt # Modal Documentation > Modal is a platform for running AI workloads in the cloud wi…”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”

   proof ↗⚑ flag

   Maritime✓fullprobed9/10

   Maritime hosts a live llms.txt (confirmed by a 200 probe) plus a dedicated CLI llms.txt and an ai-agents docs page explicitly designed for agents (Claude Code, Cursor, etc.) to consume, authenticate, and act on — directly matching the story. Missing for 10: no independent third-party confirmation that agents actually navigate these docs successfully in practice.

   • [probe] “PROBE llms.txt: HTTP 200 at https://maritime.sh/llms.txt # Maritime > Maritime is a cloud platform for deploying, hosting, and managing AI …”
   • [claimed-docs] “Templates are cloneable agent packages; publishers can share them via link or the public library.”
   • [claimed-docs] “Pass --json on EVERY command. Success: one JSON value on stdout, stderr empty.”
   • [claimed-docs] “The Maritime CLI is built to be operated by other AI agents, not just humans: machine-readable output on every command, stable exit codes, a…”
   • [claimed-docs] “Paste one prompt into Claude, Cursor, or any assistant and it can install the CLI, authenticate, and deploy agents for you.”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round drawn
   Modal Sandboxes✓fullprobed8/10

   Modal Sandboxes is built for programmatic/headless use via Python, JS/Go SDKs and a CLI, with sandboxes created and controlled entirely through code (modal-docs-2, modal-docs-19, modal-docs-20, modal-gh-1). Community evidence confirms real-world CI/automation usage — a developer built GitHub Actions self-hosted runners on Modal Sandboxes explicitly for scalable, pay-per-use automation (modal-comm-1), and another wired gRPC control for an agent runtime (modal-comm-2). Missing for 10: no first-party CI/CD integration guide (e.g., official GitHub Actions/Jenkins docs) or dedicated 'headless CI' documentation section beyond community projects.

   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “Create a Sandbox with the specified Image, and mount a Volume”
   • [github] “The JS and Go SDKs allow you to use Modal Sandboxes, invoke deployed Modal Functions, and interact with some Modal platform resources.”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”
   • [claimed-docs] “Run a command or interactive shell inside a Modal container.”
   • [probe] “official CLI documented at https://modal.com/docs/reference/cli”

   proof ↗⚑ flag

   Maritime✓fullprobed8/10

   Maritime provides a CLI with --json flag on every command, machine-readable output, stable exit codes, and env-var (non-interactive) authentication via MARITIME_TOKEN, explicitly designed for automation/CI use and control by other AI agents rather than humans, plus a RESTful API and idempotent provisioning for scripted workflows. This directly supports headless/CI operation across CLI and SDK/API paths. Missing for 10: no explicit CI pipeline example (e.g., GitHub Actions/GitLab CI snippet) or independent third-party confirmation of CI usage.

   • [claimed-docs] “Pass --json on EVERY command. Success: one JSON value on stdout, stderr empty.”
   • [claimed-docs] “The Maritime CLI is built to be operated by other AI agents, not just humans: machine-readable output on every command, stable exit codes, a…”
   • [claimed-docs] “Authenticate with an env var, never interactively. Mint a long-lived key (`mk_…`) with `maritime keys create` and export it as `MARITIME_TOK…”
   • [claimed-docs] “Mint a long-lived key (`mk_…`) with `maritime keys create` and export it as `MARITIME_TOKEN`. Keys never expire; browser-login tokens do.”
   • [claimed-docs] “RESTful API for managing agents programmatically.”
   • [claimed-docs] “provision is idempotent on it: it returns the existing agent if there is one, else creates a new one. Safe to call on every sign-in.”
   • [probe] “official CLI documented at https://maritime.sh/docs/cli”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round drawn
   Modal Sandboxes—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   Maritime—none0/10

   Maritime is a platform for deploying and hosting agents built with frameworks (CrewAI, LangGraph, etc.), but the evidence pack shows no mention of MCP server integration or a mechanism for agents to consume external MCP tool servers. Nothing describes plugging in MCP servers to extend tool use.

   ⚑ flag

4. ai-native userConnect an agent via an official MCP server

   weight 3 · round drawn
   Modal Sandboxes—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   Maritime—none0/10

   Maritime is a platform for deploying/hosting agents (not itself an agent), so an official MCP server is a fair axis to expect, but no evidence pack item mentions MCP, an MCP server, or MCP-compatible endpoints — only REST API, CLI, SDKs, and webhooks are documented.

   • [claimed-docs] “RESTful API for managing agents programmatically.”
   • [claimed-docs] “The Maritime CLI lets you deploy and manage agents from your terminal.”
   • [claimed-docs] “Mint a personal API key with `maritime keys create` and send it as a Bearer token, the same token the CLI itself uses.”

   proof ↗⚑ flag

5. ai-native userUse an official CLI

   weight 2 · round to Maritime
   Modal Sandboxes✓fullprobed8/10

   Modal ships an official CLI (documented at modal.com/docs/reference/cli) supporting sandbox-related operations like running commands/shells and managing/connecting to containers, confirmed by probe access to the CLI reference page. This CLI is core to Modal's agentic/AI-native workflows (creating and interacting with sandboxes for untrusted/agent code). missing for 10: independent hands-on user reports specifically praising/using the CLI (vs. SDK), and more detail on CLI coverage of full sandbox lifecycle (snapshots, networking) beyond the two documented subcommands.

   • [claimed-docs] “Run a command or interactive shell inside a Modal container.”
   • [claimed-docs] “Manage and connect to running containers.”
   • [probe] “official CLI documented at https://modal.com/docs/reference/cli”
   • [claimed-docs] “Running commands”

   proof ↗⚑ flag

   Maritime✓fullprobed9/10

   Maritime ships an official npm-installable CLI explicitly designed for AI-native use: machine-readable/--json output on every command, stable exit codes, a self-describing contract, and env-var (non-interactive) authentication via MARITIME_TOKEN, all documented at /docs/cli and /docs/ai-agents. It also supports agent-driven install/auth/deploy flows via prompted assistants. Missing for 10: independent/hands-on verification beyond vendor docs.

   • [claimed-docs] “The Maritime CLI lets you deploy and manage agents from your terminal.”
   • [claimed-docs] “npm install -g maritime-cli”
   • [claimed-docs] “The Maritime CLI is built to be operated by other AI agents, not just humans: machine-readable output on every command, stable exit codes, a…”
   • [claimed-docs] “Authenticate with an env var, never interactively. Mint a long-lived key (`mk_…`) with `maritime keys create` and export it as `MARITIME_TOK…”
   • [claimed-docs] “Pass --json on EVERY command. Success: one JSON value on stdout, stderr empty.”
   • [claimed-docs] “Paste one prompt into Claude, Cursor, or any assistant and it can install the CLI, authenticate, and deploy agents for you.”
   • [probe] “official CLI documented at https://maritime.sh/docs/cli”

   proof ↗⚑ flag

6. ai-native userDrive the product through a documented public API

   weight 3 · round drawn
   Modal Sandboxes✓fullprobed8/10

   Modal exposes documented Python, JS, and Go SDKs plus a CLI for programmatically creating, controlling, and interacting with Sandboxes (exec, filesystem, snapshots, networking, logs), and community developers confirm driving it via gRPC/SDK for agent runtimes. missing for 10: no public OpenAPI/REST spec (probe found only 404s), so the 'documented public API' is SDK/CLI-based rather than a formal API contract, and there's no independent audit of API stability/versioning.

   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “Create a Sandbox with the specified Image, and mount a Volume”
   • [claimed-docs] “const p = await sb.exec(["cat", "/mnt/volume/message.txt"]);”
   • [github] “The JS and Go SDKs allow you to use Modal Sandboxes, invoke deployed Modal Functions, and interact with some Modal platform resources.”
   • [claimed-docs] “Run a command or interactive shell inside a Modal container.”
   • [claimed-docs] “Manage and connect to running containers.”
   • [claimed-docs] “We've added a modal.Sandbox.logs API, allowing you to fetch() logs from a specific date/time range or to tail() the most recent logs.”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://modal.com/openapi.json, https://modal.com/swagger.json, https://modal.com/api/openapi.json, …”

   proof ↗⚑ flag

   Maritime✓fullprobed8/10

   Maritime documents a full public API (REST endpoints, TypeScript/Python SDKs, idempotent provisioning, webhooks, scoped bearer-token auth) plus a CLI explicitly built for AI agents with machine-readable JSON output and stable exit codes, directly enabling AI-native driving of the platform. missing for 10: a discoverable machine-readable spec (openapi.json/swagger.json returned 404) and independent third-party corroboration of the API's stability.

   • [claimed-docs] “RESTful API for managing agents programmatically.”
   • [claimed-docs] “Mint a personal API key with `maritime keys create` and send it as a Bearer token, the same token the CLI itself uses.”
   • [claimed-docs] “Tag each agent with your own id via `externalId`. `provision` is **idempotent** on it: it returns the existing agent if there is one, else c…”
   • [claimed-docs] “provision is idempotent on it: it returns the existing agent if there is one, else creates a new one. Safe to call on every sign-in.”
   • [claimed-docs] “Get notified when a customer's agent changes state, instead of polling. Subscribe a URL, and Maritime POSTs a signed JSON body on every life…”
   • [claimed-docs] “The Maritime CLI is built to be operated by other AI agents, not just humans: machine-readable output on every command, stable exit codes, a…”
   • [claimed-docs] “Authenticate with an env var, never interactively. Mint a long-lived key (`mk_…`) with `maritime keys create` and export it as `MARITIME_TOK…”
   • [claimed-docs] “Pass --json on EVERY command. Success: one JSON value on stdout, stderr empty.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://maritime.sh/openapi.json, https://maritime.sh/swagger.json, https://maritime.sh/api/openapi.…”

   proof ↗⚑ flag

7. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round to Maritime
   Modal Sandboxes—none0/10

   The evidence describes secure sandboxed execution environments and general platform features, but there is no concrete evidence of a mechanism for issuing scoped/least-privilege API credentials specifically to agents—only a vague marketing line about 'credentials already in place' (modal-docs-3) with no documentation of credential scoping, token permissions, or per-agent access control. Missing for 10: any documented API/credential-scoping mechanism, permission granularity docs, or examples of issuing least-privilege tokens to agents.

   • [claimed-docs] “Autonomous agents with the right tools, context, and credentials already in place — running securely in a full, isolated dev environment.”

   proof ↗⚑ flag

   Maritime✓fullclaimed8/10

   Maritime documents scoped API keys explicitly: every key carries scopes, users should hand subsystems the narrowest key they need, and a dedicated example shows a Computers-only key being refused on all other endpoints — a concrete least-privilege mechanism. This pairs with CLI/env-based key minting (mk_ tokens) suited to agent automation. Missing for 10: an exhaustive scope catalog/reference and independent (non-vendor) verification of scope enforcement in practice.

   • [claimed-docs] “Hand a subsystem the narrowest key it needs. A leaked key can then only do what its scope allows.”
   • [claimed-docs] “To hand out a key that reaches computers and nothing else, create a **Computers key** in the Computers section of Settings, API keys”
   • [claimed-docs] “Every key carries scopes. Hand a subsystem the _narrowest_ key it needs. A leaked key can then only do what its scope allows.”
   • [claimed-docs] “a key whose only scope is `computers` is refused on every other Maritime endpoint”
   • [claimed-docs] “Hand a subsystem the _narrowest_ key it needs. A leaked key can then only do what its scope allows.”
   • [claimed-docs] “Authenticate with an env var, never interactively. Mint a long-lived key (`mk_…`) with `maritime keys create` and export it as `MARITIME_TOK…”
   • [claimed-docs] “Mint a long-lived key (`mk_…`) with `maritime keys create` and export it as `MARITIME_TOKEN`. Keys never expire; browser-login tokens do.”

   proof ↗⚑ flag

8. ai-native userBuild against official SDKs

   weight 2 · round drawn
   Modal Sandboxes✓fullcommunity8/10

   Modal ships official Python SDK plus JS/Go SDKs for Sandboxes with documented APIs (exec, filesystem, snapshots, networking, logs) and community developers confirm building against them (GitHub Actions runners, gRPC-based JS agent runtime). missing for 10: independent third-party SDK reference/benchmarks beyond Modal's own docs and one community mention, and no evidence of broader language SDK coverage (e.g. Rust, Java).

   • [github] “The JS and Go SDKs allow you to use Modal Sandboxes, invoke deployed Modal Functions, and interact with some Modal platform resources.”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “Create a Sandbox with the specified Image, and mount a Volume”
   • [claimed-docs] “const p = await sb.exec(["cat", "/mnt/volume/message.txt"]);”
   • [claimed-docs] “We've added a modal.Sandbox.logs API, allowing you to fetch() logs from a specific date/time range or to tail() the most recent logs.”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”

   proof ↗⚑ flag

   Maritime✓fullprobed8/10

   Maritime documents official TypeScript and Python SDKs with idempotent agent provisioning, scoped authentication, webhooks, messaging, and a Files API, plus an npm-installable CLI and RESTful API, all clearly aimed at building AI-native integrations. missing for 10: no public OpenAPI/swagger spec found (404s) and no independent/hands-on corroboration beyond first-party docs.

   • [claimed-docs] “Tag each agent with your own id via `externalId`. `provision` is **idempotent** on it: it returns the existing agent if there is one, else c…”
   • [claimed-docs] “Mint a personal API key with `maritime keys create` and send it as a Bearer token, the same token the CLI itself uses.”
   • [claimed-docs] “Give every one of your users their own agent with the TypeScript or Python SDK.”
   • [claimed-docs] “Send a message and wait for the reply. Sleeping agents wake automatically.”
   • [claimed-docs] “RESTful API for managing agents programmatically.”
   • [claimed-docs] “The Maritime Files API: eight endpoints and both SDKs to browse, upload, download, and edit an agent's disk over HTTP, even while the agent …”
   • [claimed-docs] “provision is idempotent on it: it returns the existing agent if there is one, else creates a new one. Safe to call on every sign-in.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://maritime.sh/openapi.json, https://maritime.sh/swagger.json, https://maritime.sh/api/openapi.…”

   proof ↗⚑ flag

9. ai-native userSubscribe to events via webhooks

   weight 2 · round to Maritime
   Modal Sandboxes—none0/10

   No evidence anywhere in the pack of a webhook subscription mechanism or event-driven notification system for Modal Sandboxes; the docs focus on programmatic spin-up, execution, logging, and filesystem access, not outbound event/webhook subscriptions.

   ⚑ flag

   Maritime✓fullclaimed7/10

   Docs explicitly describe subscribing a URL to receive signed JSON POSTs on every agent lifecycle event, replacing polling, which directly matches the story. missing for 10: independent/hands-on confirmation of webhook delivery reliability, payload schema details, and signature verification specifics.

   • [claimed-docs] “Get notified when a customer's agent changes state, instead of polling. Subscribe a URL, and Maritime POSTs a signed JSON body on every life…”
   • [claimed-docs] “Hand a subsystem the narrowest key it needs. A leaked key can then only do what its scope allows.”

   proof ↗⚑ flag

Agentic features

1. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Maritime
   Modal Sandboxes~partialcommunity6/10

   Modal Sandboxes explicitly supports running autonomous agent code in isolated environments at scale (docs-3, docs-4, docs-22), and community evidence shows real-world use for background automation (self-hosted CI runners on modal-comm-1, agent runtimes on modal-comm-2). However, the evidence pack lacks explicit documentation of scheduling/cron triggers or persistent background job orchestration—sandboxes are framed as on-demand execution environments rather than a native 'set it and forget it' automation scheduler. Missing for 10: native scheduling/cron trigger docs, first-party persistent orchestration for long-running background automations.

   • [claimed-docs] “Autonomous agents with the right tools, context, and credentials already in place — running securely in a full, isolated dev environment.”
   • [claimed-docs] “Spin up hundreds of thousands of concurrent rollout environments in seconds.”
   • [claimed-docs] “We execute the agent's code in a Modal Sandbox, which allows us to run arbitrary code in a safe environment.”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”

   proof ↗⚑ flag

   Maritime✓fullclaimed8/10

   Maritime provides cron triggers with standard five-field expressions that wake agents on schedule and deliver prompts, plus agents can self-schedule their own wake triggers (e.g., 'send me a digest every weekday at 9'), and agents run persistently on always-on micro-VMs with sleep/wake economics, enabling true background autonomous automation. Missing for 10: independent/hands-on verification of trigger reliability and no mention of monitoring/alerting on automation failures.

   • [claimed-docs] “A cron trigger takes a standard five-field expression, wakes the agent on schedule, and delivers its `prompt` as a chat message on every fir…”
   • [claimed-docs] “Ask the agent (OpenClaw-family and Hermes templates): tell it "send me a digest every weekday at 9" and the platform syncs the schedule it s…”
   • [claimed-docs] “tell it "send me a digest every weekday at 9" and the platform syncs the schedule it sets for itself into real wake triggers”
   • [claimed-docs] “Always-on is a flat add-on for the rare agent that must react in milliseconds around the clock.”
   • [claimed-docs] “Send a message and wait for the reply. Sleeping agents wake automatically.”
   • [claimed-docs] “Maritime gives it what a laptop session can't: its own always-on micro-VM in the cloud, persistent storage, sleep/wake economics, and every …”

   proof ↗⚑ flag

2. ai-native userOperate the product with natural-language commands

   weight 2 · round to Maritime
   Modal Sandboxes—none0/10

   Modal Sandboxes is controlled exclusively via SDK (Python/JS/Go) calls and a traditional CLI (modal run, modal shell, etc.); there is no evidence of a natural-language command interface or chat-driven control layer for operating sandboxes. Evidence shows programmatic/API-first design (modal-docs-2, modal-docs-15, modal-docs-19) rather than any NL-command capability.

   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Run a command or interactive shell inside a Modal container.”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [probe] “official CLI documented at https://modal.com/docs/reference/cli”

   proof ↗⚑ flag

   Maritime~partialclaimed6/10

   Maritime documents natural-language operation at the agent level (chat with agents, cron-triggered prompts set via natural language like 'send me a digest every weekday at 9') and CLI/API designed to be driven by AI agents via prompts (paste a prompt into Claude/Cursor to install and operate). However, the core Maritime platform itself (dashboard, agent management) is not shown to be operable purely via natural-language commands by the end user — the NL interface is mediated through the CLI/API being agent-friendly rather than a native NL command surface for platform operations. missing for 10: evidence of a native NL interface for platform-level operations (not just agent chat/prompts), independent/hands-on confirmation of this workflow succeeding.

   • [claimed-docs] “Ask the agent (OpenClaw-family and Hermes templates): tell it "send me a digest every weekday at 9" and the platform syncs the schedule it s…”
   • [claimed-docs] “Paste one prompt into Claude, Cursor, or any assistant and it can install the CLI, authenticate, and deploy agents for you.”
   • [claimed-docs] “tell it "send me a digest every weekday at 9" and the platform syncs the schedule it sets for itself into real wake triggers”
   • [claimed-docs] “The Maritime CLI is built to be operated by other AI agents, not just humans: machine-readable output on every command, stable exit codes, a…”
   • [claimed-docs] “point Claude Code or your Maritime agent at a provider's runbook page and it can inventory the old provider with read-only credentials, recr…”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   Modal Sandboxes—none0/10

   Evidence shows standard docs pages, CLI reference, and SDK code snippets, but no interactive API reference or runnable-example playground; the openapi probe returned 404s and no docs mention an in-browser code runner or live API explorer.

   • [probe] “PROBE openapi: all candidate paths 404 (https://modal.com/openapi.json, https://modal.com/swagger.json, https://modal.com/api/openapi.json, …”
   • [claimed-docs] “Run a command or interactive shell inside a Modal container.”
   • [claimed-docs] “const p = await sb.exec(["cat", "/mnt/volume/message.txt"]);”

   proof ↗⚑ flag

   Maritime—none0/10

   Docs describe a RESTful API and CLI but there's no evidence of an interactive API reference (e.g., Swagger/OpenAPI explorer) with runnable examples; probes explicitly show no openapi.json/swagger.json spec found at any candidate path. missing for 10: interactive API console/playground, OpenAPI spec, runnable code examples in docs.

   • [claimed-docs] “RESTful API for managing agents programmatically.”
   • [claimed-docs] “Mint a personal API key with `maritime keys create` and send it as a Bearer token, the same token the CLI itself uses.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://maritime.sh/openapi.json, https://maritime.sh/swagger.json, https://maritime.sh/api/openapi.…”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   Modal Sandboxes—none0/10

   The probe explicitly found all candidate OpenAPI/swagger endpoints return 404, and no other evidence pack item mentions a machine-readable API spec being published; Modal's public interface is SDK/CLI-based rather than a REST API with a spec.

   • [probe] “PROBE openapi: all candidate paths 404 (https://modal.com/openapi.json, https://modal.com/swagger.json, https://modal.com/api/openapi.json, …”

   proof ↗⚑ flag

   Maritime—none0/10

   Maritime documents a RESTful API and CLI with --json output, but there is no evidence of a downloadable machine-readable spec (OpenAPI/Swagger); active probes for openapi.json, swagger.json, and similar paths all returned 404.

   • [probe] “PROBE openapi: all candidate paths 404 (https://maritime.sh/openapi.json, https://maritime.sh/swagger.json, https://maritime.sh/api/openapi.…”
   • [claimed-docs] “RESTful API for managing agents programmatically.”
   • [claimed-docs] “Mint a personal API key with `maritime keys create` and send it as a Bearer token, the same token the CLI itself uses.”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round to Modal Sandboxes
   Modal Sandboxes✓fullprobed7/10

   Modal Sandboxes are explicitly designed as ephemeral, isolated containers for running untrusted or agent code, with filesystem, networking/security, and snapshot controls documented (modal-docs-2, modal-docs-8, modal-docs-9, modal-docs-10, modal-probe-2), and community examples show real usage isolating workloads from production infra (modal-comm-1, modal-comm-2). This satisfies the 'test without touching production' need since each sandbox is a fresh, isolated environment rather than shared prod state. Missing for 10: explicit documentation or case study framing isolation specifically in terms of 'production data' separation (e.g., no prod DB/staging parity guidance), and no independent audit confirming true isolation guarantees.

   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Filesystem Access”
   • [claimed-docs] “Snapshots”
   • [claimed-docs] “Networking and security”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “We execute the agent's code in a Modal Sandbox, which allows us to run arbitrary code in a safe environment.”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”

   proof ↗⚑ flag

   Maritime—none0/10

   Maritime's docs cover deployment, keys/scopes, templates, triggers, and agent lifecycle, but nothing describes a sandbox/staging environment isolated from production data for testing purposes. Missing for 10: any mention of a sandbox/test mode, staging environment, or data isolation guarantees between test and production agents.

   ⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Modal Sandboxes—none0/10

   Evidence shows release notes and beta/alpha labels for some sandbox features, but no documentation of API versioning scheme or a deprecation policy; OpenAPI spec probes also 404. missing for 10: explicit versioning scheme documentation, deprecation policy page, changelog with deprecation notices, stability guarantees for GA vs beta APIs.

   • [claimed-docs] “VM Sandboxes Beta”
   • [claimed-docs] “Sidecars Alpha”
   • [claimed-docs] “We've added a modal.Sandbox.logs API, allowing you to fetch() logs from a specific date/time range or to tail() the most recent logs.”
   • [claimed-docs] “it's now possible to opt into a more performant Sandbox backend, which affords substantially higher creation rates and concurrency”
   • [probe] “PROBE openapi: all candidate paths 404 (https://modal.com/openapi.json, https://modal.com/swagger.json, https://modal.com/api/openapi.json, …”

   proof ↗⚑ flag

   Maritime—none0/10

   No evidence of API versioning scheme (e.g., v1/v2 paths) or any documented deprecation policy; OpenAPI spec probes all 404, and no docs mention version support windows or breaking-change policy.

   • [probe] “PROBE openapi: all candidate paths 404 (https://maritime.sh/openapi.json, https://maritime.sh/swagger.json, https://maritime.sh/api/openapi.…”
   • [claimed-docs] “RESTful API for managing agents programmatically.”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Modal Sandboxes
   Modal Sandboxes~partialcommunity6/10

   Modal supports spinning up hundreds of thousands of concurrent sandboxes/rollout environments programmatically, which is strong evidence of bulk/parallel scaling (modal-docs-4, modal-docs-14, modal-docs-5), and this is corroborated by community builds using it for scaled runner fleets (modal-comm-1). However, this is infrastructure-level bulk provisioning, not a documented 'bulk operations' API/pattern for operating across many existing items (e.g., batch file ops, batch snapshot management, or a dedicated batch API) — missing for 10: a first-party batch/bulk-operation API or workflow example (vs just scaling raw sandbox count), independent benchmarks confirming claimed concurrency at scale, and documentation of bulk management operations (e.g., bulk terminate/list/update) rather than just creation.

   • [claimed-docs] “Spin up hundreds of thousands of concurrent rollout environments in seconds.”
   • [claimed-docs] “Spin up hundreds of thousands of concurrent rollout environments in seconds. Fast enough to keep your GPU inference resources saturated acro…”
   • [claimed-docs] “H100s, A100s, A10Gs available on demand. Attach to any sandbox, scale to thousands of concurrent runs, pay by the second with no reserved ca…”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [claimed-docs] “Integrated logging and full visibility into every function, sandbox, and container.”

   proof ↗⚑ flag

   Maritime~partialclaimed4/10

   There is explicit support for bulk import of config/env vars (maritime-docs-11) and a scriptable CLI/API with JSON output (maritime-docs-9, maritime-docs-36) that an AI-native user could loop over to perform bulk actions, but there is no documented batch/bulk endpoint for operating on many agents or resources at once. missing for 10: a dedicated bulk-operation API/CLI command (e.g. bulk create/delete/update agents), evidence of scaling to many items reliably, and independent confirmation of bulk workflows in practice.

   • [claimed-docs] “Bulk import from a .env file (or stdin), then hot-reload”
   • [claimed-docs] “Pass --json on EVERY command. Success: one JSON value on stdout, stderr empty.”
   • [claimed-docs] “RESTful API for managing agents programmatically.”
   • [claimed-docs] “The Maritime CLI is built to be operated by other AI agents, not just humans: machine-readable output on every command, stable exit codes, a…”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to Maritime
   Modal Sandboxes—none0/10

   The evidence describes Modal Sandboxes as a programmatic, on-demand execution environment for untrusted/agent code, but nothing in the pack shows a rules/trigger engine that fires actions automatically on external events (e.g., webhooks, event-driven policies). Absent such evidence, this applicable automation-depth axis is unmet.

   ⚑ flag

   Maritime~partialclaimed6/10

   Maritime supports triggers (cron with five-field expressions, natural-language schedule setting, webhooks for lifecycle events) that wake agents and fire actions automatically, which covers rule-based automation on events. However, this is event/schedule-based triggering rather than a general-purpose conditional rules engine (e.g., no evidence of arbitrary if/then logic across multiple event types or complex conditions). missing for 10: evidence of a general conditional rules/policy engine beyond cron and webhook triggers, and independent/hands-on confirmation of trigger reliability.

   • [claimed-docs] “A cron trigger takes a standard five-field expression, wakes the agent on schedule, and delivers its `prompt` as a chat message on every fir…”
   • [claimed-docs] “Ask the agent (OpenClaw-family and Hermes templates): tell it "send me a digest every weekday at 9" and the platform syncs the schedule it s…”
   • [claimed-docs] “tell it "send me a digest every weekday at 9" and the platform syncs the schedule it sets for itself into real wake triggers”
   • [claimed-docs] “Get notified when a customer's agent changes state, instead of polling. Subscribe a URL, and Maritime POSTs a signed JSON body on every life…”
   • [claimed-docs] “Always-on is a flat add-on for the rare agent that must react in milliseconds around the clock.”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round to Maritime
   Modal Sandboxes—none0/10

   Evidence covers ephemeral sandbox creation, execution, networking, snapshots, and SDKs, but there is no mention of cron-like scheduling, recurring triggers, or persistent workflow orchestration for Modal Sandboxes. missing for 10: any scheduling/cron API, recurring job trigger mechanism, or workflow orchestration documentation tied to Sandboxes.

   • [claimed-docs] “VM Sandboxes Beta”
   • [claimed-docs] “Running commands”
   • [claimed-docs] “Snapshots”

   proof ↗⚑ flag

   Maritime✓fullclaimed8/10

   Maritime explicitly supports cron triggers with standard five-field expressions that wake agents and deliver prompts on schedule, plus a natural-language flow where an agent sets its own recurring schedule ('send me a digest every weekday at 9') that syncs into real wake triggers. This directly covers recurring job/workflow scheduling for AI-native users. Missing for 10: independent/hands-on verification beyond vendor docs, and no detail on managing/editing/cancelling existing scheduled triggers via CLI/API.

   • [claimed-docs] “A cron trigger takes a standard five-field expression, wakes the agent on schedule, and delivers its `prompt` as a chat message on every fir…”
   • [claimed-docs] “Ask the agent (OpenClaw-family and Hermes templates): tell it "send me a digest every weekday at 9" and the platform syncs the schedule it s…”
   • [claimed-docs] “tell it "send me a digest every weekday at 9" and the platform syncs the schedule it sets for itself into real wake triggers”
   • [claimed-docs] “Always-on is a flat add-on for the rare agent that must react in milliseconds around the clock.”

   proof ↗⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round drawn
   Modal Sandboxes—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   Maritime—none0/10

   No evidence of version history, review workflows, or rollback for automations/agent configs; docs cover deployment, triggers, builds, and CLI but nothing about versioning or reverting agent configuration/automation changes.

   ⚑ flag

Agent workloads

1. ai-native userRun coding agents like Claude Code or Codex inside the sandbox following the vendor's own recipe

   weight 2 · round to Maritime
   Modal Sandboxes—none0/10

   Modal's docs show general support for running 'agents' and 'untrusted code' in Sandboxes (modal-docs-3, modal-docs-22, modal-docs-19), but no evidence pack item mentions a vendor recipe, tutorial, or example specifically for running Claude Code or Codex inside a Sandbox.

   ⚑ flag

   Maritime✓fullclaimed8/10

   Maritime documents dedicated template recipes for running Claude Code and Codex inside its sandboxed micro-VMs (persistent storage, sleep/wake, metered LLM access by default, chat channels), directly matching the vendor's own recipe framing for these named coding agents. missing for 10: independent/hands-on verification beyond vendor docs, and no direct evidence of a step-by-step 'follow this exact recipe' walkthrough output for a live run.

   • [claimed-docs] “Maritime gives it what a laptop session can't: its own always-on micro-VM in the cloud, persistent storage, sleep/wake economics, and every …”
   • [claimed-docs] “No key is required: the agent uses Maritime's metered LLM access by default.”
   • [claimed-docs] “Every ready-made template is a real agent framework, pre-configured and one image pull from live. Pick one, name it, deploy.”
   • [claimed-docs] “point Claude Code or your Maritime agent at a provider's runbook page and it can inventory the old provider with read-only credentials, recr…”
   • [claimed-docs] “Paste one prompt into Claude, Cursor, or any assistant and it can install the CLI, authenticate, and deploy agents for you.”

   proof ↗⚑ flag

Workloads

1. developerRun a headless browser or full desktop environment inside the sandbox

   weight 2 · round to Maritime
   Modal Sandboxes—none0/10

   Evidence describes Modal Sandboxes as isolated containers/VMs for running arbitrary code, agents, filesystem access, networking, and command execution, but nowhere mentions headless browser or full desktop/GUI environment support (e.g., Xvfb, VNC, browser automation tooling). No docs or community reports reference browser or desktop environments running inside sandboxes.

   ⚑ flag

   Maritime✓fullclaimed7/10

   Maritime docs explicitly describe 'Computers': a persistent Linux desktop (XFCE, Chromium, LibreOffice) per user driven by a model, and the changelog confirms agents get a real Linux desktop with screenshot/mouse/keyboard control and live takeover — this covers full desktop environments including a browser (Chromium) inside the sandbox. missing for 10: no independent/hands-on corroboration of performance or headless-only browser mode, and no detail on resource limits or concurrency.

   • [claimed-docs] “A persistent Linux desktop (XFCE, Chromium, LibreOffice) for each of _your_ end users, driven by _your_ model.”
   • [claimed-docs] “Agents are getting a real Linux desktop they can drive: screenshot, mouse, keyboard, and a live view you can take over.”
   • [claimed-docs] “To hand out a key that reaches computers and nothing else, create a **Computers key** in the Computers section of Settings, API keys”

   proof ↗⚑ flag

2. developerAttach GPUs to sandboxed workloads

   weight 2 · round to Modal Sandboxes
   Modal Sandboxes✓fullcommunity8/10

   Modal docs explicitly state GPUs (H100s, A100s, A10Gs) can be attached to any sandbox on demand with per-second billing, and community evidence (HN runner project) confirms real-world GPU-backed sandbox usage. missing for 10: independent hands-on benchmark of GPU attach performance/reliability, and explicit code example showing GPU parameter in sandbox creation API.

   • [claimed-docs] “H100s, A100s, A10Gs available on demand. Attach to any sandbox, scale to thousands of concurrent runs, pay by the second with no reserved ca…”
   • [claimed-docs] “Attach to any sandbox, scale to thousands of concurrent runs, pay by the second with no reserved capacity.”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”

   proof ↗⚑ flag

   Maritime—none0/10

   No evidence anywhere in the pack mentions GPUs, GPU-accelerated instances, or hardware acceleration for sandboxed agent workloads; all compute descriptions reference micro-VMs, sleep/wake, and CPU-based containers only. Missing for 10: any mention of GPU attachment, GPU instance types, or hardware accelerator support.

   • [claimed-docs] “Maritime gives it what a laptop session can't: its own always-on micro-VM in the cloud, persistent storage, sleep/wake economics, and every …”
   • [claimed-docs] “Any Docker container becomes an agent: three endpoints get you sleep/wake, channels, and triggers.”

   proof ↗⚑ flag

Files

1. developerRead, write, upload, and download files in the sandbox filesystem via the SDK

   weight 2 · round to Modal Sandboxes
   Modal Sandboxes✓fullprobed8/10

   Modal explicitly documents a dedicated 'Filesystem Access' guide for sandboxes (modal-docs-8) alongside SDK examples showing file read/write via mounted volumes and exec commands (modal-docs-20, modal-docs-21), confirming read/write/upload/download-style operations through the SDK. missing for 10: no independent/hands-on community confirmation specifically of file upload/download methods (only exec/volume examples), and no explicit code snippet showing a dedicated upload/download API method distinct from volume mounting.

   • [claimed-docs] “Filesystem Access”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “Create a Sandbox with the specified Image, and mount a Volume”
   • [claimed-docs] “const p = await sb.exec(["cat", "/mnt/volume/message.txt"]);”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”

   proof ↗⚑ flag

   Maritime—none0/10

   No evidence pack item mentions SDK-based file read/write/upload/download operations against a sandbox filesystem; the docs cover agent provisioning, webhooks, auth scoping, triggers, computers (desktop), CLI, and migration but nothing about filesystem file I/O via SDK. Missing for full: no filesystem API/SDK method docs, no upload/download endpoint references, no code samples showing file operations.

   ⚑ flag

Runtimes

1. developerDefine custom sandbox templates or bring my own container image

   weight 2 · round to Modal Sandboxes
   Modal Sandboxes✓fullcommunity8/10

   Modal explicitly supports creating Sandboxes with custom Images ('Custom images, any dependency' and 'Create a Sandbox with the specified Image, and mount a Volume'), and community projects (GitHub Actions runners, MCP-JS) confirm developers configuring custom sandbox environments in practice. Missing for 10: explicit documentation of importing arbitrary Docker/OCI registry images or a dedicated 'bring your own container' guide beyond the Image API reference.

   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Create a Sandbox with the specified Image, and mount a Volume”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”

   proof ↗⚑ flag

   Maritime~partialclaimed6/10

   Docs confirm bring-your-own-image support via a documented three-endpoint contract that works with any Docker image (CrewAI, LangGraph, AutoGen, custom scripts), plus a build log and template system for packaging custom agents. However, there's no detailed spec of the contract itself, no example custom Dockerfile, and no independent/hands-on confirmation of successfully running a fully custom image end-to-end. missing for 10: full technical spec of the three-endpoint contract, a worked example of a custom image deployment, independent verification of custom image support beyond docs claims.

   • [claimed-docs] “CrewAI, LangGraph, AutoGen, your own Python script: any Docker image that speaks a three-endpoint contract gets the whole platform.”
   • [claimed-docs] “Templates are cloneable agent packages; publishers can share them via link or the public library.”
   • [claimed-docs] “The agent page streams the build log live under **Overview → Builds**.”
   • [claimed-docs] “Every ready-made template is a real agent framework, pre-configured and one image pull from live. Pick one, name it, deploy.”

   proof ↗⚑ flag

2. developerExecute code in multiple language runtimes (Python, JavaScript, and more) and get rich results back

   weight 2 · round to Modal Sandboxes
   Modal Sandboxes~partialclaimed6/10

   Modal Sandboxes let you run arbitrary commands/code in isolated containers with custom images (any dependency, any language stack), including SDK examples for JS/Go exec and Python examples of running agent code (modal-docs-2, modal-docs-13, modal-docs-19-21). However, evidence never shows explicit multi-language runtime presets (Python vs JS vs others) or structured/rich result payloads (e.g., images, dataframes, mime-typed outputs) — outputs shown are plain stdout/exec results. missing for 10: explicit built-in multi-language runtime templates, documented rich/structured result formats beyond raw stdout.

   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Running commands”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “Create a Sandbox with the specified Image, and mount a Volume”
   • [claimed-docs] “const p = await sb.exec(["cat", "/mnt/volume/message.txt"]);”
   • [claimed-docs] “We execute the agent's code in a Modal Sandbox, which allows us to run arbitrary code in a safe environment.”

   proof ↗⚑ flag

   Maritime—none0/10

   Maritime's evidence describes deploying persistent Docker-based agents (any Docker image with sleep/wake/channels/triggers), a Files API, and a Linux desktop 'Computers' feature, but nothing describes a code-execution/sandbox API that runs snippets in multiple language runtimes (Python, JS, etc.) and returns structured/rich results (e.g., stdout, images, dataframes) the way a code-interpreter tool would.

   • [claimed-docs] “Any Docker container becomes an agent: three endpoints get you sleep/wake, channels, and triggers.”
   • [claimed-docs] “A persistent Linux desktop (XFCE, Chromium, LibreOffice) for each of _your_ end users, driven by _your_ model.”
   • [claimed-docs] “The Maritime Files API: eight endpoints and both SDKs to browse, upload, download, and edit an agent's disk over HTTP, even while the agent …”
   • [claimed-docs] “CrewAI, LangGraph, AutoGen, your own Python script: any Docker image that speaks a three-endpoint contract gets the whole platform.”

   proof ↗⚑ flag

Untrusted code

1. developerExecute untrusted, AI-generated code without risking my own infrastructure

   weight 3 · round to Modal Sandboxes
   Modal Sandboxes✓fullprobed8/10

   Modal Sandboxes is explicitly built for running untrusted/AI-generated code in isolated, ephemeral containers with networking/filesystem controls, resource limits, and logging, and community examples show real-world use (self-hosted CI runners, agent runtimes) confirming the isolation model works in practice. Missing for 10: independent security audit or adversarial escape-testing evidence, and clearer documentation of resource/network isolation guarantees beyond marketing copy.

   • [claimed-docs] “Programmatically scale secure, ephemeral environments for running untrusted code.”
   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Networking and security”
   • [claimed-docs] “VM Sandboxes Beta”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “We execute the agent's code in a Modal Sandbox, which allows us to run arbitrary code in a safe environment.”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”

   proof ↗⚑ flag

   Maritime~partialprobed5/10

   Maritime hosts agents as Docker images on 'serverless infrastructure' (maritime-docs-1, maritime-probe-1) and provides persistent isolated Linux desktops per end user (maritime-docs-5), which implies agent code runs off the developer's own machines/infra. However, there is no explicit documentation of sandboxing guarantees, resource/network isolation, or security boundaries specifically for running untrusted or AI-generated code safely. Missing for 10: explicit sandbox/isolation security model, resource-limit guarantees, and any independent verification that untrusted code cannot escape or affect other tenants/infra.

   • [claimed-docs] “CrewAI, LangGraph, AutoGen, your own Python script: any Docker image that speaks a three-endpoint contract gets the whole platform.”
   • [claimed-docs] “A persistent Linux desktop (XFCE, Chromium, LibreOffice) for each of _your_ end users, driven by _your_ model.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://maritime.sh/llms.txt # Maritime > Maritime is a cloud platform for deploying, hosting, and managing AI …”

   proof ↗⚑ flag

2. developerRun arbitrary shell commands and install packages inside the sandbox

   weight 2 · round to Modal Sandboxes
   Modal Sandboxes✓fullprobed9/10

   Modal Sandboxes documentation explicitly covers running arbitrary shell commands (sb.exec, CLI 'run a command or interactive shell inside a Modal container'), custom images with any dependency installation, filesystem access, and community reports (HN self-hosted runners, JS agent runtime) corroborate real-world use for arbitrary command execution and untrusted code. Missing for 10: no independent benchmark of package installation limits or explicit documented apt/pip install examples beyond generic 'any dependency' claim.

   • [claimed-docs] “Running commands”
   • [claimed-docs] “Run a command or interactive shell inside a Modal container.”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “const p = await sb.exec(["cat", "/mnt/volume/message.txt"]);”
   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”

   proof ↗⚑ flag

   Maritime—none0/10

   Maritime's docs describe deploying agents via Docker images, templates, and a custom framework contract, but there is no evidence of an interactive shell/terminal access or arbitrary package installation inside a running sandbox for a developer to use directly. The 'Computers' feature offers a persistent desktop for end-users, not a shell for arbitrary command execution by the developer.

   ⚑ flag

Agent lifecycle

1. ai-native userMy agent can provision its own sandbox, execute code, read the results, and tear it down — end to end without a human

   weight 3 · round to Modal Sandboxes
   Modal Sandboxes✓fullprobed8/10

   Modal Sandboxes are explicitly designed for programmatic, API-driven provisioning, code execution, log/result retrieval, and teardown, with SDK examples (Python/JS/Go) showing sandbox creation, exec, file access, and logs, plus a documented agent-code-execution example. missing for 10: independent hands-on report showing a fully autonomous agent loop (provision->execute->read->teardown) with no human intervention, and clearer documentation of teardown/termination API specifics.

   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Autonomous agents with the right tools, context, and credentials already in place — running securely in a full, isolated dev environment.”
   • [claimed-docs] “Running commands”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “const p = await sb.exec(["cat", "/mnt/volume/message.txt"]);”
   • [claimed-docs] “We execute the agent's code in a Modal Sandbox, which allows us to run arbitrary code in a safe environment.”
   • [claimed-docs] “We've added a modal.Sandbox.logs API, allowing you to fetch() logs from a specific date/time range or to tail() the most recent logs.”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”

   proof ↗⚑ flag

   Maritime~partialclaimed6/10

   Maritime's CLI and SDK let an agent provision (idempotent) itself, run in a sandboxed micro-VM/container, and be managed via authenticated API/CLI calls with JSON output designed for machine consumption, and agents can be torn down/put to sleep automatically when idle. However, the evidence is docs-only with no hands-on trace of an agent actually reading back code-execution results or explicit 'delete/destroy' teardown call, and no independent verification of the full autonomous loop. missing for 10: concrete evidence of an agent reading execution output programmatically, an explicit teardown/delete API example, and independent/hands-on confirmation of the full provision→execute→read→teardown loop.

   • [claimed-docs] “Tag each agent with your own id via `externalId`. `provision` is **idempotent** on it: it returns the existing agent if there is one, else c…”
   • [claimed-docs] “Pass --json on EVERY command. Success: one JSON value on stdout, stderr empty.”
   • [claimed-docs] “The Maritime CLI is built to be operated by other AI agents, not just humans: machine-readable output on every command, stable exit codes, a…”
   • [claimed-docs] “Authenticate with an env var, never interactively. Mint a long-lived key (`mk_…`) with `maritime keys create` and export it as `MARITIME_TOK…”
   • [claimed-docs] “Send a message and wait for the reply. Sleeping agents wake automatically.”
   • [claimed-docs] “provision is idempotent on it: it returns the existing agent if there is one, else creates a new one. Safe to call on every sign-in.”
   • [claimed-docs] “Any Docker container becomes an agent: three endpoints get you sleep/wake, channels, and triggers.”

   proof ↗⚑ flag

Isolation

1. platform-engineerRely on a documented hard isolation boundary (microVM or equivalent) between sandboxes and my systems

   weight 3 · round to Modal Sandboxes
   Modal Sandboxes~partialprobed5/10

   Modal repeatedly describes Sandboxes as 'secure,' 'isolated' containers for untrusted code, and references a 'VM Sandboxes Beta' page implying microVM-based isolation, but no evidence pack item actually documents the underlying isolation technology (e.g., Firecracker/gVisor specifics, threat model, or security boundary guarantees) that a platform engineer would need to rely on. Missing for 10: explicit technical documentation of the isolation mechanism (hypervisor/kernel boundary), any third-party security audit or independent verification of the boundary's robustness.

   • [claimed-docs] “VM Sandboxes Beta”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “We execute the agent's code in a Modal Sandbox, which allows us to run arbitrary code in a safe environment.”

   proof ↗⚑ flag

   Maritime—none0/10

   No evidence pack item documents a specific isolation mechanism (microVM, gVisor, container hardening, VM boundary) between sandboxes and host systems; docs mention Docker images, serverless infra, and scoped API keys but not the underlying execution isolation boundary. Missing for 10: any explicit statement of sandbox isolation technology, security model documentation, or third-party audit/confirmation of hard isolation.

   ⚑ flag

2. ai-native userGive an agent a sandbox where host secrets and credentials are unreachable by the code it runs

   weight 2 · round to Maritime
   Modal Sandboxes~partialprobed5/10

   Modal Sandboxes are documented as isolated, ephemeral containers for running untrusted/agent code with dedicated networking-and-security guides, supporting the general isolation story (modal-docs-1, modal-docs-2, modal-docs-10, modal-probe-2, modal-docs-19, modal-docs-22). However, no evidence explicitly states that host secrets/credentials are unreachable by default or describes the secrets-injection model that would confirm this — one doc even implies credentials are provisioned into the agent environment (modal-docs-3), which cuts against a strong 'unreachable unless granted' guarantee. Missing for 10: explicit secrets-isolation documentation, details of the opt-in secrets mechanism, and independent/hands-on confirmation that host credentials cannot leak into sandboxed code.

   • [claimed-docs] “Programmatically scale secure, ephemeral environments for running untrusted code.”
   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Autonomous agents with the right tools, context, and credentials already in place — running securely in a full, isolated dev environment.”
   • [claimed-docs] “Networking and security”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “We execute the agent's code in a Modal Sandbox, which allows us to run arbitrary code in a safe environment.”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”

   proof ↗⚑ flag

   Maritime~partialclaimed6/10

   Maritime's docs describe per-agent micro-VM isolation with secrets 'injected only into that agent's VM' and encrypted at rest, plus scoped API keys so a leaked key 'can only do what its scope allows,' which together imply host-level credentials aren't reachable by agent code. However, there's no explicit documentation of sandbox mechanics like blocking cloud metadata endpoints, filesystem isolation from the host, or independent security testing confirming secrets are truly unreachable. missing for 10: explicit host-credential isolation guarantees (e.g., no metadata service access, filesystem boundary details), independent/hands-on security verification.

   • [claimed-docs] “Variables are **secret by default**: stored encrypted at rest, injected only into that agent's VM, and always masked in the UI.”
   • [claimed-docs] “Hand a subsystem the narrowest key it needs. A leaked key can then only do what its scope allows.”
   • [claimed-docs] “To hand out a key that reaches computers and nothing else, create a **Computers key** in the Computers section of Settings, API keys”
   • [claimed-docs] “a key whose only scope is `computers` is refused on every other Maritime endpoint”
   • [claimed-docs] “Every key carries scopes. Hand a subsystem the _narrowest_ key it needs. A leaked key can then only do what its scope allows.”

   proof ↗⚑ flag

Network policy

1. platform-engineerRestrict or allow the sandbox's network egress with explicit policy

   weight 3 · round to Modal Sandboxes
   Modal Sandboxes~partialprobed5/10

   Modal has a dedicated 'Networking and security' guide for Sandboxes, confirming the platform documents network controls for sandboxes, but the evidence pack only shows the page title without excerpts detailing how egress allow/deny policies are configured. missing for 10: actual policy syntax/examples (e.g. allowlist/blocklist configuration), confirmation of default-deny behavior, independent/hands-on verification that egress restrictions work as documented.

   • [claimed-docs] “Networking and security”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”

   proof ↗⚑ flag

   Maritime—none0/10

   No evidence pack item mentions network egress controls, firewall rules, or any explicit allow/deny policy for outbound traffic from the sandbox/agent VM; the closest material covers API key scoping and secret management, not network policy.

   ⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round drawn
   Modal Sandboxes~partialprobed6/10

   Modal Sandboxes is fundamentally API/SDK-driven (Python, JS/Go) with a full CLI for running commands and managing containers, and docs describe programmatic control over image building, execution, filesystem, networking, snapshots, and logs — suggesting the API is the primary and most complete interface. However, there's no explicit documentation stating full UI/API parity or listing any UI-exclusive features, and no OpenAPI spec is discoverable (probe found 404s), making it hard to confirm completeness of a public REST API surface beyond the SDKs/CLI. Missing for 10: explicit parity statement between dashboard UI and API/CLI capabilities, and a public API reference/spec confirming no UI-only actions.

   • [claimed-docs] “Filesystem Access”
   • [claimed-docs] “Snapshots”
   • [claimed-docs] “Networking and security”
   • [claimed-docs] “Running commands”
   • [claimed-docs] “Run a command or interactive shell inside a Modal container.”
   • [claimed-docs] “Manage and connect to running containers.”
   • [github] “The JS and Go SDKs allow you to use Modal Sandboxes, invoke deployed Modal Functions, and interact with some Modal platform resources.”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://modal.com/openapi.json, https://modal.com/swagger.json, https://modal.com/api/openapi.json, …”
   • [probe] “official CLI documented at https://modal.com/docs/reference/cli”

   proof ↗⚑ flag

   Maritime~partialprobed6/10

   Docs show a broad API/SDK/CLI surface (provisioning, keys, webhooks, triggers, computers, templates, builds) and CLI --json mode implying the CLI itself is just an API wrapper, suggesting an API-first architecture. However there's no explicit statement of full UI/API parity and no discoverable OpenAPI spec (probes return 404s for all schema locations), so completeness of API coverage relative to every UI feature can't be confirmed. missing for 10: explicit parity statement, published OpenAPI/API reference, independent confirmation that every UI action has an API equivalent.

   • [claimed-docs] “Mint a personal API key with `maritime keys create` and send it as a Bearer token, the same token the CLI itself uses.”
   • [claimed-docs] “Pass --json on EVERY command. Success: one JSON value on stdout, stderr empty.”
   • [claimed-docs] “Tag each agent with your own id via `externalId`. `provision` is **idempotent** on it: it returns the existing agent if there is one, else c…”
   • [claimed-docs] “Hand a subsystem the narrowest key it needs. A leaked key can then only do what its scope allows.”
   • [claimed-docs] “A persistent Linux desktop (XFCE, Chromium, LibreOffice) for each of _your_ end users, driven by _your_ model.”
   • [claimed-docs] “The agent page streams the build log live under **Overview → Builds**.”
   • [claimed-docs] “Templates are cloneable agent packages; publishers can share them via link or the public library.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://maritime.sh/openapi.json, https://maritime.sh/swagger.json, https://maritime.sh/api/openapi.…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round drawn
   Modal Sandboxes—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   Maritime—none0/10

   Evidence shows import/migration INTO Maritime (from AWS/Heroku/VPS, .env bulk import) and a CLI with --json output, but nothing documents exporting agent configs, data, secrets, or state OUT in open formats to leave the platform. missing for 10: an export command or documented data-portability feature, open-format export of agent state/configs, any migration-out guide.

   • [claimed-docs] “Bulk import from a .env file (or stdin), then hot-reload”
   • [claimed-docs] “Migrate from AWS, Heroku, or a VPS”
   • [claimed-docs] “Pass --json on EVERY command. Success: one JSON value on stdout, stderr empty.”

   proof ↗⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round drawn
   Modal Sandboxes—none0/10

   Evidence only shows a GitHub repo for the client SDK (modal-client) used to invoke Modal Sandboxes, not the underlying sandbox/runtime platform itself, and no license terms are mentioned. There is no evidence that Modal Sandboxes' core product source is published under an open license.

   • [github] “The JS and Go SDKs allow you to use Modal Sandboxes, invoke deployed Modal Functions, and interact with some Modal platform resources.”

   proof ↗⚑ flag

   Maritime—none0/10

   Maritime is a closed commercial SaaS platform; there is no evidence of an open-source license or public source repository anywhere in the docs or probes. missing for 10: an open-source license, a public source code repository, any mention of licensing terms for the platform's own code.

   ⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round drawn
   Modal Sandboxes—none0/10

   Modal is a fully-managed cloud platform for running Sandboxes; there is no evidence of any self-hostable/on-prem version of the core product, and everything described (pay-per-second billing, GPU access, hosted CLI) implies a proprietary SaaS deployment model only.

   ⚑ flag

   Maritime—none0/10

   Maritime is explicitly described as a cloud platform with agents running on serverless infrastructure, and no documentation offers a self-hosted deployment option for the core platform itself (Docker images are for agent frameworks running atop Maritime's cloud, not for self-hosting Maritime).

   • [probe] “PROBE llms.txt: HTTP 200 at https://maritime.sh/llms.txt # Maritime > Maritime is a cloud platform for deploying, hosting, and managing AI …”
   • [claimed-docs] “Migrate from AWS, Heroku, or a VPS”

   proof ↗⚑ flag

Latency

1. developerStart sandboxes with documented sub-second-to-few-second cold starts

   weight 3 · round to Modal Sandboxes
   Modal Sandboxes~partialclaimed4/10

   Evidence shows general speed/scale claims ("spin up hundreds of thousands of concurrent rollout environments in seconds," faster backend for higher creation rates) but no explicit documented cold-start latency benchmark (e.g., sub-second or few-second startup time) for sandboxes specifically. Missing for 10: a concrete cold-start time benchmark/number, independent performance verification, and comparison across sandbox sizes/images.

   • [claimed-docs] “Spin up hundreds of thousands of concurrent rollout environments in seconds.”
   • [claimed-docs] “Spin up hundreds of thousands of concurrent rollout environments in seconds. Fast enough to keep your GPU inference resources saturated acro…”
   • [claimed-docs] “it's now possible to opt into a more performant Sandbox backend, which affords substantially higher creation rates and concurrency”
   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”

   proof ↗⚑ flag

   Maritime—none0/10

   Maritime's docs describe sleep/wake behavior (e.g., 'Sleeping agents wake automatically', 'puts it to sleep when idle, and wakes it on the next visit') and an 'always-on' add-on for millisecond reaction, but no evidence anywhere documents actual cold-start latency numbers (sub-second to few-second) for waking a sandboxed agent from sleep.

   • [claimed-docs] “Send a message and wait for the reply. Sleeping agents wake automatically.”
   • [claimed-docs] “Any repo with a Dockerfile can be served on a public, no-login HTTPS URL. Maritime builds it, runs it serverlessly, puts it to sleep when id…”
   • [claimed-docs] “Always-on is a flat add-on for the rare agent that must react in milliseconds around the clock.”

   proof ↗⚑ flag

Scale

1. platform-engineerRun large concurrent fleets of sandboxes with documented concurrency limits

   weight 3 · round to Modal Sandboxes
   Modal Sandboxes~partialcommunity5/10

   Modal markets very high concurrency ('hundreds of thousands of concurrent rollout environments', 'scale to thousands of concurrent runs') and even documents a backend upgrade for higher creation rates/concurrency, plus community reports of building scalable runner fleets on Sandboxes, showing the capability is real and used at scale. However there is no evidence of documented hard limits, quotas, or per-account/per-plan concurrency ceilings that a platform engineer could plan capacity against. Missing for 10: explicit documented concurrency/rate limits or quotas, benchmark/load-test data, and any per-tier scaling caps.

   • [claimed-docs] “Spin up hundreds of thousands of concurrent rollout environments in seconds.”
   • [claimed-docs] “H100s, A100s, A10Gs available on demand. Attach to any sandbox, scale to thousands of concurrent runs, pay by the second with no reserved ca…”
   • [claimed-docs] “Spin up hundreds of thousands of concurrent rollout environments in seconds. Fast enough to keep your GPU inference resources saturated acro…”
   • [claimed-docs] “Attach to any sandbox, scale to thousands of concurrent runs, pay by the second with no reserved capacity.”
   • [claimed-docs] “it's now possible to opt into a more performant Sandbox backend, which affords substantially higher creation rates and concurrency”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”

   proof ↗⚑ flag

   Maritime—none0/10

   No evidence of documented concurrency limits or guidance for running large fleets of sandboxes at scale; docs cover provisioning, idempotency, and billing per-agent but never state concurrency ceilings, throughput benchmarks, or fleet-scale operational guidance.

   ⚑ flag

2. developerKeep a sandbox session running for hours or days for long agent tasks

   weight 2 · round drawn
   Modal Sandboxes~partialcommunity5/10

   Modal docs describe sandboxes as isolated, scalable execution environments with snapshotting, filesystem, and networking support, and a community project (self-hosted GitHub Actions runners) demonstrates sandboxes being kept alive for extended, real workloads. However, no evidence pack item states explicit maximum session duration, timeout configuration, or guarantees for multi-day persistence, so long-running-session support is inferred rather than directly documented. Missing for 10: explicit docs on configurable/no timeout limits, guidance for multi-day sandbox lifecycles, and independent confirmation of hours/days-long sessions beyond one community anecdote.

   • [claimed-docs] “Snapshots”
   • [claimed-docs] “VM Sandboxes Beta”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [claimed-docs] “Integrated logging and full visibility into every function, sandbox, and container.”
   • [claimed-docs] “We've added a modal.Sandbox.logs API, allowing you to fetch() logs from a specific date/time range or to tail() the most recent logs.”

   proof ↗⚑ flag

   Maritime~partialprobed5/10

   Maritime advertises serverless infra by default with an 'always-on' add-on for agents that must run continuously, and a persistent Linux desktop model for end users, suggesting long-lived sessions are possible but only via a paid add-on rather than a default guarantee for hours/days-long sandbox sessions. missing for 10: explicit documentation of session/session-timeout limits, confirmation that a sandbox session can run unattended for hours or days without the always-on add-on, and independent verification of long-running task stability.

   • [claimed-docs] “Always-on is a flat add-on for the rare agent that must react in milliseconds around the clock.”
   • [claimed-docs] “A persistent Linux desktop (XFCE, Chromium, LibreOffice) for each of _your_ end users, driven by _your_ model.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://maritime.sh/llms.txt # Maritime > Maritime is a cloud platform for deploying, hosting, and managing AI …”

   proof ↗⚑ flag

Pricing

1. developerStart building with a free tier or included credits without talking to sales

   weight 1 · round to Maritime
   Modal Sandboxes—none0/10

   The evidence pack includes pricing-related docs (pay-per-use, burst capacity) but no mention of a free tier, free credits, or self-serve signup without contacting sales; nothing confirms developers can start building without sales contact.

   ⚑ flag

   Maritime✓fullclaimed8/10

   Docs explicitly state the free plan includes 3 agents with no card required, and quickstart/CLI docs show a self-serve path (npm install, deploy in under 5 minutes) with no sales contact implied anywhere. missing for 10: independent/third-party confirmation of the free tier's real-world limits and no hands-on verification of the signup flow.

   • [claimed-docs] “the free plan includes 3 agents, no card required”
   • [claimed-docs] “Deploy your first agent and talk to it, in under 5 minutes.”
   • [claimed-docs] “npm install -g maritime-cli”
   • [claimed-docs] “One subscription, a set number of agents, flat prices for extras. No meters, no hourly rates: how much your agents run never changes the bil…”

   proof ↗⚑ flag

2. platform-engineerPay per second only for the compute a sandbox actually uses

   weight 3 · round to Modal Sandboxes
   Modal Sandboxes✓fullcommunity8/10

   Modal's pricing docs and marketing explicitly state per-second billing with no reserved capacity ('pay by the second with no reserved capacity', 'only pay for what you use... burst up to what you need without over-allocating'), directly matching the story, and a community user corroborates paying only for resources actually consumed when building GPU runners on Sandboxes. Missing for 10: independent verification of exact billing granularity/metering mechanics and no first-party pricing breakdown showing per-second line items in the evidence pack.

   • [claimed-docs] “Only pay for what you use. Burst up to what you need without over-allocating CPU or memory in advance.”
   • [claimed-docs] “Attach to any sandbox, scale to thousands of concurrent runs, pay by the second with no reserved capacity.”
   • [claimed-docs] “Burst up to what you need without over-allocating CPU or memory in advance.”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”

   proof ↗⚑ flag

   Maritime—none0/10

   Maritime's own billing docs explicitly state the opposite of per-second usage billing: 'One subscription, a set number of agents, flat prices for extras. No meters, no hourly rates: how much your agents run never changes the bill' and 'Always-on is a flat add-on.' There is no evidence of any per-second or usage-metered compute pricing for sandboxes.

   • [claimed-docs] “One subscription, a set number of agents, flat prices for extras. No meters, no hourly rates: how much your agents run never changes the bil…”
   • [claimed-docs] “Always-on is a flat add-on for the rare agent that must react in milliseconds around the clock.”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round drawn
   Modal Sandboxes—none0/10

   No evidence pack item mentions data residency, region selection, or geographic data storage controls for Modal Sandboxes; the pack covers execution, scaling, GPUs, filesystem, and networking but nothing about choosing storage region/residency.

   ⚑ flag

   Maritime—none0/10

   No evidence in the pack mentions data residency, region selection, or geographic storage controls anywhere in Maritime's docs or probes; nothing addresses where agent/user data is physically stored.

   ⚑ flag

2. ai-native userControl data retention and deletion

   weight 2 · round drawn
   Modal Sandboxes—none0/10

   Evidence covers sandbox ephemerality, filesystem access, snapshots, and networking/security, but nothing describes data retention policies, deletion controls, or compliance/data-handling settings that a user could configure. missing for 10: explicit data retention policy documentation, user-controllable deletion/purge mechanisms, compliance certifications (e.g., SOC2/GDPR data handling), retention period configuration.

   ⚑ flag

   Maritime—none0/10

   No evidence describes data retention policies, deletion controls, or export/purge mechanisms for agent data, files, or session logs; docs mention encrypted-at-rest secrets and append-only session logs but nothing about user-controlled retention or deletion.

   ⚑ flag

3. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Modal Sandboxes—none0/10

   No evidence in the pack mentions telemetry, usage tracking, analytics collection, or an opt-out mechanism for Modal Sandboxes; the docs cover sandboxing, GPUs, filesystem, and logging but not data-collection/privacy controls.

   ⚑ flag

   Maritime—none0/10

   No evidence pack item mentions telemetry, usage tracking, analytics collection, or an opt-out mechanism anywhere in Maritime's docs; the closest topics (scoped API keys, secret variables) address access control, not telemetry preferences.

   ⚑ flag

Lifecycle

1. developerSet timeouts so sandboxes shut down automatically and stop billing when idle or done

   weight 2 · round drawn
   Modal Sandboxes—none0/10

   The evidence pack only mentions general pay-per-second billing and bursty scaling, but never documents an explicit sandbox timeout or idle-shutdown parameter/config. No citation confirms configurable timeouts that auto-terminate sandboxes when idle or complete.

   ⚑ flag

   Maritime—none0/10

   The docs describe Maritime as serverless agent hosting and mention an 'always-on' paid add-on (implying default agents are not always running), but no evidence describes a developer-configurable idle timeout or automatic shutdown-to-stop-billing mechanism. Missing for 10: any documented timeout setting, idle-detection policy, or billing-pause behavior tied to inactivity.

   • [claimed-docs] “Always-on is a flat add-on for the rare agent that must react in milliseconds around the clock.”

   proof ↗⚑ flag

2. developerSpin up an isolated sandbox with one API/SDK call and get a live environment in seconds

   weight 3 · round to Modal Sandboxes
   Modal Sandboxes✓fullprobed8/10

   Docs and SDK references (Python, JS/Go) show a single call (e.g. modal.Sandbox.create) spins up an isolated, ephemeral container/VM in seconds, with fast creation rates highlighted (hundreds of thousands of concurrent environments in seconds, faster backend for higher creation rates). Community builders (HN posts) independently confirm using Modal Sandboxes programmatically for real workloads like CI runners and agent runtimes. Missing for 10: independent benchmark of actual cold-start latency and a public OpenAPI spec (probe found 404s), so exact 'seconds' claim isn't third-party verified.

   • [claimed-docs] “Spin up fresh, isolated sandboxes programmatically. Custom images, any dependency — built for the latency and scale consumer AI products dem…”
   • [claimed-docs] “Spin up hundreds of thousands of concurrent rollout environments in seconds.”
   • [claimed-docs] “Using Sandboxes in JS/Go projects, to safely execute arbitrary commands, run untrusted user code, or as a safe environment for AI agents.”
   • [claimed-docs] “Create a Sandbox with the specified Image, and mount a Volume”
   • [claimed-docs] “it's now possible to opt into a more performant Sandbox backend, which affords substantially higher creation rates and concurrency”
   • [community] “Built GitHub Actions self-hosted runners on Modal Sandboxes: 'was looking for a long time for gpu-native scalable and highly available runne…”
   • [community] “Developer built MCP-JS, a JavaScript runtime for agents, and got gRPC working to control Modal Sandboxes: 'been working on a javascript runt…”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”

   proof ↗⚑ flag

   Maritime✓fullprobed7/10

   Maritime's SDK exposes an idempotent `provision` call that creates/returns a live agent environment, and docs claim full agent deployment (image pull → live, chat-ready) in under 5 minutes, with CLI/API equivalents (`maritime keys create`, RESTful API). This matches the 'one call, live environment fast' story well, though the fastest documented time is minutes not literal seconds and there's no independent/hands-on latency benchmark or public OpenAPI spec confirming the exact call shape. Missing for 10: independent timing/benchmark evidence corroborating 'seconds', and a verifiable OpenAPI/API schema (probe shows openapi.json 404).

   • [claimed-docs] “Tag each agent with your own id via `externalId`. `provision` is **idempotent** on it: it returns the existing agent if there is one, else c…”
   • [claimed-docs] “provision is idempotent on it: it returns the existing agent if there is one, else creates a new one. Safe to call on every sign-in.”
   • [claimed-docs] “Deploy your first agent and talk to it, in under 5 minutes.”
   • [claimed-docs] “Every ready-made template is a real agent framework, pre-configured and one image pull from live. Pick one, name it, deploy.”
   • [claimed-docs] “RESTful API for managing agents programmatically.”
   • [claimed-docs] “The Maritime CLI lets you deploy and manage agents from your terminal.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://maritime.sh/openapi.json, https://maritime.sh/swagger.json, https://maritime.sh/api/openapi.…”

   proof ↗⚑ flag

Preview access

1. developerExpose a port from the sandbox on a public preview URL to reach services running inside

   weight 2 · round to Maritime
   Modal Sandboxes~partialprobed3/10

   The docs pack includes a 'Networking and security' guide for sandboxes (modal-docs-10) which is the likely location for tunnel/port-exposure features, but no evidence text actually describes exposing a port to a public preview URL, tunnel URLs, or reachability details. missing for 10: explicit documentation or example showing sandbox.tunnels()/port forwarding, a public URL format, and independent confirmation of accessing services inside a sandbox via that URL.

   • [claimed-docs] “Networking and security”
   • [claimed-docs] “VM Sandboxes Beta”
   • [probe] “PROBE docs-md: HTTP 200 at https://modal.com/docs/guide/sandboxes.md # Sandboxes This page is a high-level guide to Sandboxes, secure conta…”

   proof ↗⚑ flag

   Maritime~partialclaimed6/10

   Maritime's docs describe deploying any Dockerfile-based repo to a public, no-login HTTPS URL that reaches the running container's service, matching the core story of exposing an internal service via a public preview URL (maritime-docs-29, maritime-docs-47). However, the docs never explicitly describe port-selection mechanics (e.g., multiple ports, custom port binding) and all evidence is first-party with no independent/hands-on corroboration. missing for 10: explicit port-selection/multi-port docs, independent verification that the public URL reaches an arbitrary internal port.

   • [claimed-docs] “Any repo with a Dockerfile can be served on a public, no-login HTTPS URL. Maritime builds it, runs it serverlessly, puts it to sleep when id…”
   • [claimed-docs] “Any repo with a Dockerfile can be served on a public, no-login HTTPS URL.”

   proof ↗⚑ flag

Snapshots

1. developerPause a running sandbox and resume it later with filesystem and memory state intact

   weight 2 · round to Modal Sandboxes
   Modal Sandboxes~partialclaimed4/10

   Modal has a dedicated 'Snapshots' doc for sandboxes suggesting some pause/resume-with-state capability, but the evidence pack only gives the page title with no detail on what state (filesystem/memory) is preserved or how resume works. Missing for 10: documented mechanics of snapshot/resume (filesystem+memory fidelity), explicit confirmation memory state is preserved, independent/hands-on validation of the feature working as described.

   • [claimed-docs] “Snapshots”
   • [claimed-docs] “Filesystem Access”

   proof ↗⚑ flag

   Maritime—none0/10

   No evidence in the pack mentions pausing/suspending a sandbox or preserving filesystem and memory state across a pause/resume cycle; docs cover provisioning, webhooks, triggers, templates, and CLI/API auth but nothing about snapshot or suspend-resume persistence.

   ⚑ flag

2. developerSnapshot a sandbox and later restore or fork new sandboxes from that snapshot

   weight 3 · round to Modal Sandboxes
   Modal Sandboxes~partialclaimed5/10

   Modal explicitly documents a 'Sandbox Snapshots' feature (modal-docs-9), confirming snapshot capability exists, but the evidence pack gives no detail on the actual snapshot/restore/fork API mechanics, no confirmation that new independent sandboxes can be forked from a single snapshot, and no community/hands-on corroboration of this specific workflow. missing for 10: documented API details on restoring/forking multiple sandboxes from one snapshot, independent/hands-on confirmation of snapshot-restore-fork workflow, and clarity on any beta/experimental limitations.

   • [claimed-docs] “Snapshots”
   • [claimed-docs] “VM Sandboxes Beta”

   proof ↗⚑ flag

   Maritime—none0/10

   No evidence in the pack mentions snapshotting a sandbox/agent state or restoring/forking new instances from such a snapshot; templates (maritime-docs-7) allow cloning a pre-configured agent package but not capturing/restoring runtime state.

   ⚑ flag