Agent Sandboxes & Code Execution Arena
Agent Sandboxes & Code Execution — procurement report
ProductArena · rankings as of 2026-09-16 · evidence as of 2026-09-16 · 8 products · 51 judged requirements · 408 judged cells
Methodology: Every product is judged against a shared taxonomy of user stories using cited evidence — hands-on probes > repository code > independent community sources > vendor claims — never opinion. Full writeup: https://ultrametric.ai/productarena/methodology
Leaderboard
| # | Product | PA Score | Coverage score | Applicable cells | Confidence |
|---|---|---|---|---|---|
| 1 | E2B | 34.1 | 47.6 | 47/51 | A |
| 2 | Daytona | 33.4 | 41.7 | 49/51 | B |
| 3 | Runloop | 29.8 | 36.4 | 49/51 | C |
| 4 | Maritime | 28.7 | 25.0 | 51/51 | C |
| 5 | Vercel Sandbox | 24.9 | 34.4 | 48/51 | C |
| 6 | Blaxel | 21.2 | 23.2 | 49/51 | C |
| 7 | Cloudflare Sandboxes | 21.0 | 25.9 | 48/51 | C |
| 8 | Modal Sandboxes | 20.1 | 32.4 | 48/51 | B |
PA Score = agent-readiness blend (see methodology). Coverage score = weighted share of judged requirements met. Confidence = how much of the score rests on tested vs claimed evidence (A–D).
Uncertainty note
The current #1/#2 gap in this arena is not close enough to qualify for the multi-judge uncertainty pass (or the pass has not covered it yet) — no extra caveat applies beyond the per-product confidence grades above.
Buyer checklist (RFP)
The arena's 51 judged user stories as requirements, grouped by theme. Priorities mirror the story weights our scoring uses (3 = must-have, 2 = should-have, 1 = nice-to-have). Interactive version with per-requirement verdicts for the top products: /arena/agent-sandboxes/checklist
Agenticness — how well agents can access and operate the productAgenticness
How well agents can access and operate the product
- ai-native userPlug MCP servers into this product so it can use their toolsmust-have
- ai-native userConnect an agent via an official MCP servermust-have
- ai-native userDrive the product through a documented public APImust-have
- ai-native userDelegate tasks to a built-in AI assistant inside the productmust-have
- ai-native userPoint an agent at llms.txt or agent-oriented docsshould-have
- ai-native userRun the product headlessly / in CI for automationshould-have
- ai-native userUse an official CLIshould-have
- ai-native userIssue scoped/least-privilege API credentials for an agentshould-have
- ai-native userBuild against official SDKsshould-have
- ai-native userSubscribe to events via webhooksshould-have
- ai-native userGet AI-generated insights and suggestions from my data inside the productshould-have
- ai-native userSet up automations that run autonomously in the backgroundshould-have
- ai-native userOperate the product with natural-language commandsshould-have
- ai-native userExplore an interactive API reference with runnable examplesshould-have
- ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)should-have
- ai-native userRely on versioned APIs with a documented deprecation policyshould-have
- ai-native userTest against a sandbox environment without touching production datanice-to-have
Automation depth — how much of the product can run unattendedAutomation depth
How much of the product can run unattended
- ai-native userDefine rules that trigger actions automatically on eventsmust-have
- ai-native userPerform bulk operations across many items at onceshould-have
- ai-native userSchedule recurring jobs or workflowsshould-have
- ai-native userVersion, review, and roll back my automationsnice-to-have
Capabilities hardware — stories about capabilities hardware in this arenaCapabilities hardware
Stories about capabilities hardware in this arena
- ai-native userRun coding agents like Claude Code or Codex inside the sandbox following the vendor's own recipeshould-have
- developerRun a headless browser or full desktop environment inside the sandboxshould-have
- developerAttach GPUs to sandboxed workloadsshould-have
Code execution — stories about code execution in this arenaCode execution
Stories about code execution in this arena
- developerExecute untrusted, AI-generated code without risking my own infrastructuremust-have
- developerRead, write, upload, and download files in the sandbox filesystem via the SDKshould-have
- developerDefine custom sandbox templates or bring my own container imageshould-have
- developerExecute code in multiple language runtimes (Python, JavaScript, and more) and get rich results backshould-have
- developerRun arbitrary shell commands and install packages inside the sandboxshould-have
Dev experience — day-to-day developer experience — setup friction, docs, debugging, iteration speedDev experience
Day-to-day developer experience — setup friction, docs, debugging, iteration speed
- ai-native userMy agent can provision its own sandbox, execute code, read the results, and tear it down — end to end without a humanmust-have
Isolation security — stories about isolation security in this arenaIsolation security
Stories about isolation security in this arena
- platform-engineerRely on a documented hard isolation boundary (microVM or equivalent) between sandboxes and my systemsmust-have
- platform-engineerRestrict or allow the sandbox's network egress with explicit policymust-have
- ai-native userGive an agent a sandbox where host secrets and credentials are unreachable by the code it runsshould-have
Openness — open source, data portability, and self-hosting storiesOpenness
Open source, data portability, and self-hosting stories
- ai-native userExport all of my data in open formats and leavemust-have
- ai-native userSelf-host the core productmust-have
- ai-native userDo everything through the API that I can do in the UIshould-have
- ai-native userRead the product's source under an open licenseshould-have
Performance scale — stories about performance scale in this arenaPerformance scale
Stories about performance scale in this arena
- developerStart sandboxes with documented sub-second-to-few-second cold startsmust-have
- platform-engineerRun large concurrent fleets of sandboxes with documented concurrency limitsmust-have
- developerKeep a sandbox session running for hours or days for long agent tasksshould-have
Pricing limits — free-tier ceilings, usage caps, and rate limits before you have to payPricing limits
Free-tier ceilings, usage caps, and rate limits before you have to pay
- platform-engineerPay per second only for the compute a sandbox actually usesmust-have
- developerStart building with a free tier or included credits without talking to salesnice-to-have
Privacy posture — data-handling and privacy storiesPrivacy posture
Data-handling and privacy stories
- ai-native userPrevent my data from being used to train AI modelsmust-have
- ai-native userChoose where my data is stored (region/residency)should-have
- ai-native userControl data retention and deletionshould-have
- ai-native userOpt out of telemetry and usage trackingshould-have
Provisioning lifecycle — creating, updating, and tearing down resources across their lifecycleProvisioning lifecycle
Creating, updating, and tearing down resources across their lifecycle
- developerSpin up an isolated sandbox with one API/SDK call and get a live environment in secondsmust-have
- developerSet timeouts so sandboxes shut down automatically and stop billing when idle or doneshould-have
- developerExpose a port from the sandbox on a public preview URL to reach services running insideshould-have
Snapshot persistence — stories about snapshot persistence in this arenaSnapshot persistence
Stories about snapshot persistence in this arena
- developerSnapshot a sandbox and later restore or fork new sandboxes from that snapshotmust-have
- developerPause a running sandbox and resume it later with filesystem and memory state intactshould-have
Appendix: recorded probes
Hands-on probe recordings — transcripts/videos a human can replay, the strongest evidence tier. Watch them at https://ultrametric.ai/productarena/proofs
- Blaxel
curl -s https://docs.blaxel.ai/Get-started.md | head -6terminal · recorded 2026-09-10 · exit 0 - Blaxel
curl -s https://blaxel.ai/llms.txt | head -6terminal · recorded 2026-09-10 · exit 0 - Blaxel
curl -si -X POST https://api.blaxel.ai/v0/mcp -H 'Content-Type: application/json' -d '<jsonrpc initialize>'terminal · recorded 2026-09-10 · exit 0 - E2B
e2b --helpterminal · recorded 2026-09-04 · exit 0 - E2B
e2b --versionterminal · recorded 2026-09-04 · exit 0 - Maritime
curl -si https://maritime.sh/api/agentsterminal · recorded 2026-09-15 · exit 0 - Maritime
npx -y maritime-cli --versionterminal · recorded 2026-09-15 · exit 0 - Maritime
curl -s https://maritime.sh/llms.txt | head -4terminal · recorded 2026-09-15 · exit 0 - Modal Sandboxes
modal --helpterminal · recorded 2026-09-04 · exit 0 - Modal Sandboxes
modal --versionterminal · recorded 2026-09-04 · exit 0
Cite as: ProductArena by Ultrametric Inc, Agent Sandboxes & Code Execution arena, rankings as of 2026-09-16 — https://ultrametric.ai/productarena/arena/agent-sandboxes
License: © 2026 Ultrametric Inc. Brief quotation of individual verdicts, scores, or evidence excerpts is permitted with attribution to "ProductArena by Ultrametric Inc (ultrametric.ai/productarena)", as is use of the data to evaluate, contest, or contribute corrections. Bulk copying, redistribution, or use to build competing datasets requires prior written permission (see DATA-LICENSE in the repository).
No liability: rankings, verdicts, and scores are research outputs derived from the cited evidence at a point in time, provided "as is", without warranties. Ultrametric Inc accepts no responsibility for procurement, purchasing, or other decisions made in reliance on them — verify against the cited evidence before acting (https://ultrametric.ai/productarena/terms).