Skip to content
ProductArena

Agent Sandboxes & Code Execution Arena

Agent Sandboxes & Code Execution arenaBuyer checklist

Every requirement we judge agent sandboxes & code execution products against, as a ready-to-send RFP checklist — with each item's priority, why it matters, and how the top-ranked products score on it today.

51 requirements · 12 themes · verdicts for 8 products · updated 2026-09-16 · priorities mirror the story weights our scoring uses (methodology)

Procurement report →
Show the markdown export
# Agent Sandboxes & Code Execution — buyer checklist (RFP)

Derived from ProductArena's evidence-graded user-story taxonomy for Agent Sandboxes & Code Execution: 51 judged requirements. Priorities mirror story weights (3 = must-have, 2 = should-have, 1 = nice-to-have).

## Agenticness

- [ ] **[must-have]** Plug MCP servers into this product so it can use their tools
- [ ] **[must-have]** Connect an agent via an official MCP server
- [ ] **[must-have]** Drive the product through a documented public API
- [ ] **[must-have]** Delegate tasks to a built-in AI assistant inside the product
- [ ] **[should-have]** Point an agent at llms.txt or agent-oriented docs
- [ ] **[should-have]** Run the product headlessly / in CI for automation
- [ ] **[should-have]** Use an official CLI
- [ ] **[should-have]** Issue scoped/least-privilege API credentials for an agent
- [ ] **[should-have]** Build against official SDKs
- [ ] **[should-have]** Subscribe to events via webhooks
- [ ] **[should-have]** Get AI-generated insights and suggestions from my data inside the product
- [ ] **[should-have]** Set up automations that run autonomously in the background
- [ ] **[should-have]** Operate the product with natural-language commands
- [ ] **[should-have]** Explore an interactive API reference with runnable examples
- [ ] **[should-have]** Download a machine-readable API spec (OpenAPI or equivalent)
- [ ] **[should-have]** Rely on versioned APIs with a documented deprecation policy
- [ ] **[nice-to-have]** Test against a sandbox environment without touching production data

## Automation depth

- [ ] **[must-have]** Define rules that trigger actions automatically on events
- [ ] **[should-have]** Perform bulk operations across many items at once
- [ ] **[should-have]** Schedule recurring jobs or workflows
- [ ] **[nice-to-have]** Version, review, and roll back my automations

## Capabilities hardware

- [ ] **[should-have]** Run coding agents like Claude Code or Codex inside the sandbox following the vendor's own recipe
- [ ] **[should-have]** Run a headless browser or full desktop environment inside the sandbox
- [ ] **[should-have]** Attach GPUs to sandboxed workloads

## Code execution

- [ ] **[must-have]** Execute untrusted, AI-generated code without risking my own infrastructure
- [ ] **[should-have]** Read, write, upload, and download files in the sandbox filesystem via the SDK
- [ ] **[should-have]** Define custom sandbox templates or bring my own container image
- [ ] **[should-have]** Execute code in multiple language runtimes (Python, JavaScript, and more) and get rich results back
- [ ] **[should-have]** Run arbitrary shell commands and install packages inside the sandbox

## Dev experience

- [ ] **[must-have]** My agent can provision its own sandbox, execute code, read the results, and tear it down — end to end without a human

## Isolation security

- [ ] **[must-have]** Rely on a documented hard isolation boundary (microVM or equivalent) between sandboxes and my systems
- [ ] **[must-have]** Restrict or allow the sandbox's network egress with explicit policy
- [ ] **[should-have]** Give an agent a sandbox where host secrets and credentials are unreachable by the code it runs

## Openness

- [ ] **[must-have]** Export all of my data in open formats and leave
- [ ] **[must-have]** Self-host the core product
- [ ] **[should-have]** Do everything through the API that I can do in the UI
- [ ] **[should-have]** Read the product's source under an open license

## Performance scale

- [ ] **[must-have]** Start sandboxes with documented sub-second-to-few-second cold starts
- [ ] **[must-have]** Run large concurrent fleets of sandboxes with documented concurrency limits
- [ ] **[should-have]** Keep a sandbox session running for hours or days for long agent tasks

## Pricing limits

- [ ] **[must-have]** Pay per second only for the compute a sandbox actually uses
- [ ] **[nice-to-have]** Start building with a free tier or included credits without talking to sales

## Privacy posture

- [ ] **[must-have]** Prevent my data from being used to train AI models
- [ ] **[should-have]** Choose where my data is stored (region/residency)
- [ ] **[should-have]** Control data retention and deletion
- [ ] **[should-have]** Opt out of telemetry and usage tracking

## Provisioning lifecycle

- [ ] **[must-have]** Spin up an isolated sandbox with one API/SDK call and get a live environment in seconds
- [ ] **[should-have]** Set timeouts so sandboxes shut down automatically and stop billing when idle or done
- [ ] **[should-have]** Expose a port from the sandbox on a public preview URL to reach services running inside

## Snapshot persistence

- [ ] **[must-have]** Snapshot a sandbox and later restore or fork new sandboxes from that snapshot
- [ ] **[should-have]** Pause a running sandbox and resume it later with filesystem and memory state intact

---

Source: https://ultrametric.ai/productarena/arena/agent-sandboxes (evidence-graded verdicts for 8 products) · methodology: https://ultrametric.ai/productarena/methodology

Chips show the top 5 ranked products' current verdict on each requirement — ✓ full · ~ partial · ! disputed · — none · n/a not applicable.

Agenticness — how well agents can access and operate the productAgenticness· 17 items

How well agents can access and operate the product

Automation depth — how much of the product can run unattendedAutomation depth· 4 items

How much of the product can run unattended

Capabilities hardware — stories about capabilities hardware in this arenaCapabilities hardware· 3 items

Stories about capabilities hardware in this arena

Code execution — stories about code execution in this arenaCode execution· 5 items

Stories about code execution in this arena

Dev experience — day-to-day developer experience — setup friction, docs, debugging, iteration speedDev experience· 1 item

Day-to-day developer experience — setup friction, docs, debugging, iteration speed

  • ai-native userMy agent can provision its own sandbox, execute code, read the results, and tear it down — end to end without a human

    Core requirement — weighs 3× in arena scoring · 6 of 8 products fully deliver this today

    E2B logoE2BDaytona logoDaytonaRunloop logoRunloopMaritime logoMaritimeVercel Sandbox logoVercel Sandbox
    must-have

Isolation security — stories about isolation security in this arenaIsolation security· 3 items

Stories about isolation security in this arena

  • platform-engineerRely on a documented hard isolation boundary (microVM or equivalent) between sandboxes and my systems

    Core requirement — weighs 3× in arena scoring · 2 of 8 products fully deliver this today

    E2B logoE2BDaytona logoDaytonaRunloop logoRunloopMaritime logoMaritimeVercel Sandbox logoVercel Sandbox
    must-have

  • platform-engineerRestrict or allow the sandbox's network egress with explicit policy

    Core requirement — weighs 3× in arena scoring · 2 of 8 products fully deliver this today

    E2B logoE2BDaytona logoDaytonaRunloop logoRunloopMaritime logoMaritimeVercel Sandbox logoVercel Sandbox
    must-have

  • ai-native userGive an agent a sandbox where host secrets and credentials are unreachable by the code it runs

    Important, not disqualifying — weighs 2× in arena scoring · 3 of 8 products fully deliver this today

    E2B logoE2BDaytona logoDaytonaRunloop logoRunloopMaritime logoMaritimeVercel Sandbox logoVercel Sandbox
    should-have

Openness — open source, data portability, and self-hosting storiesOpenness· 4 items

Open source, data portability, and self-hosting stories

Performance scale — stories about performance scale in this arenaPerformance scale· 3 items

Stories about performance scale in this arena

Pricing limits — free-tier ceilings, usage caps, and rate limits before you have to payPricing limits· 2 items

Free-tier ceilings, usage caps, and rate limits before you have to pay

Privacy posture — data-handling and privacy storiesPrivacy posture· 4 items

Data-handling and privacy stories

Provisioning lifecycle — creating, updating, and tearing down resources across their lifecycleProvisioning lifecycle· 3 items

Creating, updating, and tearing down resources across their lifecycle

Snapshot persistence — stories about snapshot persistence in this arenaSnapshot persistence· 2 items

Stories about snapshot persistence in this arena

Full evidence behind every verdict lives on the arena page and each product page — chips above deep-link straight to the judged story.