Steel vs Smooth

Caching

1. developerCache resolved actions or generated code so repeat runs replay deterministically at lower cost and latency than re-prompting the LLM

   weight 2 · round drawn
   Steel—none0/10

   Steel's evidence covers session management, stealth, proxies, human-in-the-loop debugging, and agent traces, but nothing about caching resolved actions or generated code to enable deterministic, lower-cost replay without re-invoking the LLM. Agent traces (steel-docs-6/7) provide observability/export, not action-cache replay for cost savings.

   ⚑ flag

   Smooth—none0/10

   No evidence of caching resolved actions/generated code for deterministic, cheaper replay; docs mention persistent sessions (auth reuse), structured outputs, and cost efficiency via small models, but nothing about caching or replay of prior task executions to skip re-prompting the LLM. Missing for 10: any mention of action/result caching, replay mechanism, or cost/latency comparison for repeat runs.

   • [claimed-docs] “Log in once, then reuse that authentication for future tasks.”
   • [claimed-docs] “Structured outputs allow you to write deterministic code based on the agent's output. To activate structured outputs, set the `response_mode…”
   • [claimed-docs] “Smooth uses small and efficient AI models, making it 7x more affordable than browser-use.”

   proof ↗⚑ flag

Dom

1. developerDrive the page through DOM-understanding action primitives (act/click/type on described elements) that survive selector and layout changes

   weight 3 · round to Smooth
   Steel—none0/10

   Steel's evidence shows only traditional CDP/Puppeteer/Selenium-based control and CLI commands like click/fill/type (steel-docs-8, steel-gh-1), which are selector-based automation primitives, not AI/DOM-understanding 'act on described element' primitives that resolve targets semantically and survive selector/layout changes. No documentation or hands-on evidence describes a Stagehand-like natural-language action resolver.

   • [claimed-docs] “The Steel CLI lets you run full browser workflows from the terminal, end-to-end. You can start a browser session, navigate pages, click/fill…”
   • [github] “Uses Puppeteer and CDP for complete control over Chrome instances -- allowing you to connect using Puppeteer, Playwright, or Selenium.”

   proof ↗⚑ flag

   Smooth~partialcommunity5/10

   Smooth's docs describe a 'Session Workflow' that lets you orchestrate smaller tasks, navigate to URLs, and extract data within a persistent session, and the whole product is framed as an AI browser agent that understands pages rather than relying on brittle selectors (smooth-docs-7, smooth-gh-1). However, there is no explicit documentation of discrete act/click/type primitives on described elements, nor any evidence/testing showing these survive selector or layout changes — community comments even note it doesn't fully close the gap versus Playwright-style tools (smooth-comm-8). Missing for 10: explicit act/click/type API reference, documented resilience testing against DOM/selector changes, independent verification of robustness claims.

   • [claimed-docs] “Session Workflow — Multi-step execution where you can orchestrate smaller tasks, navigate to URLs, and extract data within a persistent brow…”
   • [github] “The Smooth CLI is a browser for AI agents, enabling tools like Claude Code to navigate the web quickly, cheaply, and reliably.”
   • [community] “agent-browser helped a lot over playwright but doesn't completely close the gap.”

   proof ↗⚑ flag

Observe

1. developerPreview candidate actions on the current page (observe/plan) before committing the agent to act

   weight 1 · round to Steel
   Steel~partialclaimed4/10

   Steel exposes browser observation primitives (page-to-markdown/readability/screenshot extraction, agent traces/timeline, and a debug URL for human-in-the-loop review) that let a developer inspect page state or a human intervene mid-session, but there is no documented 'plan' or dry-run API that lets an agent preview a set of candidate actions before committing to execute them. missing for 10: an explicit plan/observe-then-act primitive or dry-run action preview API, evidence of independent developers using it specifically for pre-commit action review.

   • [github] “Browser Tools: Exposes APIs to quick convert pages to markdown, readability, screenshots, or PDFs.”
   • [claimed-docs] “Steel's debug URL feature allows you to implement human-in-the-loop workflows where users can directly interact with and control browser ses…”
   • [claimed-docs] “It turns the run into a timeline of agent activity, so you can see what happened without scrubbing through the whole recording.”

   proof ↗⚑ flag

   Smooth—none0/10

   Smooth's docs describe live viewing of actions as they execute (live_url) and data extraction, but there is no evidence of a distinct observe/plan step that lets a developer preview candidate actions before committing the agent to act.

   ⚑ flag

Vision

1. developerSwitch to a vision or computer-use action mode that operates on screenshots for canvases and UIs the DOM path can't handle

   weight 2 · round drawn
   Steel—none0/10

   Evidence shows Steel can capture screenshots, convert pages to markdown/PDF, and offers a debug URL for human-in-the-loop control, but there is no mention of a vision/computer-use action mode where an agent issues click/type actions based on screenshot coordinates instead of DOM selectors.

   ⚑ flag

   Smooth—none0/10

   The evidence pack covers Smooth's session workflows, extraction, custom tools, proxies, and CAPTCHA solving, but nowhere mentions a vision/computer-use mode operating on screenshots for canvases or non-DOM UI elements. This axis is plausible for a browser-automation agent, but no documentation or community report confirms such a capability exists.

   • [claimed-docs] “Session Workflow — Multi-step execution where you can orchestrate smaller tasks, navigate to URLs, and extract data within a persistent brow…”
   • [claimed-docs] “Extract structured data from the current page by providing a schema.”
   • [claimed-docs] “Custom tools allow you to give Smooth any arbitrary function as a tool.”
   • [github] “The Smooth CLI is a browser for AI agents, enabling tools like Claude Code to navigate the web quickly, cheaply, and reliably.”

   proof ↗⚑ flag

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to Steel
   Steel✓fullprobed9/10

   Probe confirms llms.txt is live at docs.steel.dev/llms.txt (HTTP 200) with agent-oriented framing, and the docs also expose an OpenAPI spec, making the docs machine/agent consumable. missing for 10: no independent third-party confirmation that agents actually consume the llms.txt file successfully in practice.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.steel.dev/llms.txt # Steel Documentation > Steel is the open-source browser API for AI agents — ma…”
   • [probe] “PROBE openapi: HTTP 200 at https://docs.steel.dev/openapi.json — contains "openapi" key”

   proof ↗⚑ flag

   Smooth✓fullprobed8/10

   A direct probe confirms docs.smooth.sh/llms.txt returns HTTP 200 with a structured index of docs pages, and the docs themselves are mirrored as .md files (e.g. live-share.md, proxies.md) enabling agent-friendly consumption. One community comment notes the docs pages aren't fully token-efficient, a minor caveat. Missing for 10: independent verification that agents actually consume llms.txt effectively, and no evidence of additional agent-specific doc formats beyond the single llms.txt file.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.smooth.sh/llms.txt # Smooth ## Docs - [Introduction](https://docs.smooth.sh/index.md): Welcome to…”
   • [claimed-docs] “When running a task, you will receive a `live_url`, which can be used to view the agent actions live.”
   • [claimed-docs] “Log in once, then reuse that authentication for future tasks.”
   • [community] “Ironically, the landing page and docs pages of Smooth aren't all that token-efficient!”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to Steel
   Steel✓fullprobed9/10

   Steel provides a documented CLI for end-to-end headless browser workflows from the terminal, a REST/SDK API for programmatic session creation, an open-source Docker image for self-hosting, and a verified probe confirming a keyless self-host roundtrip (docker run, health check, session creation via API, CLI install, SDK install) — all strongly supporting CI/headless automation use. missing for 10: no explicit first-party CI pipeline example (e.g., GitHub Actions template) or independent hands-on report of running Steel inside an actual CI system.

   • [claimed-docs] “The Steel CLI lets you run full browser workflows from the terminal, end-to-end. You can start a browser session, navigate pages, click/fill…”
   • [github] “Pre-built Docker Image (combined API + UI)”
   • [github] “Uses Puppeteer and CDP for complete control over Chrome instances -- allowing you to connect using Puppeteer, Playwright, or Selenium.”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”
   • [probe] “official CLI documented at https://docs.steel.dev/overview/steel-cli”
   • [claimed-docs] “the Sessions API lets your agents spin up isolated browser instances on demand. Each session maintains its own state, cookies, and storage”

   proof ↗⚑ flag

   Smooth~partialprobed6/10

   Smooth is explicitly designed as an SDK/API and CLI for programmatic browser automation, with 'plug-and-play' 4-line-of-code task execution and custom tools/session workflows suited to unattended automation, and a documented CLI (smooth-gh-1, smooth-docs-1, smooth-probe-3). However there is no explicit documentation or example of running it inside a CI pipeline (e.g., GitHub Actions), headless flags, or exit-code/automation-specific guidance. Missing for 10: explicit CI/CD integration docs or examples, confirmation of non-interactive/headless auth flow for pipelines, independent confirmation of CI usage.

   • [claimed-docs] “Plug-and-play: Run a task in just 4 lines of code, making it easy to integrate into your workflow.”
   • [github] “The Smooth CLI is a browser for AI agents, enabling tools like Claude Code to navigate the web quickly, cheaply, and reliably.”
   • [probe] “official CLI documented at https://docs.smooth.sh/cli/overview”
   • [claimed-docs] “Custom tools allow you to give Smooth any arbitrary function as a tool.”
   • [claimed-docs] “Session Workflow — Multi-step execution where you can orchestrate smaller tasks, navigate to URLs, and extract data within a persistent brow…”

   proof ↗⚑ flag

3. ai-native userConnect an agent via an official MCP server

   weight 3 · round to Steel
   Steel~partialclaimed5/10

   Steel's docs mention an integration with the Claude Agent SDK that 'exposes a cloud browser as in-process MCP tools,' showing some official MCP tool exposure for agents, but there's no evidence of a standalone, general-purpose official MCP server endpoint independent of this one SDK integration. missing for 10: a dedicated/standalone MCP server doc or endpoint usable by any agent framework, independent corroboration or hands-on proof of MCP connectivity.

   • [claimed-docs] “The Steel integration exposes a cloud browser as in-process MCP tools, so the SDK runs the agent loop and Steel handles the browser.”

   proof ↗⚑ flag

   Smooth—none0/10

   Smooth is a browser-automation SDK/CLI used by agents like Claude Code, but the evidence pack contains no mention of an official MCP server or MCP protocol integration anywhere in its docs, GitHub repo, or community discussion — only generic SDK/CLI methods and custom tool functions are documented.

   • [github] “The Smooth CLI is a browser for AI agents, enabling tools like Claude Code to navigate the web quickly, cheaply, and reliably.”
   • [claimed-docs] “Custom tools allow you to give Smooth any arbitrary function as a tool.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.smooth.sh/llms.txt # Smooth ## Docs - [Introduction](https://docs.smooth.sh/index.md): Welcome to…”
   • [probe] “official CLI documented at https://docs.smooth.sh/cli/overview”

   proof ↗⚑ flag

4. ai-native userUse an official CLI

   weight 2 · round to Steel
   Steel✓fullprobed8/10

   Steel ships a documented official CLI (steel-docs-8) that supports end-to-end browser workflows from the terminal, and probe evidence confirms real installation via setup.steel.dev installing 'steel CLI 0.4.4' into a fresh environment (steel-probe-rt-1), corroborating the docs. Missing for 10: independent third-party reviews of the CLI's UX/reliability beyond the vendor-run probe, and more detail on advanced CLI subcommands/scripting capabilities.

   • [claimed-docs] “The Steel CLI lets you run full browser workflows from the terminal, end-to-end. You can start a browser session, navigate pages, click/fill…”
   • [probe] “official CLI documented at https://docs.steel.dev/overview/steel-cli”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”

   proof ↗⚑ flag

   Smooth✓fullprobed7/10

   GitHub repo describes Smooth CLI explicitly as 'a browser for AI agents, enabling tools like Claude Code to navigate the web' and a docs probe confirms an official CLI overview page exists, showing a first-party CLI built for AI-agent workflows. Missing for 10: independent hands-on confirmation of the CLI's usage/reliability and more detailed CLI documentation content beyond the overview link.

   • [github] “The Smooth CLI is a browser for AI agents, enabling tools like Claude Code to navigate the web quickly, cheaply, and reliably.”
   • [probe] “official CLI documented at https://docs.smooth.sh/cli/overview”

   proof ↗⚑ flag

5. ai-native userDrive the product through a documented public API

   weight 3 · round to Steel
   Steel✓fullprobed9/10

   Steel publishes a full OpenAPI spec (steel-probe-2), documented Sessions API with SDKs, and a CLI, all confirmed hands-on by a runtime probe showing session creation, health checks, and SDK usage working end-to-end. This is strong first-party documentation plus independent verification of a working public API. Missing for 10: no third-party community deep-dive validating API completeness beyond the probe.

   • [probe] “PROBE openapi: HTTP 200 at https://docs.steel.dev/openapi.json — contains "openapi" key”
   • [claimed-docs] “the Sessions API lets your agents spin up isolated browser instances on demand. Each session maintains its own state, cookies, and storage”
   • [claimed-docs] “The Steel CLI lets you run full browser workflows from the terminal, end-to-end. You can start a browser session, navigate pages, click/fill…”
   • [probe] “official CLI documented at https://docs.steel.dev/overview/steel-cli”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”

   proof ↗⚑ flag

   Smooth~partialprobed6/10

   Smooth provides documented SDK/API methods (task execution, session workflows, structured outputs, custom tools, proxies) and an llms.txt docs index plus a CLI, showing a documented programmatic interface for AI-native use. However, no formal OpenAPI/REST spec was found (404s on all standard paths), and there is no independent corroboration of API robustness beyond vendor docs. missing for 10: a discoverable OpenAPI/REST spec, independent/hands-on verification of API completeness and stability.

   • [claimed-docs] “Plug-and-play: Run a task in just 4 lines of code, making it easy to integrate into your workflow.”
   • [claimed-docs] “Custom tools allow you to give Smooth any arbitrary function as a tool.”
   • [claimed-docs] “Session Workflow — Multi-step execution where you can orchestrate smaller tasks, navigate to URLs, and extract data within a persistent brow…”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.smooth.sh/llms.txt # Smooth ## Docs - [Introduction](https://docs.smooth.sh/index.md): Welcome to…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.smooth.sh/openapi.json, https://docs.smooth.sh/swagger.json, https://docs.smooth.sh/api…”
   • [probe] “official CLI documented at https://docs.smooth.sh/cli/overview”

   proof ↗⚑ flag

6. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round drawn
   Steel—none0/10

   No evidence anywhere in the pack of scoped or least-privilege API key/credential issuance (e.g., role-based keys, permission scopes, restricted tokens) for agents — only session isolation, proxies, and CLI features are documented, none of which address credential scoping.

   ⚑ flag

   Smooth—none0/10

   No evidence of scoped or least-privilege API credential issuance, API key scoping, or permission management for agents; docs cover task execution, sessions, proxies, and privacy features but nothing about credential scoping. missing for 10: scoped API key/token generation, permission/role controls, credential revocation or least-privilege access management.

   ⚑ flag

7. ai-native userBuild against official SDKs

   weight 2 · round to Steel
   Steel✓fullprobed8/10

   Steel ships an official steel-sdk npm package (verified working via probe: exports a Steel client class), a documented OpenAPI spec, and an official CLI for full browser workflows, all covered in first-party docs and confirmed by a hands-on runtime probe. missing for 10: explicit multi-language SDK coverage (e.g., Python/other languages) and independent community confirmation of SDK usage beyond the CLI/API.

   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”
   • [probe] “PROBE openapi: HTTP 200 at https://docs.steel.dev/openapi.json — contains "openapi" key”
   • [probe] “official CLI documented at https://docs.steel.dev/overview/steel-cli”
   • [claimed-docs] “The Steel CLI lets you run full browser workflows from the terminal, end-to-end. You can start a browser session, navigate pages, click/fill…”
   • [claimed-docs] “The Steel integration exposes a cloud browser as in-process MCP tools, so the SDK runs the agent loop and Steel handles the browser.”

   proof ↗⚑ flag

   Smooth~partialprobed6/10

   Smooth documents SDK-style integration (4-line task execution, custom tools, structured outputs, session workflows) and a CLI positioned as "a browser for AI agents" usable with tools like Claude Code, indicating official first-party SDK/CLI support for AI-native workflows. However, there is no OpenAPI spec, no evidence of multi-language SDKs, and no independent/hands-on confirmation of SDK reliability beyond docs and a GitHub repo. missing for 10: OpenAPI/API spec availability, multi-language SDK coverage, independent developer corroboration of SDK usage/quality.

   • [claimed-docs] “Plug-and-play: Run a task in just 4 lines of code, making it easy to integrate into your workflow.”
   • [claimed-docs] “Custom tools allow you to give Smooth any arbitrary function as a tool.”
   • [claimed-docs] “Session Workflow — Multi-step execution where you can orchestrate smaller tasks, navigate to URLs, and extract data within a persistent brow…”
   • [github] “The Smooth CLI is a browser for AI agents, enabling tools like Claude Code to navigate the web quickly, cheaply, and reliably.”
   • [probe] “official CLI documented at https://docs.smooth.sh/cli/overview”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.smooth.sh/openapi.json, https://docs.smooth.sh/swagger.json, https://docs.smooth.sh/api…”

   proof ↗⚑ flag

8. ai-native userSubscribe to events via webhooks

   weight 2 · round drawn
   Steel—none0/10

   No evidence anywhere in the pack mentions webhooks or event subscriptions; Steel's documented surfaces are Sessions API, CLI, MCP tools, and agent traces, none of which describe a webhook/event notification mechanism.

   ⚑ flag

   Smooth—none0/10

   No evidence pack item mentions webhooks or event subscriptions; Smooth's documented features (live_url, structured outputs, custom tools, sessions) do not include a webhook/event notification mechanism.

   ⚑ flag

Agentic features

1. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Smooth
   Steel—none0/10

   Steel provides on-demand browser sessions, CLI, SDK, and agent-trace tooling for agents to control browsers, but nothing in the evidence describes a scheduling/trigger mechanism or persistent background job runner that lets a user set up automations to run autonomously without invocation — sessions are explicitly spun up 'on demand' by an agent/script, not scheduled by Steel itself.

   • [claimed-docs] “the Sessions API lets your agents spin up isolated browser instances on demand. Each session maintains its own state, cookies, and storage”
   • [claimed-docs] “The Steel CLI lets you run full browser workflows from the terminal, end-to-end. You can start a browser session, navigate pages, click/fill…”
   • [claimed-docs] “Steel generates a session ID for you, but `create` also accepts one. Pass your own UUID when the ID has to exist before the browser does”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”

   proof ↗⚑ flag

   Smooth~partialclaimed5/10

   Smooth lets users kick off browser-agent tasks programmatically that run autonomously (navigating, extracting data, solving CAPTCHAs) and provides a live_url to monitor progress, which supports hands-off execution once started. However there's no evidence of scheduling, triggers, webhooks, or persistent 'set it and forget it' background jobs that run without an explicit API call — the model shown is synchronous task invocation, not autonomous background automation setup. Missing for 10: scheduling/cron or event-trigger support, evidence of long-running unattended jobs, and independent confirmation of background execution beyond a single task call.

   • [claimed-docs] “Plug-and-play: Run a task in just 4 lines of code, making it easy to integrate into your workflow.”
   • [claimed-docs] “When running a task, you will receive a `live_url`, which can be used to view the agent actions live.”
   • [claimed-docs] “Session Workflow — Multi-step execution where you can orchestrate smaller tasks, navigate to URLs, and extract data within a persistent brow…”
   • [claimed-docs] “Auto-CAPTCHA solvers: Bypass CAPTCHA challenges automatically, allowing for uninterrupted task execution.”

   proof ↗⚑ flag

2. ai-native userOperate the product with natural-language commands

   weight 2 · round to Smooth
   Steel~partialclaimed5/10

   Steel exposes its browser control as MCP tools within agent SDKs (e.g., Claude Agent SDK) so an AI agent can translate natural-language user requests into Steel API calls, and the CLI/SDK/API allow full programmatic control — but there's no evidence of a native natural-language interface to Steel itself (e.g., a chat command layer); control still requires structured API/CLI calls or a separate agent framework. Missing for 10: a first-party NL command interface or chat-driven control surface, and independent confirmation that NL-driven agent use works end-to-end in production.

   • [claimed-docs] “The Steel integration exposes a cloud browser as in-process MCP tools, so the SDK runs the agent loop and Steel handles the browser.”
   • [claimed-docs] “The Steel CLI lets you run full browser workflows from the terminal, end-to-end. You can start a browser session, navigate pages, click/fill…”
   • [github] “Uses Puppeteer and CDP for complete control over Chrome instances -- allowing you to connect using Puppeteer, Playwright, or Selenium.”

   proof ↗⚑ flag

   Smooth~partialclaimed6/10

   Smooth's core interaction model is task-based: you give it a task description that an agent executes in a browser (session workflow, extract, navigate), and it is explicitly positioned as "a browser for AI agents" usable by tools like Claude Code, implying natural-language task instructions. However, no evidence shows an explicit example of a natural-language prompt/command syntax or confirms this is exposed to end-users beyond agent-to-agent orchestration. Missing for 10: explicit example of a natural-language task string/command, confirmation of human-facing NL command interface, independent corroboration of NL usability.

   • [claimed-docs] “Plug-and-play: Run a task in just 4 lines of code, making it easy to integrate into your workflow.”
   • [claimed-docs] “Session Workflow — Multi-step execution where you can orchestrate smaller tasks, navigate to URLs, and extract data within a persistent brow…”
   • [github] “The Smooth CLI is a browser for AI agents, enabling tools like Claude Code to navigate the web quickly, cheaply, and reliably.”
   • [claimed-docs] “Extract structured data from the current page by providing a schema.”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round to Steel
   Steel~partialprobed6/10

   Steel publishes a full OpenAPI spec (docs.steel.dev/openapi.json) and an llms.txt, and a community commenter independently praised the docs/API reference quality, suggesting an interactive, well-documented API surface. However, there's no explicit evidence of an in-browser 'try it' / runnable-example console distinct from static docs. Missing for 10: direct confirmation of an interactive try-it console with live runnable code snippets, and independent hands-on verification of that specific feature.

   • [probe] “PROBE openapi: HTTP 200 at https://docs.steel.dev/openapi.json — contains "openapi" key”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.steel.dev/llms.txt # Steel Documentation > Steel is the open-source browser API for AI agents — ma…”
   • [community] “beautiful docs + api ref! what are you using? (cool that you're doing open-source browserbase also, excited to check this out)”

   proof ↗⚑ flag

   Smooth—none0/10

   The evidence pack shows only static markdown docs (llms.txt, feature pages) and explicitly shows the openapi.json/swagger endpoints returning 404, indicating no interactive API reference or runnable-example playground exists. No mention of a Swagger UI, Postman collection, or in-browser code runner is present anywhere in docs or community discussion.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.smooth.sh/llms.txt # Smooth ## Docs - [Introduction](https://docs.smooth.sh/index.md): Welcome to…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.smooth.sh/openapi.json, https://docs.smooth.sh/swagger.json, https://docs.smooth.sh/api…”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round to Steel
   Steel✓fullprobed9/10

   A probe confirms a live, valid OpenAPI JSON spec served at docs.steel.dev/openapi.json (HTTP 200 with 'openapi' key), directly satisfying the machine-readable spec requirement, complemented by an llms.txt index for discoverability. missing for 10: independent third-party corroboration beyond the automated probe.

   • [probe] “PROBE openapi: HTTP 200 at https://docs.steel.dev/openapi.json — contains "openapi" key”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.steel.dev/llms.txt # Steel Documentation > Steel is the open-source browser API for AI agents — ma…”

   proof ↗⚑ flag

   Smooth—none0/10

   A direct probe for OpenAPI/Swagger spec files at all standard paths returned 404, and no docs page references a downloadable machine-readable API spec; only an llms.txt (docs index) is available.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.smooth.sh/openapi.json, https://docs.smooth.sh/swagger.json, https://docs.smooth.sh/api…”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.smooth.sh/llms.txt # Smooth ## Docs - [Introduction](https://docs.smooth.sh/index.md): Welcome to…”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round to Steel
   Steel✓fullprobed7/10

   Steel's core Sessions API spins up isolated, on-demand browser instances each with their own state, cookies, and storage, and this isolation was independently verified via a self-hosted runtime probe that created a live, separate browser session from a throwaway Docker instance — effectively a sandbox with no shared production state. Missing for 10: explicit documentation framing sessions as a 'test vs production' environment, and no first-party guidance on staging/production data separation policies beyond session isolation.

   • [claimed-docs] “the Sessions API lets your agents spin up isolated browser instances on demand. Each session maintains its own state, cookies, and storage”
   • [claimed-docs] “Steel generates a session ID for you, but `create` also accepts one. Pass your own UUID when the ID has to exist before the browser does”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”
   • [github] “Pre-built Docker Image (combined API + UI)”

   proof ↗⚑ flag

   Smooth—none0/10

   Smooth's docs describe browser-automation features (live sessions, proxies, persistent auth, structured outputs) but nowhere mention a sandbox/staging mode or any mechanism to isolate test runs from production data or accounts. Community feedback even flags unresolved concerns about data handling and security, but no concrete sandbox capability is described or corroborated.

   ⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Steel—none0/10

   No evidence pack item mentions API versioning scheme or a documented deprecation policy; only an OpenAPI spec and general docs are shown, neither of which discuss version lifecycle or deprecation commitments.

   ⚑ flag

   Smooth—none0/10

   No evidence of API versioning scheme or a documented deprecation policy; OpenAPI spec probe returned 404s and docs show no changelog/versioning references. missing for 10: versioned API scheme, deprecation policy documentation, changelog/migration guides.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.smooth.sh/openapi.json, https://docs.smooth.sh/swagger.json, https://docs.smooth.sh/api…”

   proof ↗⚑ flag

Compat

1. developerConnect my existing Playwright, Puppeteer, or CDP automation code to the product's browsers instead of rewriting it

   weight 2 · round to Steel
   Steel✓fullprobed9/10

   Steel explicitly supports connecting existing Puppeteer, Playwright, or Selenium code via CDP to control its browser instances, and a runtime probe confirms live sessions expose websocket/debugger URLs consistent with CDP connectivity. Docs and SDK further corroborate first-class session management compatible with standard automation libraries. Missing for 10: independent third-party hands-on confirmation specifically of a rewritten Playwright/Puppeteer script running unmodified against Steel.

   • [github] “Uses Puppeteer and CDP for complete control over Chrome instances -- allowing you to connect using Puppeteer, Playwright, or Selenium.”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”
   • [claimed-docs] “the Sessions API lets your agents spin up isolated browser instances on demand. Each session maintains its own state, cookies, and storage”

   proof ↗⚑ flag

   Smooth—none0/10

   Smooth's docs describe its own SDK/task API (structured outputs, sessions, custom tools) but there is no mention of a CDP endpoint, Playwright/Puppeteer connect() compatibility, or any way to point existing automation code at Smooth's browsers; one commenter even notes 'agent-browser helped a lot over playwright but doesn't completely close the gap,' underscoring the absence of such interoperability. Missing for 10: any CDP/WebSocket endpoint, official Playwright/Puppeteer connect examples, or documented browser-endpoint compatibility.

   • [claimed-docs] “Session Workflow — Multi-step execution where you can orchestrate smaller tasks, navigate to URLs, and extract data within a persistent brow…”
   • [claimed-docs] “Set to `"self"` to create a P2P tunnel through your machine, routing traffic via your IP and enabling access to localhost.”
   • [community] “agent-browser helped a lot over playwright but doesn't completely close the gap.”

   proof ↗⚑ flag

Credentials

1. automation-engineerStore credentials in a vault and have the agent complete logins including TOTP/2FA challenges without exposing secrets to the model

   weight 2 · round drawn
   Steel—none0/10

   Steel's docs show session-level auth persistence (reusing cookies/storage across sessions) but no evidence of a credentials vault, secret injection without model exposure, or TOTP/2FA challenge automation — the core asks of this story are unaddressed.

   • [claimed-docs] “This is particularly useful for maintaining authenticated states across multiple sessions, helping your AI agents access protected resources…”

   proof ↗⚑ flag

   Smooth—none0/10

   The docs describe persistent sessions (log in once, reuse authentication) but there is no mention of a credential vault, secret injection to avoid model exposure, or TOTP/2FA handling anywhere in the evidence pack. This axis clearly applies to a browser-automation agent product, but no capability matching the story is documented.

   • [claimed-docs] “Log in once, then reuse that authentication for future tasks.”

   proof ↗⚑ flag

Profiles

1. developerPersist logged-in browser state in reusable profiles so agents skip the login wall on every subsequent run

   weight 3 · round to Steel
   Steel✓fullprobed8/10

   Steel docs explicitly document reusing auth context across sessions to let agents skip repeated logins, backed by session isolation, custom session IDs, and a real API/CLI/SDK confirmed via runtime probe. Missing for 10: independent third-party hands-on confirmation of the reuse-auth-context feature specifically (only vendor docs cite it) and no explicit profile-export/import UX details beyond the docs description.

   • [claimed-docs] “This is particularly useful for maintaining authenticated states across multiple sessions, helping your AI agents access protected resources…”
   • [claimed-docs] “the Sessions API lets your agents spin up isolated browser instances on demand. Each session maintains its own state, cookies, and storage”
   • [claimed-docs] “Steel generates a session ID for you, but `create` also accepts one. Pass your own UUID when the ID has to exist before the browser does”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”

   proof ↗⚑ flag

   Smooth✓fullclaimed7/10

   Smooth's docs explicitly describe a persistent-sessions feature ('Log in once, then reuse that authentication for future tasks') and a session workflow that maintains a persistent browser session across multi-step tasks, directly matching the story. However, there is no independent/hands-on corroboration of this specific feature working reliably, and no detail on profile management (multiple reusable profiles, storage/export). Missing for 10: independent verification of session persistence in practice, documentation on managing multiple reusable profiles.

   • [claimed-docs] “Log in once, then reuse that authentication for future tasks.”
   • [claimed-docs] “Session Workflow — Multi-step execution where you can orchestrate smaller tasks, navigate to URLs, and extract data within a persistent brow…”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Steel
   Steel~partialclaimed4/10

   Steel's Sessions API allows spinning up isolated browser sessions on demand and reusing auth context across multiple sessions, which implies you could programmatically launch many sessions for parallel/bulk tasks, but there is no explicit documentation of a batch/bulk API, concurrency limits, or guidance for orchestrating many items at once. missing for 10: explicit bulk/batch API or documented pattern for running many operations concurrently, concurrency/rate limits, and independent evidence of large-scale parallel session usage.

   • [claimed-docs] “the Sessions API lets your agents spin up isolated browser instances on demand. Each session maintains its own state, cookies, and storage”
   • [claimed-docs] “This is particularly useful for maintaining authenticated states across multiple sessions, helping your AI agents access protected resources…”
   • [claimed-docs] “Steel generates a session ID for you, but `create` also accepts one. Pass your own UUID when the ID has to exist before the browser does”

   proof ↗⚑ flag

   Smooth—none0/10

   Smooth's docs describe single-task execution, session workflows, and structured extraction, but nothing about running/orchestrating bulk operations across many items (e.g., batch task queues, parallel task fan-out) is documented or mentioned by users.

   • [claimed-docs] “Plug-and-play: Run a task in just 4 lines of code, making it easy to integrate into your workflow.”
   • [claimed-docs] “Session Workflow — Multi-step execution where you can orchestrate smaller tasks, navigate to URLs, and extract data within a persistent brow…”
   • [claimed-docs] “Extract structured data from the current page by providing a schema.”

   proof ↗⚑ flag

2. ai-native userSchedule recurring jobs or workflows

   weight 2 · round drawn
   Steel—none0/10

   Steel is a browser automation/session API for AI agents (sessions, stealth, proxies, CLI, MCP tools), but nothing in the evidence describes a scheduler, cron-like trigger, or recurring job/workflow orchestration feature; usage is invoked on-demand per session, not on a schedule.

   ⚑ flag

   Smooth—none0/10

   No evidence of scheduling, cron-like triggers, or recurring workflow orchestration; Smooth is documented as a task-execution/browser-automation tool (session workflows, extraction, structured output) with no mention of recurring/scheduled jobs. Missing for 10: any scheduling API, cron/trigger mechanism, or recurring workflow docs.

   • [claimed-docs] “Session Workflow — Multi-step execution where you can orchestrate smaller tasks, navigate to URLs, and extract data within a persistent brow…”
   • [claimed-docs] “Plug-and-play: Run a task in just 4 lines of code, making it easy to integrate into your workflow.”

   proof ↗⚑ flag

Local

1. developerRun the agent against a local browser on my own machine for development, without any cloud account

   weight 2 · round to Steel
   Steel✓fullprobed9/10

   Steel Browser is open-source and can be self-hosted via Docker with no cloud account, confirmed by a runtime probe showing a local Docker container booting the browser API, creating live sessions, and working with the official CLI and SDK entirely locally. missing for 10: independent third-party (non-vendor) confirmation of long-term local dev workflow beyond the single recorded probe.

   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”
   • [github] “Pre-built Docker Image (combined API + UI)”
   • [github] “Uses Puppeteer and CDP for complete control over Chrome instances -- allowing you to connect using Puppeteer, Playwright, or Selenium.”
   • [probe] “official CLI documented at https://docs.steel.dev/overview/steel-cli”

   proof ↗⚑ flag

   Smooth—none0/10

   Smooth is documented as a cloud-hosted browser agent service (task execution via live_url, proxies, zero-data-retention as an 'enterprise' add-on), with no docs describing a local-browser/offline mode; a P2P tunnel feature only lets the cloud agent reach your localhost, not run without an account. Community feedback explicitly asks for self-hosting ('Make it self-hostable, the conversation can change'), confirming no local/no-account mode exists.

   • [claimed-docs] “Zero Data Retention is an enterprise feature that provides enhanced data privacy by allowing you to delete all data associated with complete…”
   • [claimed-docs] “Set to `"self"` to create a P2P tunnel through your machine, routing traffic via your IP and enabling access to localhost.”
   • [community] “I'm unwilling to send my data to a 3rd party that is so new on the scene... Make it self-hostable, the conversation can change”
   • [community] “Way too expensive, I'll wait for a free/open source browser optimized to be used by agents.”

   proof ↗⚑ flag

Frameworks

1. developerPlug the browser layer into agent frameworks (Claude Agent SDK, Vercel AI SDK, LangChain, CrewAI) through documented adapters

   weight 2 · round to Steel
   Steel~partialclaimed5/10

   Steel documents a concrete integration with the Claude Agent SDK exposing its browser as in-process MCP tools (steel-docs-9), showing at least one first-party framework adapter exists. However, the evidence pack contains no documented adapters or integration guides for Vercel AI SDK, LangChain, or CrewAI, so the broader multi-framework claim is only partially substantiated. missing for 10: documented adapters for Vercel AI SDK, LangChain, and CrewAI, plus independent corroboration of any of these integrations working in practice.

   • [claimed-docs] “The Steel integration exposes a cloud browser as in-process MCP tools, so the SDK runs the agent loop and Steel handles the browser.”

   proof ↗⚑ flag

   Smooth—none0/10

   Evidence shows Smooth positions itself as a browser tool usable by agents like Claude Code, but there is no documentation of adapters for Claude Agent SDK, Vercel AI SDK, LangChain, or CrewAI specifically. Missing for 10: any documented integration guide or adapter package for these named frameworks.

   • [github] “The Smooth CLI is a browser for AI agents, enabling tools like Claude Code to navigate the web quickly, cheaply, and reliably.”
   • [claimed-docs] “Custom tools allow you to give Smooth any arbitrary function as a tool.”

   proof ↗⚑ flag

Models

1. developerBring my own LLM provider — the framework is model-agnostic rather than locked to one vendor's models

   weight 2 · round drawn
   Steel—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   Smooth—none0/10

   Smooth's docs describe it as using its own internal 'small and efficient AI models' for browser-agent tasks, and none of the documentation, CLI, or API references mention any mechanism for developers to configure or swap in their own LLM provider (e.g., OpenAI, Anthropic, custom model endpoints). This suggests the opposite of model-agnosticism — the models are an implementation detail of the service, not a pluggable choice.

   • [claimed-docs] “Smooth uses small and efficient AI models, making it 7x more affordable than browser-use.”
   • [github] “The Smooth CLI is a browser for AI agents, enabling tools like Claude Code to navigate the web quickly, cheaply, and reliably.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.smooth.sh/llms.txt # Smooth ## Docs - [Introduction](https://docs.smooth.sh/index.md): Welcome to…”

   proof ↗⚑ flag

Tasks

1. ai agentSubmit a browser task over a hosted HTTP API and receive the result by polling or webhook, without managing any browser myself

   weight 2 · round to Smooth
   Steel—none0/10

   Steel's docs describe a Sessions API that hands agents a raw, controllable browser (via CDP/Puppeteer/Playwright) plus a CLI for scripted step-by-step actions, but there is no evidence of a higher-level 'submit a task, poll or get a webhook for the result' abstraction — the agent still must drive the browser session itself rather than delegate a task and retrieve a finished output.

   • [claimed-docs] “the Sessions API lets your agents spin up isolated browser instances on demand. Each session maintains its own state, cookies, and storage”
   • [claimed-docs] “The Steel CLI lets you run full browser workflows from the terminal, end-to-end. You can start a browser session, navigate pages, click/fill…”
   • [github] “Uses Puppeteer and CDP for complete control over Chrome instances -- allowing you to connect using Puppeteer, Playwright, or Selenium.”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”

   proof ↗⚑ flag

   Smooth~partialprobed5/10

   Docs confirm a hosted task-submission model (4-line integration, live_url for tracking, persistent sessions, structured outputs) consistent with an agent submitting tasks without managing a browser, but no evidence pack item explicitly documents a polling endpoint or webhook delivery mechanism, and probes found no public OpenAPI/REST spec. missing for 10: explicit polling endpoint docs, explicit webhook/callback docs, confirmed REST API schema (openapi probe 404s).

   • [claimed-docs] “Plug-and-play: Run a task in just 4 lines of code, making it easy to integrate into your workflow.”
   • [claimed-docs] “When running a task, you will receive a `live_url`, which can be used to view the agent actions live.”
   • [claimed-docs] “Session Workflow — Multi-step execution where you can orchestrate smaller tasks, navigate to URLs, and extract data within a persistent brow…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.smooth.sh/openapi.json, https://docs.smooth.sh/swagger.json, https://docs.smooth.sh/api…”
   • [probe] “official CLI documented at https://docs.smooth.sh/cli/overview”

   proof ↗⚑ flag

2. developerHand the product a natural-language goal and it completes a multi-step web task end to end — navigating, filling forms, and clicking through flows

   weight 3 · round to Smooth
   Steel~partialprobed5/10

   Steel provides the browser primitives (sessions, navigate/click/fill/extract via CLI or API, CDP control) that a multi-step web task requires, and its CLI/MCP integrations let external agent frameworks drive those actions from natural-language goals. However, Steel's own docs state the agent loop and NL reasoning are handled by the paired SDK (e.g., Claude Agent SDK), not by Steel itself — Steel 'handles the browser' while the SDK runs the reasoning loop, so Steel alone does not accept a raw NL goal and autonomously plan/execute it end to end. missing for 10: evidence of Steel natively parsing/planning from a raw NL instruction without an external agent/LLM orchestrating the steps, and independent hands-on proof of a full NL-driven multi-step flow completed unattended.

   • [claimed-docs] “The Steel CLI lets you run full browser workflows from the terminal, end-to-end. You can start a browser session, navigate pages, click/fill…”
   • [claimed-docs] “The Steel integration exposes a cloud browser as in-process MCP tools, so the SDK runs the agent loop and Steel handles the browser.”
   • [github] “Uses Puppeteer and CDP for complete control over Chrome instances -- allowing you to connect using Puppeteer, Playwright, or Selenium.”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”

   proof ↗⚑ flag

   Smooth✓fullcommunity7/10

   Docs describe exactly this capability: multi-step 'Session Workflow' that navigates URLs, orchestrates sub-tasks, and extracts data, plus a live_url to watch the agent act, and a community commenter confirms 'I just wrote a complex prompt and it did a good job.' This matches the natural-language, end-to-end web task story well. Missing for 10: independently reproducible benchmarks/evals (a commenter explicitly asks for third-party reproducible comparisons and gets no clear answer), and broader hands-on validation beyond a single anecdote.

   • [claimed-docs] “Session Workflow — Multi-step execution where you can orchestrate smaller tasks, navigate to URLs, and extract data within a persistent brow…”
   • [claimed-docs] “Extract structured data from the current page by providing a schema.”
   • [claimed-docs] “When running a task, you will receive a `live_url`, which can be used to view the agent actions live.”
   • [community] “I just wrote a complex prompt and it did a good job. How do you do evals or testing of your project?”
   • [community] “are your evals / comparisons publicly/3rd party reproducible? If it's 'trust me, I did a fair comparison', that's not going to fly today.”

   proof ↗⚑ flag

Workflows

1. automation-engineerCompose repeatable multi-step workflows with loops, conditionals, and parameters instead of one-shot prompts

   weight 2 · round to Smooth
   Steel—none0/10

   Steel's docs describe browser session management, stealth, CLI scripting of sequential steps (create session → navigate → click/fill → extract → stop), and MCP tool exposure, but nothing in the evidence pack shows a workflow-composition layer with loops, conditionals, or parameterized branching — the CLI and SDK are linear step sequences, not a control-flow DSL. missing for 10: any documented loop/conditional constructs, parameterized workflow templates, or reusable multi-branch automation definitions.

   • [claimed-docs] “The Steel CLI lets you run full browser workflows from the terminal, end-to-end. You can start a browser session, navigate pages, click/fill…”
   • [claimed-docs] “Steel generates a session ID for you, but `create` also accepts one. Pass your own UUID when the ID has to exist before the browser does”
   • [github] “Uses Puppeteer and CDP for complete control over Chrome instances -- allowing you to connect using Puppeteer, Playwright, or Selenium.”

   proof ↗⚑ flag

   Smooth~partialclaimed4/10

   Smooth documents a 'Session Workflow' method for multi-step execution—orchestrating smaller tasks, navigating URLs, and extracting data within a persistent browser session—plus structured outputs and custom tools that let developers build deterministic logic around agent calls. However, there is no explicit documentation of native loop/conditional constructs or parameterized workflow templates; any control flow would rely on the surrounding SDK code rather than a built-in workflow engine. Missing for 10: explicit loop/conditional primitives, parameterization/templating of workflows, and independent evidence of repeatable multi-step automations beyond simple session chaining.

   • [claimed-docs] “Session Workflow — Multi-step execution where you can orchestrate smaller tasks, navigate to URLs, and extract data within a persistent brow…”
   • [claimed-docs] “Structured outputs allow you to write deterministic code based on the agent's output. To activate structured outputs, set the `response_mode…”
   • [claimed-docs] “Custom tools allow you to give Smooth any arbitrary function as a tool.”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Steel
   Steel~partialprobed6/10

   Steel is fundamentally API-first: the Sessions API, CAPTCHA/proxy/auth-reuse, human-in-the-loop debug URLs, agent traces, and even the CLI are all documented as thin wrappers over the same REST/OpenAPI surface (steel-docs-1..10, steel-probe-2, steel-probe-3), and the runtime probe confirms the API alone (no UI) can create/list live sessions end-to-end. However, there's no explicit evidence enumerating the dashboard/UI feature set and confirming 1:1 parity with the API — it's inferred rather than directly stated. Missing for 10: an explicit doc/comparison confirming every UI-only feature (e.g. dashboard analytics, trace viewer) is also exposed via API, and independent confirmation of full parity.

   • [claimed-docs] “the Sessions API lets your agents spin up isolated browser instances on demand. Each session maintains its own state, cookies, and storage”
   • [claimed-docs] “One flag covers detection, solving, and verification for the whole session”
   • [claimed-docs] “This is particularly useful for maintaining authenticated states across multiple sessions, helping your AI agents access protected resources…”
   • [claimed-docs] “Steel's debug URL feature allows you to implement human-in-the-loop workflows where users can directly interact with and control browser ses…”
   • [claimed-docs] “It turns the run into a timeline of agent activity, so you can see what happened without scrubbing through the whole recording.”
   • [claimed-docs] “The Steel CLI lets you run full browser workflows from the terminal, end-to-end. You can start a browser session, navigate pages, click/fill…”
   • [probe] “PROBE openapi: HTTP 200 at https://docs.steel.dev/openapi.json — contains "openapi" key”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”

   proof ↗⚑ flag

   Smooth—none0/10

   The evidence pack shows Smooth as an API/CLI/SDK-first browser-automation tool with docs for tasks, sessions, proxies, custom tools, and a live_url for viewing agent actions, but there is no mention of a separate web dashboard/UI or any comparison of UI-only vs API-only capabilities. Without evidence of what a UI offers (or that all UI features are mirrored in the API), the parity claim can't be substantiated. Missing for 10: any documented web UI/dashboard, and an explicit statement or demonstration that all UI actions are also achievable via API.

   • [claimed-docs] “Plug-and-play: Run a task in just 4 lines of code, making it easy to integrate into your workflow.”
   • [claimed-docs] “Session Workflow — Multi-step execution where you can orchestrate smaller tasks, navigate to URLs, and extract data within a persistent brow…”
   • [probe] “official CLI documented at https://docs.smooth.sh/cli/overview”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round to Steel
   Steel~partialprobed5/10

   Steel's agent-traces feature explicitly supports exporting run data as markdown, JSON, or a ZIP with markdown+screenshots (steel-docs-7), and the product itself is open-source and self-hostable (steel-gh-4, steel-probe-rt-1), meaning users are never locked into a proprietary cloud and can run/keep everything themselves. However, there's no documented comprehensive 'export all my data' capability covering sessions, auth contexts, or account-level data beyond traces. Missing for 10: a documented full-account data export/portability feature covering sessions, auth states, and configs, not just trace recordings.

   • [claimed-docs] “Copy the run as markdown, download JSON, or grab a ZIP with markdown plus screenshots.”
   • [github] “Pre-built Docker Image (combined API + UI)”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”

   proof ↗⚑ flag

   Smooth—none0/10

   Smooth's docs mention Zero Data Retention (deletion of task data) but there is no evidence of a bulk data export feature or open-format export for users to take their data and leave — the closest related item is deletion, not portability. missing for 10: any documented export mechanism, open format specification, or user data portability tooling.

   • [claimed-docs] “Zero Data Retention is an enterprise feature that provides enhanced data privacy by allowing you to delete all data associated with complete…”

   proof ↗⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round to Steel
   Steel~partialprobed6/10

   The product's llms.txt explicitly states 'Steel is the open-source browser API for AI agents,' there's a public GitHub repo (steel-dev/steel-browser) with feature docs, and a runtime probe confirms the self-hostable OSS image can be pulled and run via Docker — all consistent with source availability. However, no evidence pack item names or shows a specific OSS license (e.g., MIT/AGPL) or a LICENSE file, so the exact openness terms are unconfirmed. Missing for 10: explicit license file/type citation, independent confirmation of license terms.

   • [github] “Uses Puppeteer and CDP for complete control over Chrome instances -- allowing you to connect using Puppeteer, Playwright, or Selenium.”
   • [github] “Pre-built Docker Image (combined API + UI)”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.steel.dev/llms.txt # Steel Documentation > Steel is the open-source browser API for AI agents — ma…”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”
   • [community] “beautiful docs + api ref! what are you using? (cool that you're doing open-source browserbase also, excited to check this out)”

   proof ↗⚑ flag

   Smooth—none0/10

   Smooth ships a GitHub repo for its SDK/CLI, but there is no evidence of an open-source license for the core product, and community comments explicitly request self-hosting/open-source alternatives ('Make it self-hostable, the conversation can change', 'I'll wait for a free/open source browser'), implying the core service is closed.

   • [github] “The Smooth CLI is a browser for AI agents, enabling tools like Claude Code to navigate the web quickly, cheaply, and reliably.”
   • [community] “I'm unwilling to send my data to a 3rd party that is so new on the scene... Make it self-hostable, the conversation can change”
   • [community] “Way too expensive, I'll wait for a free/open source browser optimized to be used by agents.”

   proof ↗⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round to Steel
   Steel✓fullprobed10/10

   Steel is explicitly open-source with a pre-built Docker image (combined API + UI), and a runtime probe confirms a full keyless self-host roundtrip: docker-running the OSS image, health check succeeding, and creating/listing live browser sessions, corroborating vendor docs and GitHub claims.

   • [github] “Pre-built Docker Image (combined API + UI)”
   • [github] “Uses Puppeteer and CDP for complete control over Chrome instances -- allowing you to connect using Puppeteer, Playwright, or Selenium.”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.steel.dev/llms.txt # Steel Documentation > Steel is the open-source browser API for AI agents — ma…”

   proof ↗⚑ flag

   Smooth—none0/10

   Smooth is offered only as a hosted cloud API/SaaS with no documented self-host option, and community feedback explicitly requests self-hosting as a missing capability ('Make it self-hostable, the conversation can change').

   • [community] “I'm unwilling to send my data to a 3rd party that is so new on the scene... Make it self-hostable, the conversation can change”
   • [community] “My first question was whether I could use this for sensitive tasks, given that it's not running on our machines. And after poking around for…”

   proof ↗⚑ flag

Pricing

1. developerSee transparent per-task or per-browser-hour pricing and documented rate/concurrency limits before committing

   weight 2 · round drawn
   Steel—none0/10

   No first-party documentation in the evidence pack lays out per-task or per-browser-hour pricing or concurrency/rate limits; the only pricing-related evidence is a community report of inconsistency between the pricing page and docs pricing ($59 vs $99), which itself signals the opposite of transparent, dependable pricing rather than confirming it.

   • [community] “Looking interesting, will definitely give it a go. Btw, there is inconsistency between pricing page and pricing on docs. Pricing page for de…”

   proof ↗⚑ flag

   Smooth—none0/10

   No evidence of documented per-task/per-browser-hour pricing tiers or rate/concurrency limits; only a vague claim of being '7x more affordable' with no actual pricing page or limits documented, and community comments call it 'too expensive' without citing specifics.

   • [claimed-docs] “Smooth uses small and efficient AI models, making it 7x more affordable than browser-use.”
   • [community] “Way too expensive, I'll wait for a free/open source browser optimized to be used by agents.”
   • [community] “I'm paying a fixed amount on Claude and other agents, so 'more tokens' is 'free' for me. There's a lot of niche tools out there but I think …”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round to Steel
   Steel~partialprobed4/10

   Steel offers a self-hostable open-source Docker image (steel-gh-4, steel-probe-rt-1) which lets users control where their data physically resides by hosting it themselves, but there is no explicit region/residency selection feature documented for the managed cloud offering. missing for 10: explicit region-selection UI/API for the managed cloud service, documentation on data residency guarantees or compliance certifications (e.g., GDPR/SOC2 region controls).

   • [github] “Pre-built Docker Image (combined API + UI)”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”

   proof ↗⚑ flag

   Smooth—none0/10

   No evidence of data residency/region selection options; only Zero Data Retention (deletion) is mentioned, which is a different capability. Missing for 10: any mention of region choice, data center locations, or residency controls.

   • [claimed-docs] “Zero Data Retention is an enterprise feature that provides enhanced data privacy by allowing you to delete all data associated with complete…”

   proof ↗⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round to Smooth
   Steel—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   Smooth~partialcommunity4/10

   Smooth documents a 'Zero Data Retention' enterprise feature that lets customers delete all data tied to completed tasks, which is adjacent to preventing data reuse, but there is no explicit statement that data is excluded from model training, and this feature is gated to enterprise tier. Community feedback also notes an absence of any detailed security/privacy documentation despite marketing claims of 'enterprise-grade security', raising trust concerns without disputing the ZDR feature itself. missing for 10: explicit AI-training opt-out policy, default (non-enterprise) privacy guarantees, independent verification of data handling.

   • [claimed-docs] “Zero Data Retention is an enterprise feature that provides enhanced data privacy by allowing you to delete all data associated with complete…”
   • [community] “My first question was whether I could use this for sensitive tasks, given that it's not running on our machines. And after poking around for…”

   proof ↗⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round to Smooth
   Steel~partialprobed4/10

   Steel is open-source and self-hostable (steel-gh-4, steel-probe-rt-1), which gives users full control over where session data lives and how long it's retained, and session lifecycle docs show sessions can be created/stopped with custom IDs (steel-docs-10). However, there is no explicit documentation of a retention policy, data-deletion API, or GDPR-style controls for the managed cloud offering. Missing for 10: documented retention/deletion controls or policy for the hosted cloud service, explicit data-purge API, independent confirmation of retention behavior.

   • [github] “Pre-built Docker Image (combined API + UI)”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”
   • [claimed-docs] “Steel generates a session ID for you, but `create` also accepts one. Pass your own UUID when the ID has to exist before the browser does”

   proof ↗⚑ flag

   Smooth~partialcommunity5/10

   Docs confirm a 'Zero Data Retention' feature letting customers delete all data tied to completed tasks, directly addressing retention/deletion control, but it's explicitly gated as an 'enterprise feature' rather than a universal capability, and no detail is given on default retention periods, deletion APIs/CLI commands, or granular controls for non-enterprise users. Community feedback (e.g., concerns about sending data to a third party, no security details found) shows some skepticism but doesn't concretely contradict the ZDR claim itself. missing for 10: default/non-enterprise retention policy, self-serve deletion mechanism (API/CLI), independent verification of ZDR working in practice.

   • [claimed-docs] “Zero Data Retention is an enterprise feature that provides enhanced data privacy by allowing you to delete all data associated with complete…”
   • [community] “My first question was whether I could use this for sensitive tasks, given that it's not running on our machines. And after poking around for…”

   proof ↗⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Steel—none0/10

   No evidence pack item mentions telemetry, analytics, usage tracking, or an opt-out mechanism; the self-hostable OSS nature is noted (steel-probe-rt-1) but that alone does not confirm any telemetry opt-out control exists or is documented.

   ⚑ flag

   Smooth—none0/10

   No evidence describes a telemetry/usage-tracking opt-out control; the only related privacy feature is 'Zero Data Retention' for enterprise customers, which addresses data deletion after tasks rather than disabling telemetry/tracking. Community comments raise general privacy/security concerns but do not confirm or deny an opt-out mechanism.

   • [claimed-docs] “Zero Data Retention is an enterprise feature that provides enhanced data privacy by allowing you to delete all data associated with complete…”
   • [community] “My first question was whether I could use this for sensitive tasks, given that it's not running on our machines. And after poking around for…”

   proof ↗⚑ flag

Live

1. automation-engineerWatch a session live and take human control mid-run when the agent gets stuck

   weight 2 · round to Steel
   Steel✓fullprobed8/10

   Steel's debug URL feature is explicitly documented for human-in-the-loop workflows enabling users to directly interact with and control a live browser session, and runtime proof confirms sessions expose a live debugger/websocket URL for real-time viewing/control. The agent-traces timeline feature complements this by letting engineers review what happened, though it's more post-hoc than live takeover. Missing for 10: explicit documentation of mid-run handoff back to the agent after human control, and independent/community corroboration of the human-in-the-loop debug feature specifically (vs. general product commentary).

   • [claimed-docs] “Steel's debug URL feature allows you to implement human-in-the-loop workflows where users can directly interact with and control browser ses…”
   • [claimed-docs] “It turns the run into a timeline of agent activity, so you can see what happened without scrubbing through the whole recording.”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”

   proof ↗⚑ flag

   Smooth~partialclaimed4/10

   Docs confirm a live_url to watch agent actions in real time (smooth-docs-2), satisfying the 'watch a session live' half of the story, but there is no evidence of any mechanism for a human to intervene or take control mid-run when the agent gets stuck. missing for 10: documented human-takeover/control API or UI, evidence of pausing/resuming agent execution, hands-on confirmation of mid-run intervention.

   • [claimed-docs] “When running a task, you will receive a `live_url`, which can be used to view the agent actions live.”
   • [claimed-docs] “Session Workflow — Multi-step execution where you can orchestrate smaller tasks, navigate to URLs, and extract data within a persistent brow…”

   proof ↗⚑ flag

Replay

1. automation-engineerDebug a failed agent run from recorded replays — video, screenshots, step-by-step action timelines

   weight 2 · round to Steel
   Steel✓fullclaimed7/10

   Steel's Agent Traces feature explicitly turns a run into a timeline of agent activity with screenshots, and lets you export as markdown, JSON, or a ZIP with markdown+screenshots, directly matching the replay-debugging story for automation engineers. Missing for 10: explicit video recording/playback evidence and independent/hands-on corroboration of the traces UI beyond first-party docs.

   • [claimed-docs] “It turns the run into a timeline of agent activity, so you can see what happened without scrubbing through the whole recording.”
   • [claimed-docs] “Copy the run as markdown, download JSON, or grab a ZIP with markdown plus screenshots.”
   • [claimed-docs] “Steel's debug URL feature allows you to implement human-in-the-loop workflows where users can directly interact with and control browser ses…”

   proof ↗⚑ flag

   Smooth~partialclaimed3/10

   Docs mention a `live_url` for viewing agent actions live during a run, but there is no evidence of persisted video recordings, screenshots, or a step-by-step action timeline that can be replayed after a run has finished and failed. Missing for 10: recorded video/screenshot artifacts, post-hoc replay viewer, structured action timeline, and any independent confirmation of replay-based debugging.

   • [claimed-docs] “When running a task, you will receive a `live_url`, which can be used to view the agent actions live.”

   proof ↗⚑ flag

Fleets

1. automation-engineerRun a fleet of concurrent browser sessions with documented concurrency limits and programmatic session management

   weight 2 · round to Steel
   Steel~partialprobed6/10

   Steel's Sessions API and CLI clearly support spinning up isolated on-demand browser sessions with custom session IDs, and a runtime probe confirms working session creation/listing via self-hosted API — solid programmatic session management. However, the evidence pack contains no documented concurrency limits, quotas, or fleet-scale guidance for running many sessions in parallel. Missing for 10: explicit documented concurrency/session limits, guidance or examples for orchestrating many simultaneous sessions, and independent corroboration of scale behavior.

   • [claimed-docs] “the Sessions API lets your agents spin up isolated browser instances on demand. Each session maintains its own state, cookies, and storage”
   • [claimed-docs] “Steel generates a session ID for you, but `create` also accepts one. Pass your own UUID when the ID has to exist before the browser does”
   • [claimed-docs] “The Steel CLI lets you run full browser workflows from the terminal, end-to-end. You can start a browser session, navigate pages, click/fill…”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”

   proof ↗⚑ flag

   Smooth—none0/10

   The evidence pack covers single-session features (persistent sessions, live URL, proxies, structured output) but contains no documentation of concurrency limits, fleet/pool management, or APIs for running many sessions in parallel. Missing for 10: documented concurrency limits, fleet/pool orchestration APIs, rate-limit or scaling guidance, evidence of parallel session usage.

   • [claimed-docs] “Session Workflow — Multi-step execution where you can orchestrate smaller tasks, navigate to URLs, and extract data within a persistent brow…”
   • [claimed-docs] “Log in once, then reuse that authentication for future tasks.”
   • [claimed-docs] “When running a task, you will receive a `live_url`, which can be used to view the agent actions live.”

   proof ↗⚑ flag

Lifecycle

1. developerGet webhook notifications when tasks and sessions finish instead of polling for status

   weight 1 · round drawn
   Steel—none0/10

   No evidence pack item mentions webhooks, event callbacks, or push notifications for session/task completion; Steel's docs cover sessions API, CLI, proxies, captcha, and traces but nothing about async status notifications, implying developers must poll session status endpoints.

   ⚑ flag

   Smooth—none0/10

   No evidence pack item mentions webhooks, callback URLs, or event-driven notifications for task/session completion; the docs describe live_url viewing, structured outputs, and session workflows but nothing about push notifications versus polling.

   ⚑ flag

Captcha

1. automation-engineerRely on a documented captcha stance — automatic solving, human fallback, or explicit non-support — instead of silent task failures

   weight 2 · round to Steel
   Steel✓fullclaimed8/10

   Steel documents a single explicit flag covering captcha detection, solving, and verification for the whole session, plus a separate human-in-the-loop debug URL feature for manual takeover when needed — giving automation engineers a documented stance rather than silent failures. Missing for 10: independent/hands-on verification that automatic captcha solving actually succeeds on real-world captchas, and clearer documentation of failure/fallback behavior when auto-solve fails.

   • [claimed-docs] “One flag covers detection, solving, and verification for the whole session”
   • [claimed-docs] “Steel's debug URL feature allows you to implement human-in-the-loop workflows where users can directly interact with and control browser ses…”

   proof ↗⚑ flag

   Smooth✓fullcommunity6/10

   Smooth explicitly documents an automatic captcha-solving stance ('Auto-CAPTCHA solvers: Bypass CAPTCHA challenges automatically, allowing for uninterrupted task execution'), giving automation engineers a clear documented behavior rather than silent failure. Community reaction (smooth-comm-3) criticizes the ethics/marketing of this feature but does not present a hands-on failure showing the solver doesn't work, so this remains a documented claim rather than a disputed one. Missing for 10: independent/hands-on verification that auto-solving actually succeeds in practice, and no documentation of fallback behavior (e.g., what happens if a captcha can't be auto-solved).

   • [claimed-docs] “Auto-CAPTCHA solvers: Bypass CAPTCHA challenges automatically, allowing for uninterrupted task execution.”
   • [community] “So you're shamelessly selling spambots? The marketing here is wild... "proxy rotation"... "auto-CAPTCHA solvers"”

   proof ↗⚑ flag

Posture

1. automation-engineerPoint to the vendor's published acceptable-use and anti-abuse posture governing what its stealth and automation features may be used for

   weight 1 · round drawn
   Steel—none0/10

   The evidence pack documents Steel's stealth/captcha-solving, proxy, and CAPTCHA features extensively, but contains no published acceptable-use policy, terms of service, or anti-abuse statement governing what these stealth capabilities may be used for. No AUP, ToS, or anti-abuse page is cited or referenced anywhere in the docs, GitHub repo, or community discussion. Missing for 10: a published acceptable-use policy, anti-abuse/misuse guidelines, or ToS language specifically addressing stealth/captcha feature usage.

   ⚑ flag

   Smooth—none0/10

   No evidence anywhere in the pack of a published acceptable-use policy, anti-abuse terms, or guidance on permissible use of the stealth/CAPTCHA-bypass and automation features; docs only describe how to use auto-CAPTCHA and proxy features, not what usage is disallowed. Community commentary even calls out the lack of any such framing (e.g., accusing the marketing of enabling spambots), reinforcing the absence rather than disputing a claim.

   • [claimed-docs] “Auto-CAPTCHA solvers: Bypass CAPTCHA challenges automatically, allowing for uninterrupted task execution.”
   • [community] “So you're shamelessly selling spambots? The marketing here is wild... "proxy rotation"... "auto-CAPTCHA solvers"”

   proof ↗⚑ flag

Stealth

1. automation-engineerEnable stealth fingerprinting and residential or geo-targeted proxies so legitimate automations aren't blocked as bots

   weight 2 · round to Steel
   Steel✓fullcommunity8/10

   Steel's docs explicitly document stealth mode (single flag covering detection evasion, captcha solving, and verification) plus both Managed Residential Proxies and Bring-Your-Own-Proxy (BYOP) for geo-targeting, directly matching the story, and community comment confirms custom proxy support works in practice. Missing for 10: no independent/hands-on evidence quantifying bot-detection bypass success rates or geo-targeting granularity, and no first-party benchmark showing reduced block rates.

   • [claimed-docs] “One flag covers detection, solving, and verification for the whole session”
   • [claimed-docs] “Steel offers two powerful ways to use proxies: our built-in **Managed Residential Proxies** or connecting to your own proxy provider with ou…”
   • [community] “It appears you can set your own proxies to not use their cloud.”

   proof ↗⚑ flag

   Smooth~partialcommunity5/10

   Docs confirm auto-CAPTCHA solving and configurable proxy server parameters plus persistent authenticated sessions, which support anti-bot automation goals, but there is no explicit mention of residential/geo-targeted proxy pools or stealth browser fingerprinting techniques. Missing for 10: explicit residential/geo-targeted proxy options, stealth fingerprinting details, and independent verification that bot-detection evasion actually works in practice.

   • [claimed-docs] “To use a proxy with Smooth, you need to specify the proxy server details in your task parameters.”
   • [claimed-docs] “Auto-CAPTCHA solvers: Bypass CAPTCHA challenges automatically, allowing for uninterrupted task execution.”
   • [claimed-docs] “Log in once, then reuse that authentication for future tasks.”
   • [community] “So you're shamelessly selling spambots? The marketing here is wild... "proxy rotation"... "auto-CAPTCHA solvers"”

   proof ↗⚑ flag

Extraction

1. developerExtract typed, schema-validated data (Zod/Pydantic-style) from pages the agent visits, not just raw text

   weight 3 · round to Smooth
   Steel—none0/10

   Steel's docs/GitHub only show raw content extraction utilities (markdown, readability, screenshots, PDF conversion) and generic 'extract content' CLI commands, with no mention of Zod/Pydantic-style schema validation or typed structured outputs. The axis is fair for a browser-automation API since competitors offer schema-based extraction, but no evidence shows Steel provides this.

   • [github] “Browser Tools: Exposes APIs to quick convert pages to markdown, readability, screenshots, or PDFs.”
   • [claimed-docs] “The Steel CLI lets you run full browser workflows from the terminal, end-to-end. You can start a browser session, navigate pages, click/fill…”

   proof ↗⚑ flag

   Smooth✓fullclaimed8/10

   Docs explicitly describe structured outputs via `response_model` for deterministic typed data and a dedicated `session-extract` method to extract structured data from a page by providing a schema, directly matching the story. Missing for 10: explicit Zod/Pydantic code examples and independent/hands-on confirmation that extraction validation works as documented.

   • [claimed-docs] “Structured outputs allow you to write deterministic code based on the agent's output. To activate structured outputs, set the `response_mode…”
   • [claimed-docs] “Extract structured data from the current page by providing a schema.”
   • [claimed-docs] “Session Workflow — Multi-step execution where you can orchestrate smaller tasks, navigate to URLs, and extract data within a persistent brow…”

   proof ↗⚑ flag

Files

1. developerMy agent can download files from and upload files to the sites it operates, with the artifacts retrievable afterwards

   weight 1 · round drawn
   Steel—none0/10

   The evidence covers session artifacts (traces, screenshots, page-to-markdown conversion) and CLI-driven browser control, but nothing documents actual file upload to web forms or downloading files from a site with persistent artifact retrieval — a distinct capability from trace export.

   ⚑ flag

   Smooth—none0/10

   The evidence pack describes Smooth's session workflows, structured extraction, live-view, proxies, and persistent auth, but nothing addresses file download/upload during a browser session or persisting artifacts for later retrieval. Since browser automation tools plausibly support file transfer (e.g., downloading a report from a site or uploading a document to a form), this axis applies but is unaddressed.

   • [claimed-docs] “Session Workflow — Multi-step execution where you can orchestrate smaller tasks, navigate to URLs, and extract data within a persistent brow…”
   • [claimed-docs] “Extract structured data from the current page by providing a schema.”
   • [claimed-docs] “When running a task, you will receive a `live_url`, which can be used to view the agent actions live.”

   proof ↗⚑ flag