Skyvern vs Steel

Caching

1. developerCache resolved actions or generated code so repeat runs replay deterministically at lower cost and latency than re-prompting the LLM

   weight 2 · round drawn
   Skyvern—none0/10

   No evidence of caching resolved actions or generated code for deterministic, cheaper replay; Skyvern's model is per-run AI-driven navigation via LLM+vision, and community feedback even complains about cost/latency of repeated LLM calls with no mention of a caching mechanism to mitigate this.

   • [community] “I tried it out and it's pretty pricey. My OpenAI API bill is $3.20 after using this on a few different pages to test it out... this is alway…”
   • [community] “This is an impressive tool. I especially like the observability around the workflow and the steps it takes to achieve the outcome. We are po…”
   • [claimed-docs] “It navigates websites it has never seen before, filling forms, extracting data, and completing multi-step tasks via a simple API.”

   proof ↗⚑ flag

   Steel—none0/10

   Steel's evidence covers session management, stealth, proxies, human-in-the-loop debugging, and agent traces, but nothing about caching resolved actions or generated code to enable deterministic, lower-cost replay without re-invoking the LLM. Agent traces (steel-docs-6/7) provide observability/export, not action-cache replay for cost savings.

   ⚑ flag

Dom

1. developerDrive the page through DOM-understanding action primitives (act/click/type on described elements) that survive selector and layout changes

   weight 3 · round to Skyvern
   Skyvern!disputedcontradicted5/10

   Skyvern's docs describe exactly this: natural-language act/click/type primitives with vision+DOM understanding that operate on sites 'never seen before' and fall back to selectors only if needed (skyvern-docs-19, skyvern-gh-1, skyvern-docs-17), positioned explicitly as a replacement for brittle Selenium scripts (skyvern-docs-13). However, a hands-on community test found it worked on the happy path but concretely failed to interact with a layout element (a popup) and struggled to hit a tab on a real site (skyvern-comm-2), contradicting the claim that it robustly survives arbitrary layout changes. Missing for 10: independent benchmark data on selector/layout-change robustness, broader corroboration beyond one hands-on report, and resolution of the observed failure mode.

   • [claimed-docs] “Drop-in AI commands on top of Playwright. Use natural language to act, extract, and validate — or fall back to selectors.”
   • [github] “Skyvern can operate on websites it's never seen before, as it's able to map visual elements to actions necessary to complete a workflow, wit…”
   • [claimed-docs] “It navigates websites it has never seen before, filling forms, extracting data, and completing multi-step tasks via a simple API.”
   • [claimed-docs] “You're replacing brittle Selenium scripts, integrating browser automation via API, or building workflows into your product.”
   • [community] “I played with the Geico example, and it seems to do a good job on the happy path. But I tried costcotravel.com... it struggled to hit the 'r…”

   proof ↗⚑ flag

   Steel—none0/10

   Steel's evidence shows only traditional CDP/Puppeteer/Selenium-based control and CLI commands like click/fill/type (steel-docs-8, steel-gh-1), which are selector-based automation primitives, not AI/DOM-understanding 'act on described element' primitives that resolve targets semantically and survive selector/layout changes. No documentation or hands-on evidence describes a Stagehand-like natural-language action resolver.

   • [claimed-docs] “The Steel CLI lets you run full browser workflows from the terminal, end-to-end. You can start a browser session, navigate pages, click/fill…”
   • [github] “Uses Puppeteer and CDP for complete control over Chrome instances -- allowing you to connect using Puppeteer, Playwright, or Selenium.”

   proof ↗⚑ flag

Observe

1. developerPreview candidate actions on the current page (observe/plan) before committing the agent to act

   weight 1 · round to Steel
   Skyvern~partialcommunity3/10

   Skyvern's docs mention human-in-the-loop pausing for approval between steps and a VNC stream to watch/take control, which offers some ability to intervene before the agent proceeds, but there is no documented explicit 'plan/preview candidate actions' step (e.g., a dry-run or action list shown before execution). A community comment even notes the absence of assertion/verification-style controls compared to Playwright, suggesting no built-in preview mechanism for validating steps before they run. Missing for 10: an explicit plan/preview UI or API that lists candidate actions before execution, and independent confirmation that the pause-for-approval flow shows planned actions rather than just pausing mid-run.

   • [claimed-docs] “Human-in-the-loop flows: pause for approval between steps without losing browser state. The VNC stream lets you watch or take control at any…”
   • [community] “I can't see that option in Skyvern which would have me worrying that process changes would be overlooked and we would unknowingly start ente…”

   proof ↗⚑ flag

   Steel~partialclaimed4/10

   Steel exposes browser observation primitives (page-to-markdown/readability/screenshot extraction, agent traces/timeline, and a debug URL for human-in-the-loop review) that let a developer inspect page state or a human intervene mid-session, but there is no documented 'plan' or dry-run API that lets an agent preview a set of candidate actions before committing to execute them. missing for 10: an explicit plan/observe-then-act primitive or dry-run action preview API, evidence of independent developers using it specifically for pre-commit action review.

   • [github] “Browser Tools: Exposes APIs to quick convert pages to markdown, readability, screenshots, or PDFs.”
   • [claimed-docs] “Steel's debug URL feature allows you to implement human-in-the-loop workflows where users can directly interact with and control browser ses…”
   • [claimed-docs] “It turns the run into a timeline of agent activity, so you can see what happened without scrubbing through the whole recording.”

   proof ↗⚑ flag

Vision

1. developerSwitch to a vision or computer-use action mode that operates on screenshots for canvases and UIs the DOM path can't handle

   weight 2 · round to Skyvern
   Skyvern✓fullclaimed7/10

   Skyvern's core action loop is vision-based: it maps visual elements to actions on pages it has never seen, without custom DOM-specific code (skyvern-gh-1), and separately uses its vision model to detect and solve CAPTCHAs, which are canvas-like elements the DOM can't parse (skyvern-docs-9, skyvern-docs-27). Docs also mention falling back to selectors when useful (skyvern-docs-19), implying vision-first with DOM as a secondary path rather than a purely DOM-based tool needing a special switch. missing for 10: explicit documentation of a discrete 'vision/computer-use mode' toggle, dedicated canvas/non-DOM UI examples (e.g., canvas-drawn widgets, non-HTML apps), and independent benchmarking confirming success on such UIs.

   • [github] “Skyvern can operate on websites it's never seen before, as it's able to map visual elements to actions necessary to complete a workflow, wit…”
   • [claimed-docs] “Skyvern detects CAPTCHAs using its vision model and solves them automatically. This works for reCAPTCHA v2/v3, hCaptcha, Cloudflare Turnstil…”
   • [claimed-docs] “Skyvern detects CAPTCHAs using its vision model and solves them automatically.”
   • [claimed-docs] “Drop-in AI commands on top of Playwright. Use natural language to act, extract, and validate — or fall back to selectors.”

   proof ↗⚑ flag

   Steel—none0/10

   Evidence shows Steel can capture screenshots, convert pages to markdown/PDF, and offers a debug URL for human-in-the-loop control, but there is no mention of a vision/computer-use action mode where an agent issues click/type actions based on screenshot coordinates instead of DOM selectors.

   ⚑ flag

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to Steel
   Skyvern✓fullprobed8/10

   A probe confirms llms.txt is live and returns a structured summary of Skyvern for agent consumption, and skyvern.com/llms provides an agent-oriented docs page listing features in a scannable format. This directly satisfies pointing an agent at llms.txt or agent-oriented docs. Missing for 10: a docs.md/markdown-mirrored docs endpoint (404) and an accessible OpenAPI spec, which would round out machine-readable documentation.

   • [probe] “PROBE llms.txt: HTTP 200 at https://skyvern.com/llms.txt # Skyvern > Skyvern is an open-source, AI-powered browser automation platform. It …”
   • [claimed-docs] “Visual workflow builder for non-developers — drag-and-drop, no code required”
   • [claimed-docs] “Browser recorder that converts manual actions into reusable automations”
   • [claimed-docs] “SOP upload — describe a process in plain English and Skyvern builds the workflow”
   • [claimed-docs] “Copilot chat for building and debugging workflows interactively”
   • [probe] “PROBE docs-md: HTTP 404 at https://skyvern.com/docs.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://skyvern.com/openapi.json, https://skyvern.com/swagger.json, https://skyvern.com/api/openapi.…”

   proof ↗⚑ flag

   Steel✓fullprobed9/10

   Probe confirms llms.txt is live at docs.steel.dev/llms.txt (HTTP 200) with agent-oriented framing, and the docs also expose an OpenAPI spec, making the docs machine/agent consumable. missing for 10: no independent third-party confirmation that agents actually consume the llms.txt file successfully in practice.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.steel.dev/llms.txt # Steel Documentation > Steel is the open-source browser API for AI agents — ma…”
   • [probe] “PROBE openapi: HTTP 200 at https://docs.steel.dev/openapi.json — contains "openapi" key”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to Steel
   Skyvern✓fullclaimed7/10

   Skyvern ships a code-first SDK/REST API (Python/TypeScript) that connects to a cloud or self-hosted Chromium instance, explicitly positioned as replacing brittle Selenium scripts and integrating browser automation via API into other products, and can run entirely on your own infrastructure with your own LLM keys, supporting headless/scriptable use suitable for CI. missing for 10: explicit CI/CD pipeline documentation or example (e.g. GitHub Actions integration), and independent confirmation of headless execution in automated environments

   • [claimed-docs] “Integrate browser automation into your product with Python, TypeScript, or REST.”
   • [claimed-docs] “Run Skyvern on your own infrastructure with your own LLM keys.”
   • [claimed-docs] “Browser Automation is the code-first way to build multi-step automations. The Skyvern SDK connects to a cloud Chromium instance over CDP, la…”
   • [claimed-docs] “Self-hosted Skyvern runs entirely on your infrastructure: your servers, your browsers, your LLM API keys.”
   • [claimed-docs] “You're replacing brittle Selenium scripts, integrating browser automation via API, or building workflows into your product.”

   proof ↗⚑ flag

   Steel✓fullprobed9/10

   Steel provides a documented CLI for end-to-end headless browser workflows from the terminal, a REST/SDK API for programmatic session creation, an open-source Docker image for self-hosting, and a verified probe confirming a keyless self-host roundtrip (docker run, health check, session creation via API, CLI install, SDK install) — all strongly supporting CI/headless automation use. missing for 10: no explicit first-party CI pipeline example (e.g., GitHub Actions template) or independent hands-on report of running Steel inside an actual CI system.

   • [claimed-docs] “The Steel CLI lets you run full browser workflows from the terminal, end-to-end. You can start a browser session, navigate pages, click/fill…”
   • [github] “Pre-built Docker Image (combined API + UI)”
   • [github] “Uses Puppeteer and CDP for complete control over Chrome instances -- allowing you to connect using Puppeteer, Playwright, or Selenium.”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”
   • [probe] “official CLI documented at https://docs.steel.dev/overview/steel-cli”
   • [claimed-docs] “the Sessions API lets your agents spin up isolated browser instances on demand. Each session maintains its own state, cookies, and storage”

   proof ↗⚑ flag

3. ai-native userConnect an agent via an official MCP server

   weight 3 · round to Skyvern
   Skyvern✓fullprobed8/10

   Skyvern documents an official MCP server (skyvern-docs-7, skyvern-probe-4) that lets AI assistants like Claude Desktop, Claude Code, Codex, Cursor, and Windsurf control a browser via Skyvern, directly matching the story. Missing for 10: independent/hands-on community verification of the MCP server specifically (community evidence covers other features, not MCP usage) and a clear setup/config example beyond the single doc page.

   • [claimed-docs] “The Skyvern MCP server lets AI assistants like Claude Desktop, Claude Code, Codex, Cursor, and Windsurf control a browser.”
   • [probe] “official MCP server documented at https://skyvern.com/docs/developers/getting-started/mcp”

   proof ↗⚑ flag

   Steel~partialclaimed5/10

   Steel's docs mention an integration with the Claude Agent SDK that 'exposes a cloud browser as in-process MCP tools,' showing some official MCP tool exposure for agents, but there's no evidence of a standalone, general-purpose official MCP server endpoint independent of this one SDK integration. missing for 10: a dedicated/standalone MCP server doc or endpoint usable by any agent framework, independent corroboration or hands-on proof of MCP connectivity.

   • [claimed-docs] “The Steel integration exposes a cloud browser as in-process MCP tools, so the SDK runs the agent loop and Steel handles the browser.”

   proof ↗⚑ flag

4. ai-native userUse an official CLI

   weight 2 · round to Steel
   Skyvern—none0/10

   Evidence covers Skyvern's Python/TypeScript SDKs, REST API, MCP server, and visual dashboard, but nowhere mentions an official CLI tool for AI-native workflows. missing for 10: any documented CLI command, npm/pip CLI package, or terminal-based interface.

   • [claimed-docs] “Integrate browser automation into your product with Python, TypeScript, or REST.”
   • [claimed-docs] “Browser Automation is the code-first way to build multi-step automations. The Skyvern SDK connects to a cloud Chromium instance over CDP, la…”
   • [claimed-docs] “The Skyvern MCP server lets AI assistants like Claude Desktop, Claude Code, Codex, Cursor, and Windsurf control a browser.”

   proof ↗⚑ flag

   Steel✓fullprobed8/10

   Steel ships a documented official CLI (steel-docs-8) that supports end-to-end browser workflows from the terminal, and probe evidence confirms real installation via setup.steel.dev installing 'steel CLI 0.4.4' into a fresh environment (steel-probe-rt-1), corroborating the docs. Missing for 10: independent third-party reviews of the CLI's UX/reliability beyond the vendor-run probe, and more detail on advanced CLI subcommands/scripting capabilities.

   • [claimed-docs] “The Steel CLI lets you run full browser workflows from the terminal, end-to-end. You can start a browser session, navigate pages, click/fill…”
   • [probe] “official CLI documented at https://docs.steel.dev/overview/steel-cli”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”

   proof ↗⚑ flag

5. ai-native userDrive the product through a documented public API

   weight 3 · round to Steel
   Skyvern✓fullprobed7/10

   Skyvern documents a public API/SDK surface (Python, TypeScript, REST) for creating tasks, running multi-step browser automations, and extracting structured data via JSON schema, matching the ai-native 'drive via documented API' story; it also ships an MCP server for agent control. Missing for 10: a discoverable OpenAPI/swagger spec (probe found 404s for all candidate paths) and independent/hands-on confirmation of API robustness beyond first-party docs.

   • [claimed-docs] “Integrate browser automation into your product with Python, TypeScript, or REST.”
   • [claimed-docs] “Browser Automation is the code-first way to build multi-step automations. The Skyvern SDK connects to a cloud Chromium instance over CDP, la…”
   • [claimed-docs] “It navigates websites it has never seen before, filling forms, extracting data, and completing multi-step tasks via a simple API.”
   • [claimed-docs] “You provide a natural-language prompt describing the goal, a starting URL, and optionally a JSON schema for structured output.”
   • [claimed-docs] “you can extract structured data from any page using `page.extract` with a JSON schema, or by passing a `data_extraction_schema` to `page.age…”
   • [claimed-docs] “The Skyvern MCP server lets AI assistants like Claude Desktop, Claude Code, Codex, Cursor, and Windsurf control a browser.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://skyvern.com/openapi.json, https://skyvern.com/swagger.json, https://skyvern.com/api/openapi.…”
   • [probe] “official MCP server documented at https://skyvern.com/docs/developers/getting-started/mcp”

   proof ↗⚑ flag

   Steel✓fullprobed9/10

   Steel publishes a full OpenAPI spec (steel-probe-2), documented Sessions API with SDKs, and a CLI, all confirmed hands-on by a runtime probe showing session creation, health checks, and SDK usage working end-to-end. This is strong first-party documentation plus independent verification of a working public API. Missing for 10: no third-party community deep-dive validating API completeness beyond the probe.

   • [probe] “PROBE openapi: HTTP 200 at https://docs.steel.dev/openapi.json — contains "openapi" key”
   • [claimed-docs] “the Sessions API lets your agents spin up isolated browser instances on demand. Each session maintains its own state, cookies, and storage”
   • [claimed-docs] “The Steel CLI lets you run full browser workflows from the terminal, end-to-end. You can start a browser session, navigate pages, click/fill…”
   • [probe] “official CLI documented at https://docs.steel.dev/overview/steel-cli”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”

   proof ↗⚑ flag

6. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round drawn
   Skyvern—none0/10

   No evidence of scoped or least-privilege API credential issuance for agents; docs mention API keys and self-hosted LLM keys but nothing about credential scoping, permissions, or restricting agent access levels. Community comments even raise concerns about handling sensitive credentials in plain text with no mitigation shown. Missing for 10: any documentation of scoped API tokens, role-based access control, or least-privilege credential management for agents.

   • [claimed-docs] “Run Skyvern on your own infrastructure with your own LLM keys.”
   • [claimed-docs] “Self-hosted Skyvern runs entirely on your infrastructure: your servers, your browsers, your LLM API keys.”
   • [community] “you are expecting them to pass over their website login credentials and apparently their credit card details too, in plain text. You had bet…”

   proof ↗⚑ flag

   Steel—none0/10

   No evidence anywhere in the pack of scoped or least-privilege API key/credential issuance (e.g., role-based keys, permission scopes, restricted tokens) for agents — only session isolation, proxies, and CLI features are documented, none of which address credential scoping.

   ⚑ flag

7. ai-native userBuild against official SDKs

   weight 2 · round to Steel
   Skyvern✓fullprobed7/10

   Skyvern explicitly documents official Python and TypeScript SDKs plus a REST API for integrating browser automation, with SDK-level primitives like page.extract and data_extraction_schema shown in docs (skyvern-docs-1, skyvern-docs-5, skyvern-docs-19, skyvern-docs-4). Missing for 10: independent/hands-on developer corroboration of SDK usage and a public API reference (OpenAPI spec probe returned 404s, skyvern-probe-3), so quality is capped below full confidence in completeness.

   • [claimed-docs] “Integrate browser automation into your product with Python, TypeScript, or REST.”
   • [claimed-docs] “Browser Automation is the code-first way to build multi-step automations. The Skyvern SDK connects to a cloud Chromium instance over CDP, la…”
   • [claimed-docs] “Drop-in AI commands on top of Playwright. Use natural language to act, extract, and validate — or fall back to selectors.”
   • [claimed-docs] “you can extract structured data from any page using `page.extract` with a JSON schema, or by passing a `data_extraction_schema` to `page.age…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://skyvern.com/openapi.json, https://skyvern.com/swagger.json, https://skyvern.com/api/openapi.…”

   proof ↗⚑ flag

   Steel✓fullprobed8/10

   Steel ships an official steel-sdk npm package (verified working via probe: exports a Steel client class), a documented OpenAPI spec, and an official CLI for full browser workflows, all covered in first-party docs and confirmed by a hands-on runtime probe. missing for 10: explicit multi-language SDK coverage (e.g., Python/other languages) and independent community confirmation of SDK usage beyond the CLI/API.

   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”
   • [probe] “PROBE openapi: HTTP 200 at https://docs.steel.dev/openapi.json — contains "openapi" key”
   • [probe] “official CLI documented at https://docs.steel.dev/overview/steel-cli”
   • [claimed-docs] “The Steel CLI lets you run full browser workflows from the terminal, end-to-end. You can start a browser session, navigate pages, click/fill…”
   • [claimed-docs] “The Steel integration exposes a cloud browser as in-process MCP tools, so the SDK runs the agent loop and Steel handles the browser.”

   proof ↗⚑ flag

8. ai-native userSubscribe to events via webhooks

   weight 2 · round drawn
   Skyvern—none0/10

   No evidence in the pack mentions webhooks or event subscriptions of any kind; Skyvern's documented integration surfaces are REST/SDK APIs, Zapier, and an MCP server, none of which constitute a webhook subscription mechanism.

   ⚑ flag

   Steel—none0/10

   No evidence anywhere in the pack mentions webhooks or event subscriptions; Steel's documented surfaces are Sessions API, CLI, MCP tools, and agent traces, none of which describe a webhook/event notification mechanism.

   ⚑ flag

Agentic features

1. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Skyvern
   Skyvern~partialcommunity7/10

   Skyvern's docs show multi-step, code-first and no-code workflows that run via API or cloud UI, persist browser state, and can pause for human approval while capturing recordings/artifacts — all indicative of autonomous background execution (skyvern-docs-5,6,10,11,21). Zapier integration and API-driven triggering (skyvern-docs-16, skyvern-docs-28) supports running without manual intervention, but there's no explicit documentation of a scheduler, cron-like triggers, or continuous monitoring dashboard for unattended runs. Missing for 10: explicit scheduling/trigger docs, evidence of long-running unattended background jobs, and independent confirmation of reliability at scale (community notes some brittleness, e.g. skyvern-comm-2).

   • [claimed-docs] “Browser Automation is the code-first way to build multi-step automations. The Skyvern SDK connects to a cloud Chromium instance over CDP, la…”
   • [claimed-docs] “Build multi-step automations visually in the Cloud UI with drag-and-drop blocks. No code required. Share templates across your team.”
   • [claimed-docs] “Human-in-the-loop flows: pause for approval between steps without losing browser state. The VNC stream lets you watch or take control at any…”
   • [claimed-docs] “Every run automatically captures what happened: recordings of the browser session, screenshots at each step, the AI's reasoning, and network…”
   • [claimed-docs] “Cookies, local storage, open tabs, and the current page all persist, so later operations pick up exactly where the previous one stopped.”
   • [claimed-docs] “Connect to Zapier”
   • [claimed-docs] “Skyvern automates browser-based workflows across these platforms — no API keys or custom connectors required.”
   • [community] “I played with the Geico example, and it seems to do a good job on the happy path. But I tried costcotravel.com... it struggled to hit the 'r…”

   proof ↗⚑ flag

   Steel—none0/10

   Steel provides on-demand browser sessions, CLI, SDK, and agent-trace tooling for agents to control browsers, but nothing in the evidence describes a scheduling/trigger mechanism or persistent background job runner that lets a user set up automations to run autonomously without invocation — sessions are explicitly spun up 'on demand' by an agent/script, not scheduled by Steel itself.

   • [claimed-docs] “the Sessions API lets your agents spin up isolated browser instances on demand. Each session maintains its own state, cookies, and storage”
   • [claimed-docs] “The Steel CLI lets you run full browser workflows from the terminal, end-to-end. You can start a browser session, navigate pages, click/fill…”
   • [claimed-docs] “Steel generates a session ID for you, but `create` also accepts one. Pass your own UUID when the ID has to exist before the browser does”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”

   proof ↗⚑ flag

2. ai-native userOperate the product with natural-language commands

   weight 2 · round to Skyvern
   Skyvern✓fullcommunity7/10

   Skyvern's core interaction model is natural-language: users provide a prompt describing the goal (docs-18), SOPs in plain English are converted to workflows (docs-24), and a Copilot chat and MCP server let AI assistants/users direct browser actions in natural language (docs-25, docs-7, docs-19). This is corroborated by community reports of using it via prompts on real sites, though with mixed reliability on complex flows. missing for 10: independent benchmarking or hands-on confirmation that natural-language commands reliably handle complex multi-step tasks, and clearer evidence of NL-driven success rates beyond anecdotal HN reports.

   • [claimed-docs] “You provide a natural-language prompt describing the goal, a starting URL, and optionally a JSON schema for structured output.”
   • [claimed-docs] “SOP upload — describe a process in plain English and Skyvern builds the workflow”
   • [claimed-docs] “Copilot chat for building and debugging workflows interactively”
   • [claimed-docs] “The Skyvern MCP server lets AI assistants like Claude Desktop, Claude Code, Codex, Cursor, and Windsurf control a browser.”
   • [claimed-docs] “Drop-in AI commands on top of Playwright. Use natural language to act, extract, and validate — or fall back to selectors.”
   • [community] “I played with the Geico example, and it seems to do a good job on the happy path. But I tried costcotravel.com... it struggled to hit the 'r…”

   proof ↗⚑ flag

   Steel~partialclaimed5/10

   Steel exposes its browser control as MCP tools within agent SDKs (e.g., Claude Agent SDK) so an AI agent can translate natural-language user requests into Steel API calls, and the CLI/SDK/API allow full programmatic control — but there's no evidence of a native natural-language interface to Steel itself (e.g., a chat command layer); control still requires structured API/CLI calls or a separate agent framework. Missing for 10: a first-party NL command interface or chat-driven control surface, and independent confirmation that NL-driven agent use works end-to-end in production.

   • [claimed-docs] “The Steel integration exposes a cloud browser as in-process MCP tools, so the SDK runs the agent loop and Steel handles the browser.”
   • [claimed-docs] “The Steel CLI lets you run full browser workflows from the terminal, end-to-end. You can start a browser session, navigate pages, click/fill…”
   • [github] “Uses Puppeteer and CDP for complete control over Chrome instances -- allowing you to connect using Puppeteer, Playwright, or Selenium.”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round to Steel
   Skyvern—none0/10

   Evidence shows only static docs describing SDKs and REST usage, with no interactive API reference or runnable-example explorer; probes explicitly found no OpenAPI/Swagger spec at any candidate path (404s), indicating no interactive reference exists.

   • [probe] “PROBE openapi: all candidate paths 404 (https://skyvern.com/openapi.json, https://skyvern.com/swagger.json, https://skyvern.com/api/openapi.…”
   • [probe] “PROBE docs-md: HTTP 404 at https://skyvern.com/docs.md”
   • [claimed-docs] “Integrate browser automation into your product with Python, TypeScript, or REST.”

   proof ↗⚑ flag

   Steel~partialprobed6/10

   Steel publishes a full OpenAPI spec (docs.steel.dev/openapi.json) and an llms.txt, and a community commenter independently praised the docs/API reference quality, suggesting an interactive, well-documented API surface. However, there's no explicit evidence of an in-browser 'try it' / runnable-example console distinct from static docs. Missing for 10: direct confirmation of an interactive try-it console with live runnable code snippets, and independent hands-on verification of that specific feature.

   • [probe] “PROBE openapi: HTTP 200 at https://docs.steel.dev/openapi.json — contains "openapi" key”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.steel.dev/llms.txt # Steel Documentation > Steel is the open-source browser API for AI agents — ma…”
   • [community] “beautiful docs + api ref! what are you using? (cool that you're doing open-source browserbase also, excited to check this out)”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round to Steel
   Skyvern—none0/10

   Skyvern offers a REST API (skyvern-docs-1) but a direct probe for OpenAPI/swagger specs at standard paths returned 404 across all candidates, and no docs mention a downloadable machine-readable spec.

   • [probe] “PROBE openapi: all candidate paths 404 (https://skyvern.com/openapi.json, https://skyvern.com/swagger.json, https://skyvern.com/api/openapi.…”
   • [probe] “PROBE docs-md: HTTP 404 at https://skyvern.com/docs.md”
   • [claimed-docs] “Integrate browser automation into your product with Python, TypeScript, or REST.”

   proof ↗⚑ flag

   Steel✓fullprobed9/10

   A probe confirms a live, valid OpenAPI JSON spec served at docs.steel.dev/openapi.json (HTTP 200 with 'openapi' key), directly satisfying the machine-readable spec requirement, complemented by an llms.txt index for discoverability. missing for 10: independent third-party corroboration beyond the automated probe.

   • [probe] “PROBE openapi: HTTP 200 at https://docs.steel.dev/openapi.json — contains "openapi" key”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.steel.dev/llms.txt # Steel Documentation > Steel is the open-source browser API for AI agents — ma…”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round to Steel
   Skyvern—none0/10

   Skyvern's docs describe cloud or self-hosted execution, credential handling, and observability, but nothing describes a dedicated sandbox/staging mode isolated from production data or systems — missing for 10: any mention of a sandbox environment, test/staging mode, or data isolation guarantees.

   ⚑ flag

   Steel✓fullprobed7/10

   Steel's core Sessions API spins up isolated, on-demand browser instances each with their own state, cookies, and storage, and this isolation was independently verified via a self-hosted runtime probe that created a live, separate browser session from a throwaway Docker instance — effectively a sandbox with no shared production state. Missing for 10: explicit documentation framing sessions as a 'test vs production' environment, and no first-party guidance on staging/production data separation policies beyond session isolation.

   • [claimed-docs] “the Sessions API lets your agents spin up isolated browser instances on demand. Each session maintains its own state, cookies, and storage”
   • [claimed-docs] “Steel generates a session ID for you, but `create` also accepts one. Pass your own UUID when the ID has to exist before the browser does”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”
   • [github] “Pre-built Docker Image (combined API + UI)”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Skyvern—none0/10

   No evidence of API versioning scheme or a documented deprecation policy; OpenAPI/spec probes all returned 404 and no changelog or versioning docs appear in the pack. Missing for 10: versioned API endpoints (e.g., /v1/), a published deprecation/support policy, and changelog documentation.

   • [probe] “PROBE openapi: all candidate paths 404 (https://skyvern.com/openapi.json, https://skyvern.com/swagger.json, https://skyvern.com/api/openapi.…”
   • [probe] “PROBE docs-md: HTTP 404 at https://skyvern.com/docs.md”

   proof ↗⚑ flag

   Steel—none0/10

   No evidence pack item mentions API versioning scheme or a documented deprecation policy; only an OpenAPI spec and general docs are shown, neither of which discuss version lifecycle or deprecation commitments.

   ⚑ flag

Compat

1. developerConnect my existing Playwright, Puppeteer, or CDP automation code to the product's browsers instead of rewriting it

   weight 2 · round to Steel
   Skyvern~partialclaimed5/10

   Docs show Skyvern's own SDK connects to a cloud Chromium instance over CDP and layers Playwright on top, and describe 'drop-in AI commands on top of Playwright' with fallback to raw selectors, implying some interoperability with existing Playwright code. However there is no explicit guidance or example showing a developer pointing an existing Playwright/Puppeteer/CDP script at Skyvern's managed browser instead of rewriting into Skyvern's task/workflow API, and no independent confirmation of this specific reuse pattern. Missing for 10: explicit BYO-script CDP endpoint docs, Puppeteer-specific support, and hands-on/community verification of dropping in existing automation code unchanged.

   • [claimed-docs] “Browser Automation is the code-first way to build multi-step automations. The Skyvern SDK connects to a cloud Chromium instance over CDP, la…”
   • [claimed-docs] “Drop-in AI commands on top of Playwright. Use natural language to act, extract, and validate — or fall back to selectors.”
   • [claimed-docs] “you can extract structured data from any page using `page.extract` with a JSON schema, or by passing a `data_extraction_schema` to `page.age…”

   proof ↗⚑ flag

   Steel✓fullprobed9/10

   Steel explicitly supports connecting existing Puppeteer, Playwright, or Selenium code via CDP to control its browser instances, and a runtime probe confirms live sessions expose websocket/debugger URLs consistent with CDP connectivity. Docs and SDK further corroborate first-class session management compatible with standard automation libraries. Missing for 10: independent third-party hands-on confirmation specifically of a rewritten Playwright/Puppeteer script running unmodified against Steel.

   • [github] “Uses Puppeteer and CDP for complete control over Chrome instances -- allowing you to connect using Puppeteer, Playwright, or Selenium.”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”
   • [claimed-docs] “the Sessions API lets your agents spin up isolated browser instances on demand. Each session maintains its own state, cookies, and storage”

   proof ↗⚑ flag

Credentials

1. automation-engineerStore credentials in a vault and have the agent complete logins including TOTP/2FA challenges without exposing secrets to the model

   weight 2 · round to Skyvern
   Skyvern✓fullcommunity7/10

   Skyvern's docs explicitly describe storing credentials via password-manager vault integrations (Bitwarden, 1Password, Azure Key Vault) and automatically handling TOTP/2FA, email, and SMS verification during login flows, matching the story closely [skyvern-docs-8][skyvern-docs-26]. However, there's no independent/hands-on verification that secrets are never exposed to the LLM, and a community comment raises concern about credentials being handled in plain text, so missing for 10: independent security audit or hands-on confirmation of secret-masking from the model, and clarification addressing the community's plaintext-handling concern.

   • [claimed-docs] “Skyvern handles logins with stored credentials, TOTP/authenticator codes, email and SMS verification, magic links, and password manager inte…”
   • [claimed-docs] “Skyvern handles authentication end-to-end, from simple passwords to multi-factor flows with TOTP codes, email verification, and magic links.”
   • [community] “you are expecting them to pass over their website login credentials and apparently their credit card details too, in plain text. You had bet…”

   proof ↗⚑ flag

   Steel—none0/10

   Steel's docs show session-level auth persistence (reusing cookies/storage across sessions) but no evidence of a credentials vault, secret injection without model exposure, or TOTP/2FA challenge automation — the core asks of this story are unaddressed.

   • [claimed-docs] “This is particularly useful for maintaining authenticated states across multiple sessions, helping your AI agents access protected resources…”

   proof ↗⚑ flag

Profiles

1. developerPersist logged-in browser state in reusable profiles so agents skip the login wall on every subsequent run

   weight 3 · round to Steel
   Skyvern~partialclaimed6/10

   Skyvern's browser-sessions feature explicitly persists cookies, local storage, and open tabs across operations so 'later operations pick up exactly where the previous one stopped,' and pauses preserve browser state — this directly supports skipping repeated logins. However, the docs don't clearly describe a named 'profile' abstraction, how long sessions persist across truly separate future runs, or how these persisted sessions are managed/reused across different agents or teams. missing for 10: explicit reusable-profile management docs, long-term persistence guarantees across independent runs, independent/hands-on confirmation of skip-login behavior.

   • [claimed-docs] “Cookies, local storage, open tabs, and the current page all persist, so later operations pick up exactly where the previous one stopped.”
   • [claimed-docs] “Human-in-the-loop flows: pause for approval between steps without losing browser state. The VNC stream lets you watch or take control at any…”
   • [claimed-docs] “Skyvern handles logins with stored credentials, TOTP/authenticator codes, email and SMS verification, magic links, and password manager inte…”

   proof ↗⚑ flag

   Steel✓fullprobed8/10

   Steel docs explicitly document reusing auth context across sessions to let agents skip repeated logins, backed by session isolation, custom session IDs, and a real API/CLI/SDK confirmed via runtime probe. Missing for 10: independent third-party hands-on confirmation of the reuse-auth-context feature specifically (only vendor docs cite it) and no explicit profile-export/import UX details beyond the docs description.

   • [claimed-docs] “This is particularly useful for maintaining authenticated states across multiple sessions, helping your AI agents access protected resources…”
   • [claimed-docs] “the Sessions API lets your agents spin up isolated browser instances on demand. Each session maintains its own state, cookies, and storage”
   • [claimed-docs] “Steel generates a session ID for you, but `create` also accepts one. Pass your own UUID when the ID has to exist before the browser does”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Steel
   Skyvern—none0/10

   No evidence describes a bulk-operation feature (e.g., running the same task across a list/CSV of items, batch triggering, or concurrent multi-item processing) — the docs focus on single-task API calls, visual workflows, and MCP integration rather than batch/bulk execution.

   ⚑ flag

   Steel~partialclaimed4/10

   Steel's Sessions API allows spinning up isolated browser sessions on demand and reusing auth context across multiple sessions, which implies you could programmatically launch many sessions for parallel/bulk tasks, but there is no explicit documentation of a batch/bulk API, concurrency limits, or guidance for orchestrating many items at once. missing for 10: explicit bulk/batch API or documented pattern for running many operations concurrently, concurrency/rate limits, and independent evidence of large-scale parallel session usage.

   • [claimed-docs] “the Sessions API lets your agents spin up isolated browser instances on demand. Each session maintains its own state, cookies, and storage”
   • [claimed-docs] “This is particularly useful for maintaining authenticated states across multiple sessions, helping your AI agents access protected resources…”
   • [claimed-docs] “Steel generates a session ID for you, but `create` also accepts one. Pass your own UUID when the ID has to exist before the browser does”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to Skyvern
   Skyvern~partialclaimed3/10

   Skyvern documents a Zapier integration, which could allow external events to trigger Skyvern workflows, but there is no evidence of a native rule/trigger engine, webhooks, or scheduled/event-based automation within Skyvern itself. Missing for 10: documented native event triggers or webhook listeners, schedule-based triggers, and any conditional rule engine inside Skyvern's workflow builder.

   • [claimed-docs] “Connect to Zapier”
   • [claimed-docs] “Build multi-step automations visually in the Cloud UI with drag-and-drop blocks. No code required. Share templates across your team.”

   proof ↗⚑ flag

   Steel—none0/10

   Steel's evidence covers sessions, proxies, captcha solving, CLI, and agent traces, but nothing describes a rules engine or event-triggered automation (e.g., 'if X happens then do Y') that a user could configure declaratively.

   ⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round drawn
   Skyvern—none0/10

   The evidence pack describes Skyvern's workflow builder, API/SDK, MCP integration, and automation features extensively, but contains no mention of scheduling, cron triggers, or recurring job execution anywhere in the docs, GitHub description, or community discussion. Since Skyvern is a workflow/automation platform, scheduling recurring runs is a fair capability to expect, but it's simply absent from the provided evidence.

   ⚑ flag

   Steel—none0/10

   Steel is a browser automation/session API for AI agents (sessions, stealth, proxies, CLI, MCP tools), but nothing in the evidence describes a scheduler, cron-like trigger, or recurring job/workflow orchestration feature; usage is invoked on-demand per session, not on a schedule.

   ⚑ flag

Local

1. developerRun the agent against a local browser on my own machine for development, without any cloud account

   weight 2 · round to Steel
   Skyvern~partialclaimed6/10

   Skyvern is open-source and its self-hosted docs explicitly state it 'runs entirely on your infrastructure: your servers, your browsers, your LLM API keys' (skyvern-docs-12, skyvern-docs-2), which supports running without a cloud account. However, the core SDK/browser-automation flow described elsewhere connects to a 'cloud Chromium instance over CDP' (skyvern-docs-5), suggesting the default path is cloud-based, and no local-machine dev setup details (docker/local browser config, install steps) are shown. Missing for 10: explicit local-browser dev walkthrough, confirmation that the local-first SDK path bypasses cloud Chromium, and independent hands-on confirmation of local-only operation.

   • [claimed-docs] “Self-hosted Skyvern runs entirely on your infrastructure: your servers, your browsers, your LLM API keys.”
   • [claimed-docs] “Run Skyvern on your own infrastructure with your own LLM keys.”
   • [claimed-docs] “Browser Automation is the code-first way to build multi-step automations. The Skyvern SDK connects to a cloud Chromium instance over CDP, la…”

   proof ↗⚑ flag

   Steel✓fullprobed9/10

   Steel Browser is open-source and can be self-hosted via Docker with no cloud account, confirmed by a runtime probe showing a local Docker container booting the browser API, creating live sessions, and working with the official CLI and SDK entirely locally. missing for 10: independent third-party (non-vendor) confirmation of long-term local dev workflow beyond the single recorded probe.

   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”
   • [github] “Pre-built Docker Image (combined API + UI)”
   • [github] “Uses Puppeteer and CDP for complete control over Chrome instances -- allowing you to connect using Puppeteer, Playwright, or Selenium.”
   • [probe] “official CLI documented at https://docs.steel.dev/overview/steel-cli”

   proof ↗⚑ flag

Frameworks

1. developerPlug the browser layer into agent frameworks (Claude Agent SDK, Vercel AI SDK, LangChain, CrewAI) through documented adapters

   weight 2 · round to Steel
   Skyvern—none0/10

   Skyvern documents a Python/TypeScript/REST SDK and an MCP server that plugs into Claude Desktop, Claude Code, Codex, Cursor, and Windsurf, but there is no evidence of documented adapters for Claude Agent SDK, Vercel AI SDK, LangChain, or CrewAI. missing for 10: any documented integration guide or adapter for LangChain, CrewAI, Vercel AI SDK, or Claude Agent SDK specifically.

   • [claimed-docs] “Integrate browser automation into your product with Python, TypeScript, or REST.”
   • [claimed-docs] “Browser Automation is the code-first way to build multi-step automations. The Skyvern SDK connects to a cloud Chromium instance over CDP, la…”
   • [claimed-docs] “The Skyvern MCP server lets AI assistants like Claude Desktop, Claude Code, Codex, Cursor, and Windsurf control a browser.”

   proof ↗⚑ flag

   Steel~partialclaimed5/10

   Steel documents a concrete integration with the Claude Agent SDK exposing its browser as in-process MCP tools (steel-docs-9), showing at least one first-party framework adapter exists. However, the evidence pack contains no documented adapters or integration guides for Vercel AI SDK, LangChain, or CrewAI, so the broader multi-framework claim is only partially substantiated. missing for 10: documented adapters for Vercel AI SDK, LangChain, and CrewAI, plus independent corroboration of any of these integrations working in practice.

   • [claimed-docs] “The Steel integration exposes a cloud browser as in-process MCP tools, so the SDK runs the agent loop and Steel handles the browser.”

   proof ↗⚑ flag

Models

1. developerBring my own LLM provider — the framework is model-agnostic rather than locked to one vendor's models

   weight 2 · round to Skyvern
   Skyvern~partialclaimed6/10

   Docs state self-hosted Skyvern runs with 'your own LLM API keys' on your own infrastructure, implying model-agnosticism rather than lock-in to a single vendor, but there is no explicit list of supported providers/models or first-party guide on swapping LLM backends, and no independent confirmation of multi-provider support. Missing for 10: an explicit supported-providers list/config docs, and community/hands-on evidence of using non-default LLMs.

   • [claimed-docs] “Run Skyvern on your own infrastructure with your own LLM keys.”
   • [claimed-docs] “Self-hosted Skyvern runs entirely on your infrastructure: your servers, your browsers, your LLM API keys.”

   proof ↗⚑ flag

   Steel—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

Tasks

1. ai agentSubmit a browser task over a hosted HTTP API and receive the result by polling or webhook, without managing any browser myself

   weight 2 · round to Skyvern
   Skyvern~partialprobed5/10

   Docs confirm a hosted REST/SDK API where you submit a prompt+URL (optionally a JSON schema) and Skyvern runs the task on cloud Chromium without the caller managing a browser (skyvern-docs-1, skyvern-docs-5, skyvern-docs-18, skyvern-docs-17). However, there is no explicit documentation of a polling endpoint or webhook callback mechanism, and probes found no discoverable OpenAPI spec, so the exact result-retrieval mechanism described in the story is unconfirmed. Missing for 10: explicit webhook/callback docs, explicit polling endpoint docs, and an accessible API reference confirming these mechanics.

   • [claimed-docs] “Integrate browser automation into your product with Python, TypeScript, or REST.”
   • [claimed-docs] “Browser Automation is the code-first way to build multi-step automations. The Skyvern SDK connects to a cloud Chromium instance over CDP, la…”
   • [claimed-docs] “You provide a natural-language prompt describing the goal, a starting URL, and optionally a JSON schema for structured output.”
   • [claimed-docs] “It navigates websites it has never seen before, filling forms, extracting data, and completing multi-step tasks via a simple API.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://skyvern.com/openapi.json, https://skyvern.com/swagger.json, https://skyvern.com/api/openapi.…”

   proof ↗⚑ flag

   Steel—none0/10

   Steel's docs describe a Sessions API that hands agents a raw, controllable browser (via CDP/Puppeteer/Playwright) plus a CLI for scripted step-by-step actions, but there is no evidence of a higher-level 'submit a task, poll or get a webhook for the result' abstraction — the agent still must drive the browser session itself rather than delegate a task and retrieve a finished output.

   • [claimed-docs] “the Sessions API lets your agents spin up isolated browser instances on demand. Each session maintains its own state, cookies, and storage”
   • [claimed-docs] “The Steel CLI lets you run full browser workflows from the terminal, end-to-end. You can start a browser session, navigate pages, click/fill…”
   • [github] “Uses Puppeteer and CDP for complete control over Chrome instances -- allowing you to connect using Puppeteer, Playwright, or Selenium.”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”

   proof ↗⚑ flag

2. developerHand the product a natural-language goal and it completes a multi-step web task end to end — navigating, filling forms, and clicking through flows

   weight 3 · round to Steel
   Skyvern!disputedcontradicted5/10

   Skyvern's docs and GitHub strongly claim natural-language goal execution across novel, multi-step web flows (forms, logins, CAPTCHAs) via a single prompt/API (skyvern-docs-17, skyvern-docs-18, skyvern-gh-1), but a hands-on community test found it succeeded only on the 'happy path' and concretely failed on a real multi-step flow (costcotravel.com), struggling to hit a tab and failing to click a popup (skyvern-comm-2). This is a specific documented counter-example contradicting the 'completes multi-step task end to end' claim, not just general skepticism. Missing for 10: independent benchmark results, more hands-on trials showing consistent success on complex/unseen sites, and resolution of the reported failure case.

   • [claimed-docs] “It navigates websites it has never seen before, filling forms, extracting data, and completing multi-step tasks via a simple API.”
   • [claimed-docs] “You provide a natural-language prompt describing the goal, a starting URL, and optionally a JSON schema for structured output.”
   • [github] “Skyvern can operate on websites it's never seen before, as it's able to map visual elements to actions necessary to complete a workflow, wit…”
   • [community] “I played with the Geico example, and it seems to do a good job on the happy path. But I tried costcotravel.com... it struggled to hit the 'r…”

   proof ↗⚑ flag

   Steel~partialprobed5/10

   Steel provides the browser primitives (sessions, navigate/click/fill/extract via CLI or API, CDP control) that a multi-step web task requires, and its CLI/MCP integrations let external agent frameworks drive those actions from natural-language goals. However, Steel's own docs state the agent loop and NL reasoning are handled by the paired SDK (e.g., Claude Agent SDK), not by Steel itself — Steel 'handles the browser' while the SDK runs the reasoning loop, so Steel alone does not accept a raw NL goal and autonomously plan/execute it end to end. missing for 10: evidence of Steel natively parsing/planning from a raw NL instruction without an external agent/LLM orchestrating the steps, and independent hands-on proof of a full NL-driven multi-step flow completed unattended.

   • [claimed-docs] “The Steel CLI lets you run full browser workflows from the terminal, end-to-end. You can start a browser session, navigate pages, click/fill…”
   • [claimed-docs] “The Steel integration exposes a cloud browser as in-process MCP tools, so the SDK runs the agent loop and Steel handles the browser.”
   • [github] “Uses Puppeteer and CDP for complete control over Chrome instances -- allowing you to connect using Puppeteer, Playwright, or Selenium.”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”

   proof ↗⚑ flag

Workflows

1. automation-engineerCompose repeatable multi-step workflows with loops, conditionals, and parameters instead of one-shot prompts

   weight 2 · round to Skyvern
   Skyvern~partialclaimed5/10

   Skyvern clearly supports multi-step, repeatable workflows via both a code-first SDK and a visual no-code drag-and-drop builder (skyvern-docs-5, skyvern-docs-6, skyvern-docs-22), plus SOP-to-workflow generation and a browser recorder for building reusable automations (skyvern-docs-23, skyvern-docs-24). However, the evidence never explicitly documents loop constructs, conditional branching, or parameterized workflow inputs as first-class workflow-builder features. Missing for 10: explicit documentation of loop/iteration blocks, conditional/branching logic, and named/typed workflow parameters in the workflow builder.

   • [claimed-docs] “Browser Automation is the code-first way to build multi-step automations. The Skyvern SDK connects to a cloud Chromium instance over CDP, la…”
   • [claimed-docs] “Build multi-step automations visually in the Cloud UI with drag-and-drop blocks. No code required. Share templates across your team.”
   • [claimed-docs] “Visual workflow builder for non-developers — drag-and-drop, no code required”
   • [claimed-docs] “Browser recorder that converts manual actions into reusable automations”
   • [claimed-docs] “SOP upload — describe a process in plain English and Skyvern builds the workflow”
   • [github] “a no-code workflow builder to help both technical and non-technical users automate manual workflows on any website, replacing brittle or unr…”

   proof ↗⚑ flag

   Steel—none0/10

   Steel's docs describe browser session management, stealth, CLI scripting of sequential steps (create session → navigate → click/fill → extract → stop), and MCP tool exposure, but nothing in the evidence pack shows a workflow-composition layer with loops, conditionals, or parameterized branching — the CLI and SDK are linear step sequences, not a control-flow DSL. missing for 10: any documented loop/conditional constructs, parameterized workflow templates, or reusable multi-branch automation definitions.

   • [claimed-docs] “The Steel CLI lets you run full browser workflows from the terminal, end-to-end. You can start a browser session, navigate pages, click/fill…”
   • [claimed-docs] “Steel generates a session ID for you, but `create` also accepts one. Pass your own UUID when the ID has to exist before the browser does”
   • [github] “Uses Puppeteer and CDP for complete control over Chrome instances -- allowing you to connect using Puppeteer, Playwright, or Selenium.”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round drawn
   Skyvern~partialprobed6/10

   Skyvern's docs show a strong code-first path (Python/TS/REST SDKs, page.extract, workflow creation via API) that covers most core automation tasks also available in the dashboard, and MCP/REST access is documented. However, several UI-only tooling features (drag-and-drop visual builder, browser recorder, SOP upload, copilot chat) are described only as dashboard capabilities with no documented API equivalent, and no public OpenAPI/swagger spec was discoverable to confirm full API-UI parity. Missing for 10: documented API equivalents for recorder/SOP-upload/copilot-chat features, and a discoverable OpenAPI reference confirming complete parity.

   • [claimed-docs] “Integrate browser automation into your product with Python, TypeScript, or REST.”
   • [claimed-docs] “Use the dashboard to run tasks and build agents visually.”
   • [claimed-docs] “Browser Automation is the code-first way to build multi-step automations. The Skyvern SDK connects to a cloud Chromium instance over CDP, la…”
   • [claimed-docs] “Build multi-step automations visually in the Cloud UI with drag-and-drop blocks. No code required. Share templates across your team.”
   • [claimed-docs] “Visual workflow builder for non-developers — drag-and-drop, no code required”
   • [claimed-docs] “Browser recorder that converts manual actions into reusable automations”
   • [claimed-docs] “SOP upload — describe a process in plain English and Skyvern builds the workflow”
   • [claimed-docs] “Copilot chat for building and debugging workflows interactively”
   • [probe] “PROBE openapi: all candidate paths 404 (https://skyvern.com/openapi.json, https://skyvern.com/swagger.json, https://skyvern.com/api/openapi.…”

   proof ↗⚑ flag

   Steel~partialprobed6/10

   Steel is fundamentally API-first: the Sessions API, CAPTCHA/proxy/auth-reuse, human-in-the-loop debug URLs, agent traces, and even the CLI are all documented as thin wrappers over the same REST/OpenAPI surface (steel-docs-1..10, steel-probe-2, steel-probe-3), and the runtime probe confirms the API alone (no UI) can create/list live sessions end-to-end. However, there's no explicit evidence enumerating the dashboard/UI feature set and confirming 1:1 parity with the API — it's inferred rather than directly stated. Missing for 10: an explicit doc/comparison confirming every UI-only feature (e.g. dashboard analytics, trace viewer) is also exposed via API, and independent confirmation of full parity.

   • [claimed-docs] “the Sessions API lets your agents spin up isolated browser instances on demand. Each session maintains its own state, cookies, and storage”
   • [claimed-docs] “One flag covers detection, solving, and verification for the whole session”
   • [claimed-docs] “This is particularly useful for maintaining authenticated states across multiple sessions, helping your AI agents access protected resources…”
   • [claimed-docs] “Steel's debug URL feature allows you to implement human-in-the-loop workflows where users can directly interact with and control browser ses…”
   • [claimed-docs] “It turns the run into a timeline of agent activity, so you can see what happened without scrubbing through the whole recording.”
   • [claimed-docs] “The Steel CLI lets you run full browser workflows from the terminal, end-to-end. You can start a browser session, navigate pages, click/fill…”
   • [probe] “PROBE openapi: HTTP 200 at https://docs.steel.dev/openapi.json — contains "openapi" key”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round to Steel
   Skyvern~partialclaimed3/10

   Skyvern is open-source and self-hostable, meaning your data (artifacts, recordings, screenshots, network traffic) stays on your own infrastructure rather than being locked in a vendor's cloud, which implicitly supports data portability. However, there is no explicit documentation of a data export feature, standard open-format export (e.g., JSON/CSV bulk export of run history), or a stated 'leave with your data' workflow. Missing for 10: explicit export functionality/documentation, named open data formats, and any independent confirmation of successful data migration out of the platform.

   • [claimed-docs] “Self-hosted Skyvern runs entirely on your infrastructure: your servers, your browsers, your LLM API keys.”
   • [claimed-docs] “Every run automatically captures what happened: recordings of the browser session, screenshots at each step, the AI's reasoning, and network…”
   • [claimed-docs] “Run Skyvern on your own infrastructure with your own LLM keys.”

   proof ↗⚑ flag

   Steel~partialprobed5/10

   Steel's agent-traces feature explicitly supports exporting run data as markdown, JSON, or a ZIP with markdown+screenshots (steel-docs-7), and the product itself is open-source and self-hostable (steel-gh-4, steel-probe-rt-1), meaning users are never locked into a proprietary cloud and can run/keep everything themselves. However, there's no documented comprehensive 'export all my data' capability covering sessions, auth contexts, or account-level data beyond traces. Missing for 10: a documented full-account data export/portability feature covering sessions, auth states, and configs, not just trace recordings.

   • [claimed-docs] “Copy the run as markdown, download JSON, or grab a ZIP with markdown plus screenshots.”
   • [github] “Pre-built Docker Image (combined API + UI)”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”

   proof ↗⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round to Steel
   Skyvern!disputedcontradicted4/10

   Skyvern is described as open-source with a GitHub repo, and probe evidence confirms it self-identifies as 'open-source' (skyvern-probe-1), but community evidence directly contradicts full open-license access, noting the project is AGPL3 licensed, which is a legally open license but is called out as a practical non-starter/restrictive for many users (skyvern-comm-3). missing for 10: explicit statement of license terms in docs, confirmation of what percentage of the product (cloud vs self-hosted) is actually open-sourced, and independent corroboration that the full source is readable without restriction.

   • [github] “Skyvern can operate on websites it's never seen before, as it's able to map visual elements to actions necessary to complete a workflow, wit…”
   • [github] “a no-code workflow builder to help both technical and non-technical users automate manual workflows on any website, replacing brittle or unr…”
   • [probe] “PROBE llms.txt: HTTP 200 at https://skyvern.com/llms.txt # Skyvern > Skyvern is an open-source, AI-powered browser automation platform. It …”
   • [community] “Exciting stuff, my employer would be interested but it's AGPL3 licensed so it's a non-starter for them.”

   proof ↗⚑ flag

   Steel~partialprobed6/10

   The product's llms.txt explicitly states 'Steel is the open-source browser API for AI agents,' there's a public GitHub repo (steel-dev/steel-browser) with feature docs, and a runtime probe confirms the self-hostable OSS image can be pulled and run via Docker — all consistent with source availability. However, no evidence pack item names or shows a specific OSS license (e.g., MIT/AGPL) or a LICENSE file, so the exact openness terms are unconfirmed. Missing for 10: explicit license file/type citation, independent confirmation of license terms.

   • [github] “Uses Puppeteer and CDP for complete control over Chrome instances -- allowing you to connect using Puppeteer, Playwright, or Selenium.”
   • [github] “Pre-built Docker Image (combined API + UI)”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.steel.dev/llms.txt # Steel Documentation > Steel is the open-source browser API for AI agents — ma…”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”
   • [community] “beautiful docs + api ref! what are you using? (cool that you're doing open-source browserbase also, excited to check this out)”

   proof ↗⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round to Steel
   Skyvern✓fullprobed8/10

   Skyvern has a dedicated self-hosted docs page stating it 'runs entirely on your infrastructure: your servers, your browsers, your LLM API keys' (skyvern-docs-12, skyvern-docs-2), and community evidence confirms it is genuinely open-source (AGPL3) rather than just marketing language (skyvern-comm-3). Missing for 10: independent hands-on confirmation of a successful self-host deployment and clarity on how AGPL licensing affects commercial self-hosting use.

   • [claimed-docs] “Self-hosted Skyvern runs entirely on your infrastructure: your servers, your browsers, your LLM API keys.”
   • [claimed-docs] “Run Skyvern on your own infrastructure with your own LLM keys.”
   • [community] “Exciting stuff, my employer would be interested but it's AGPL3 licensed so it's a non-starter for them.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://skyvern.com/llms.txt # Skyvern > Skyvern is an open-source, AI-powered browser automation platform. It …”

   proof ↗⚑ flag

   Steel✓fullprobed10/10

   Steel is explicitly open-source with a pre-built Docker image (combined API + UI), and a runtime probe confirms a full keyless self-host roundtrip: docker-running the OSS image, health check succeeding, and creating/listing live browser sessions, corroborating vendor docs and GitHub claims.

   • [github] “Pre-built Docker Image (combined API + UI)”
   • [github] “Uses Puppeteer and CDP for complete control over Chrome instances -- allowing you to connect using Puppeteer, Playwright, or Selenium.”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.steel.dev/llms.txt # Steel Documentation > Steel is the open-source browser API for AI agents — ma…”

   proof ↗⚑ flag

Pricing

1. developerSee transparent per-task or per-browser-hour pricing and documented rate/concurrency limits before committing

   weight 2 · round drawn
   Skyvern—none0/10

   The pricing page is referenced only for its target-audience blurb (skyvern-docs-13); no evidence pack item shows actual per-task or per-browser-hour rates, tiers, or documented rate/concurrency limits. Community comments only express general cost concerns ('pretty pricey', wanting cost down 'at scale') without citing concrete published pricing or limits.

   • [claimed-docs] “You're replacing brittle Selenium scripts, integrating browser automation via API, or building workflows into your product.”
   • [community] “I tried it out and it's pretty pricey. My OpenAI API bill is $3.20 after using this on a few different pages to test it out... this is alway…”
   • [community] “This is an impressive tool. I especially like the observability around the workflow and the steps it takes to achieve the outcome. We are po…”

   proof ↗⚑ flag

   Steel—none0/10

   No first-party documentation in the evidence pack lays out per-task or per-browser-hour pricing or concurrency/rate limits; the only pricing-related evidence is a community report of inconsistency between the pricing page and docs pricing ($59 vs $99), which itself signals the opposite of transparent, dependable pricing rather than confirming it.

   • [community] “Looking interesting, will definitely give it a go. Btw, there is inconsistency between pricing page and pricing on docs. Pricing page for de…”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round to Skyvern
   Skyvern~partialclaimed5/10

   Skyvern offers a self-hosted deployment mode where 'your servers, your browsers, your LLM API keys' run entirely on the user's own infrastructure, which lets an AI-native user control where data resides by choosing their hosting region themselves — but this is achieved only by self-hosting, not via an explicit region/residency selector in the managed cloud product. Missing for 10: documented data residency/region options in the hosted Skyvern Cloud offering, compliance certifications, or explicit multi-region storage controls.

   • [claimed-docs] “Run Skyvern on your own infrastructure with your own LLM keys.”
   • [claimed-docs] “Self-hosted Skyvern runs entirely on your infrastructure: your servers, your browsers, your LLM API keys.”

   proof ↗⚑ flag

   Steel~partialprobed4/10

   Steel offers a self-hostable open-source Docker image (steel-gh-4, steel-probe-rt-1) which lets users control where their data physically resides by hosting it themselves, but there is no explicit region/residency selection feature documented for the managed cloud offering. missing for 10: explicit region-selection UI/API for the managed cloud service, documentation on data residency guarantees or compliance certifications (e.g., GDPR/SOC2 region controls).

   • [github] “Pre-built Docker Image (combined API + UI)”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”

   proof ↗⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round to Skyvern
   Skyvern~partialclaimed4/10

   Skyvern offers self-hosted deployment using your own infrastructure and your own LLM API keys, which implicitly lets users avoid sending data to Skyvern-controlled models/training pipelines, but there is no explicit privacy policy, data-retention statement, or 'we do not train on your data' commitment in the evidence for the hosted/cloud offering. Missing for 10: explicit no-training/data-use policy documentation, opt-out mechanism for the cloud product, and independent confirmation of data handling practices.

   • [claimed-docs] “Run Skyvern on your own infrastructure with your own LLM keys.”
   • [claimed-docs] “Self-hosted Skyvern runs entirely on your infrastructure: your servers, your browsers, your LLM API keys.”

   proof ↗⚑ flag

   Steel—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round drawn
   Skyvern~partialcommunity4/10

   Skyvern offers self-hosting (docs-2, docs-12) which gives infrastructure-level control over where data lives, and it captures artifacts (recordings, screenshots, network traffic) per run (docs-11), implying some data exists to manage, but there is no documented retention policy, data deletion API/UI, or export/purge controls for the cloud/hosted product. missing for 10: explicit data retention policy, user-facing deletion/export controls, documentation on how long artifacts/credentials are stored in cloud mode, and independent confirmation that self-hosting actually eliminates vendor-side data retention.

   • [claimed-docs] “Run Skyvern on your own infrastructure with your own LLM keys.”
   • [claimed-docs] “Self-hosted Skyvern runs entirely on your infrastructure: your servers, your browsers, your LLM API keys.”
   • [claimed-docs] “Every run automatically captures what happened: recordings of the browser session, screenshots at each step, the AI's reasoning, and network…”
   • [community] “you are expecting them to pass over their website login credentials and apparently their credit card details too, in plain text. You had bet…”

   proof ↗⚑ flag

   Steel~partialprobed4/10

   Steel is open-source and self-hostable (steel-gh-4, steel-probe-rt-1), which gives users full control over where session data lives and how long it's retained, and session lifecycle docs show sessions can be created/stopped with custom IDs (steel-docs-10). However, there is no explicit documentation of a retention policy, data-deletion API, or GDPR-style controls for the managed cloud offering. Missing for 10: documented retention/deletion controls or policy for the hosted cloud service, explicit data-purge API, independent confirmation of retention behavior.

   • [github] “Pre-built Docker Image (combined API + UI)”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”
   • [claimed-docs] “Steel generates a session ID for you, but `create` also accepts one. Pass your own UUID when the ID has to exist before the browser does”

   proof ↗⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Skyvern—none0/10

   No evidence pack item discusses telemetry, usage tracking, or an opt-out mechanism; while self-hosting exists, there is no explicit statement about data collection or opt-out controls for the cloud/hosted product. missing for 10: any mention of telemetry collection, privacy policy on usage data, or an opt-out setting/flag.

   ⚑ flag

   Steel—none0/10

   No evidence pack item mentions telemetry, analytics, usage tracking, or an opt-out mechanism; the self-hostable OSS nature is noted (steel-probe-rt-1) but that alone does not confirm any telemetry opt-out control exists or is documented.

   ⚑ flag

Live

1. automation-engineerWatch a session live and take human control mid-run when the agent gets stuck

   weight 2 · round drawn
   Skyvern✓fullclaimed8/10

   Docs explicitly state a VNC stream lets you watch a live session and take control at any point, plus pause-for-approval human-in-the-loop flows that preserve browser state — directly matching the story. missing for 10: independent/hands-on confirmation of the live takeover UX and details on how control handoff works mid-run beyond the docs description.

   • [claimed-docs] “Human-in-the-loop flows: pause for approval between steps without losing browser state. The VNC stream lets you watch or take control at any…”
   • [claimed-docs] “Cookies, local storage, open tabs, and the current page all persist, so later operations pick up exactly where the previous one stopped.”

   proof ↗⚑ flag

   Steel✓fullprobed8/10

   Steel's debug URL feature is explicitly documented for human-in-the-loop workflows enabling users to directly interact with and control a live browser session, and runtime proof confirms sessions expose a live debugger/websocket URL for real-time viewing/control. The agent-traces timeline feature complements this by letting engineers review what happened, though it's more post-hoc than live takeover. Missing for 10: explicit documentation of mid-run handoff back to the agent after human control, and independent/community corroboration of the human-in-the-loop debug feature specifically (vs. general product commentary).

   • [claimed-docs] “Steel's debug URL feature allows you to implement human-in-the-loop workflows where users can directly interact with and control browser ses…”
   • [claimed-docs] “It turns the run into a timeline of agent activity, so you can see what happened without scrubbing through the whole recording.”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”

   proof ↗⚑ flag

Replay

1. automation-engineerDebug a failed agent run from recorded replays — video, screenshots, step-by-step action timelines

   weight 2 · round to Skyvern
   Skyvern✓fullcommunity8/10

   Skyvern docs explicitly state every run captures session recordings, per-step screenshots, AI reasoning traces, and network traffic for debugging, and community feedback corroborates strong observability into workflow steps. missing for 10: independent hands-on verification of the video/timeline UI itself, and no mention of a true step-by-step interactive timeline scrubber beyond artifact capture.

   • [claimed-docs] “Every run automatically captures what happened: recordings of the browser session, screenshots at each step, the AI's reasoning, and network…”
   • [community] “This is an impressive tool. I especially like the observability around the workflow and the steps it takes to achieve the outcome. We are po…”

   proof ↗⚑ flag

   Steel✓fullclaimed7/10

   Steel's Agent Traces feature explicitly turns a run into a timeline of agent activity with screenshots, and lets you export as markdown, JSON, or a ZIP with markdown+screenshots, directly matching the replay-debugging story for automation engineers. Missing for 10: explicit video recording/playback evidence and independent/hands-on corroboration of the traces UI beyond first-party docs.

   • [claimed-docs] “It turns the run into a timeline of agent activity, so you can see what happened without scrubbing through the whole recording.”
   • [claimed-docs] “Copy the run as markdown, download JSON, or grab a ZIP with markdown plus screenshots.”
   • [claimed-docs] “Steel's debug URL feature allows you to implement human-in-the-loop workflows where users can directly interact with and control browser ses…”

   proof ↗⚑ flag

Fleets

1. automation-engineerRun a fleet of concurrent browser sessions with documented concurrency limits and programmatic session management

   weight 2 · round to Steel
   Skyvern—none0/10

   Evidence covers session persistence, VNC control, self-hosting, and SDK/API access, but nowhere documents concurrency limits, fleet-level session orchestration, or programmatic management of multiple simultaneous browser sessions. Missing for 10: documented concurrency limits, APIs for spinning up/managing many parallel sessions, and any scaling/throughput guidance.

   • [claimed-docs] “Human-in-the-loop flows: pause for approval between steps without losing browser state. The VNC stream lets you watch or take control at any…”
   • [claimed-docs] “Cookies, local storage, open tabs, and the current page all persist, so later operations pick up exactly where the previous one stopped.”
   • [claimed-docs] “Browser Automation is the code-first way to build multi-step automations. The Skyvern SDK connects to a cloud Chromium instance over CDP, la…”
   • [claimed-docs] “Self-hosted Skyvern runs entirely on your infrastructure: your servers, your browsers, your LLM API keys.”

   proof ↗⚑ flag

   Steel~partialprobed6/10

   Steel's Sessions API and CLI clearly support spinning up isolated on-demand browser sessions with custom session IDs, and a runtime probe confirms working session creation/listing via self-hosted API — solid programmatic session management. However, the evidence pack contains no documented concurrency limits, quotas, or fleet-scale guidance for running many sessions in parallel. Missing for 10: explicit documented concurrency/session limits, guidance or examples for orchestrating many simultaneous sessions, and independent corroboration of scale behavior.

   • [claimed-docs] “the Sessions API lets your agents spin up isolated browser instances on demand. Each session maintains its own state, cookies, and storage”
   • [claimed-docs] “Steel generates a session ID for you, but `create` also accepts one. Pass your own UUID when the ID has to exist before the browser does”
   • [claimed-docs] “The Steel CLI lets you run full browser workflows from the terminal, end-to-end. You can start a browser session, navigate pages, click/fill…”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/browser-agents/proofs/steel/): full keyless self-host roundtrip — `docker run ghcr.io/steel-dev…”

   proof ↗⚑ flag

Lifecycle

1. developerGet webhook notifications when tasks and sessions finish instead of polling for status

   weight 1 · round drawn
   Skyvern—none0/10

   No evidence pack item mentions webhooks, callback URLs, or push notifications for task/session completion; the docs discuss artifacts, VNC streaming, and human-in-the-loop review but nothing about event-driven notification instead of polling. missing for 10: any documentation of webhook/callback support, event subscription API, or notification configuration.

   ⚑ flag

   Steel—none0/10

   No evidence pack item mentions webhooks, event callbacks, or push notifications for session/task completion; Steel's docs cover sessions API, CLI, proxies, captcha, and traces but nothing about async status notifications, implying developers must poll session status endpoints.

   ⚑ flag

Captcha

1. automation-engineerRely on a documented captcha stance — automatic solving, human fallback, or explicit non-support — instead of silent task failures

   weight 2 · round drawn
   Skyvern✓fullclaimed8/10

   Skyvern's docs give an explicit, detailed captcha stance: automatic detection and solving via its vision model for reCAPTCHA v2/v3, hCaptcha, Cloudflare Turnstile, FunCaptcha, MTCaptcha, and text/image captchas, avoiding silent failure ambiguity. Missing for 10: independent/hands-on confirmation that captcha solving works reliably in practice (community evidence discusses pricing, mobile UX, and credential handling but not captcha outcomes specifically), and no documented fallback/human-in-the-loop behavior specifically tied to captcha failures.

   • [claimed-docs] “Skyvern detects CAPTCHAs using its vision model and solves them automatically. This works for reCAPTCHA v2/v3, hCaptcha, Cloudflare Turnstil…”
   • [claimed-docs] “Skyvern detects CAPTCHAs using its vision model and solves them automatically.”

   proof ↗⚑ flag

   Steel✓fullclaimed8/10

   Steel documents a single explicit flag covering captcha detection, solving, and verification for the whole session, plus a separate human-in-the-loop debug URL feature for manual takeover when needed — giving automation engineers a documented stance rather than silent failures. Missing for 10: independent/hands-on verification that automatic captcha solving actually succeeds on real-world captchas, and clearer documentation of failure/fallback behavior when auto-solve fails.

   • [claimed-docs] “One flag covers detection, solving, and verification for the whole session”
   • [claimed-docs] “Steel's debug URL feature allows you to implement human-in-the-loop workflows where users can directly interact with and control browser ses…”

   proof ↗⚑ flag

Posture

1. automation-engineerPoint to the vendor's published acceptable-use and anti-abuse posture governing what its stealth and automation features may be used for

   weight 1 · round drawn
   Skyvern—none0/10

   No evidence in the pack of any published acceptable-use policy, terms of service, or anti-abuse statement covering CAPTCHA-solving/stealth automation features; docs describe capabilities (CAPTCHA bypass, bot bypass) but no governance/AUP language is cited. missing for 10: a published acceptable-use policy, anti-abuse terms, or statement on permitted use of stealth/CAPTCHA-bypass features.

   ⚑ flag

   Steel—none0/10

   The evidence pack documents Steel's stealth/captcha-solving, proxy, and CAPTCHA features extensively, but contains no published acceptable-use policy, terms of service, or anti-abuse statement governing what these stealth capabilities may be used for. No AUP, ToS, or anti-abuse page is cited or referenced anywhere in the docs, GitHub repo, or community discussion. Missing for 10: a published acceptable-use policy, anti-abuse/misuse guidelines, or ToS language specifically addressing stealth/captcha feature usage.

   ⚑ flag

Stealth

1. automation-engineerEnable stealth fingerprinting and residential or geo-targeted proxies so legitimate automations aren't blocked as bots

   weight 2 · round to Steel
   Skyvern—none0/10

   Evidence covers CAPTCHA solving and authentication/2FA handling, but there is no mention anywhere of stealth fingerprinting, browser fingerprint spoofing, or residential/geo-targeted proxy support. Missing for 10: any documentation of proxy configuration, geo-targeting, or anti-fingerprinting/stealth mode features.

   • [claimed-docs] “Skyvern detects CAPTCHAs using its vision model and solves them automatically. This works for reCAPTCHA v2/v3, hCaptcha, Cloudflare Turnstil…”
   • [claimed-docs] “Skyvern detects CAPTCHAs using its vision model and solves them automatically.”

   proof ↗⚑ flag

   Steel✓fullcommunity8/10

   Steel's docs explicitly document stealth mode (single flag covering detection evasion, captcha solving, and verification) plus both Managed Residential Proxies and Bring-Your-Own-Proxy (BYOP) for geo-targeting, directly matching the story, and community comment confirms custom proxy support works in practice. Missing for 10: no independent/hands-on evidence quantifying bot-detection bypass success rates or geo-targeting granularity, and no first-party benchmark showing reduced block rates.

   • [claimed-docs] “One flag covers detection, solving, and verification for the whole session”
   • [claimed-docs] “Steel offers two powerful ways to use proxies: our built-in **Managed Residential Proxies** or connecting to your own proxy provider with ou…”
   • [community] “It appears you can set your own proxies to not use their cloud.”

   proof ↗⚑ flag

Extraction

1. developerExtract typed, schema-validated data (Zod/Pydantic-style) from pages the agent visits, not just raw text

   weight 3 · round to Skyvern
   Skyvern✓fullclaimed7/10

   Skyvern's docs explicitly support structured, schema-based extraction via `page.extract` with a JSON schema or `data_extraction_schema` param, matching the developer's need for typed output rather than raw text (skyvern-docs-4, skyvern-docs-20, skyvern-docs-18). However, evidence only shows JSON-schema validation, not native Zod/Pydantic model binding, and there's no independent/hands-on confirmation of this specific feature. missing for 10: explicit Zod/Pydantic model integration examples, independent verification of extraction accuracy/schema enforcement.

   • [claimed-docs] “you can extract structured data from any page using `page.extract` with a JSON schema, or by passing a `data_extraction_schema` to `page.age…”
   • [claimed-docs] “you can extract structured data from any page using page.extract with a JSON schema”
   • [claimed-docs] “You provide a natural-language prompt describing the goal, a starting URL, and optionally a JSON schema for structured output.”
   • [claimed-docs] “Browser Automation is the code-first way to build multi-step automations. The Skyvern SDK connects to a cloud Chromium instance over CDP, la…”

   proof ↗⚑ flag

   Steel—none0/10

   Steel's docs/GitHub only show raw content extraction utilities (markdown, readability, screenshots, PDF conversion) and generic 'extract content' CLI commands, with no mention of Zod/Pydantic-style schema validation or typed structured outputs. The axis is fair for a browser-automation API since competitors offer schema-based extraction, but no evidence shows Steel provides this.

   • [github] “Browser Tools: Exposes APIs to quick convert pages to markdown, readability, screenshots, or PDFs.”
   • [claimed-docs] “The Steel CLI lets you run full browser workflows from the terminal, end-to-end. You can start a browser session, navigate pages, click/fill…”

   proof ↗⚑ flag

Files

1. developerMy agent can download files from and upload files to the sites it operates, with the artifacts retrievable afterwards

   weight 1 · round to Skyvern
   Skyvern~partialclaimed5/10

   Docs show Skyvern can log into vendor portals and download PDFs (skyvern-docs-15) and captures per-run artifacts like recordings, screenshots, and network traffic retrievable afterward (skyvern-docs-11), implying file download support, but there is no explicit documentation of file upload capability to sites, nor of a dedicated API/UI for retrieving downloaded artifacts as opposed to just run/debug artifacts. missing for 10: explicit upload-to-site capability documentation, a documented file-download/artifact storage API distinct from debugging screenshots, and independent/hands-on confirmation of file transfer working in practice.

   • [claimed-docs] “Log into vendor portals, find invoices, download PDFs.”
   • [claimed-docs] “Every run automatically captures what happened: recordings of the browser session, screenshots at each step, the AI's reasoning, and network…”
   • [claimed-docs] “Auto-fill and submit applications on Lever, Greenhouse, and more.”

   proof ↗⚑ flag

   Steel—none0/10

   The evidence covers session artifacts (traces, screenshots, page-to-markdown conversion) and CLI-driven browser control, but nothing documents actual file upload to web forms or downloading files from a site with persistent artifact retrieval — a distinct capability from trace export.

   ⚑ flag