Skip to content
ProductArena

Browser Automation for Agents Arena

Browser Automation for Agents — procurement report

ProductArena · rankings as of 2026-09-16 · evidence as of 2026-09-16 · 7 products · 52 judged requirements · 364 judged cells

Methodology: Every product is judged against a shared taxonomy of user stories using cited evidence — hands-on probes > repository code > independent community sources > vendor claims — never opinion. Full writeup: https://ultrametric.ai/productarena/methodology

Leaderboard

#ProductPA ScoreCoverage scoreApplicable cellsConfidence
1Steel40.535.348/52B
2Skyvern30.832.351/52B
3Hyperbrowser29.729.451/52C
4Stagehand28.131.147/52B
5Notte24.426.952/52B
6Browser Use23.329.350/52B
7Smooth15.920.949/52B

PA Score = agent-readiness blend (see methodology). Coverage score = weighted share of judged requirements met. Confidence = how much of the score rests on tested vs claimed evidence (A–D).

Uncertainty note

The current #1/#2 gap in this arena is not close enough to qualify for the multi-judge uncertainty pass — no extra caveat applies beyond the per-product confidence grades above.

Buyer checklist (RFP)

The arena's 52 judged user stories as requirements, grouped by theme. Priorities mirror the story weights our scoring uses (3 = must-have, 2 = should-have, 1 = nice-to-have). Interactive version with per-requirement verdicts for the top products: /arena/browser-agents/checklist

Action primitives — stories about action primitives in this arenaAction primitives

Stories about action primitives in this arena

  • developerDrive the page through DOM-understanding action primitives (act/click/type on described elements) that survive selector and layout changesmust-have
  • developerCache resolved actions or generated code so repeat runs replay deterministically at lower cost and latency than re-prompting the LLMshould-have
  • developerSwitch to a vision or computer-use action mode that operates on screenshots for canvases and UIs the DOM path can't handleshould-have
  • developerPreview candidate actions on the current page (observe/plan) before committing the agent to actnice-to-have

Agenticness — how well agents can access and operate the productAgenticness

How well agents can access and operate the product

  • ai-native userPlug MCP servers into this product so it can use their toolsmust-have
  • ai-native userConnect an agent via an official MCP servermust-have
  • ai-native userDrive the product through a documented public APImust-have
  • ai-native userDelegate tasks to a built-in AI assistant inside the productmust-have
  • ai-native userPoint an agent at llms.txt or agent-oriented docsshould-have
  • ai-native userRun the product headlessly / in CI for automationshould-have
  • ai-native userUse an official CLIshould-have
  • ai-native userIssue scoped/least-privilege API credentials for an agentshould-have
  • ai-native userBuild against official SDKsshould-have
  • ai-native userSubscribe to events via webhooksshould-have
  • ai-native userGet AI-generated insights and suggestions from my data inside the productshould-have
  • ai-native userSet up automations that run autonomously in the backgroundshould-have
  • ai-native userOperate the product with natural-language commandsshould-have
  • ai-native userExplore an interactive API reference with runnable examplesshould-have
  • ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)should-have
  • ai-native userRely on versioned APIs with a documented deprecation policyshould-have
  • ai-native userTest against a sandbox environment without touching production datanice-to-have

Auth session persistence — stories about auth session persistence in this arenaAuth session persistence

Stories about auth session persistence in this arena

  • developerPersist logged-in browser state in reusable profiles so agents skip the login wall on every subsequent runmust-have
  • developerConnect my existing Playwright, Puppeteer, or CDP automation code to the product's browsers instead of rewriting itshould-have
  • automation-engineerStore credentials in a vault and have the agent complete logins including TOTP/2FA challenges without exposing secrets to the modelshould-have

Automation depth — how much of the product can run unattendedAutomation depth

How much of the product can run unattended

  • ai-native userDefine rules that trigger actions automatically on eventsmust-have
  • ai-native userPerform bulk operations across many items at onceshould-have
  • ai-native userSchedule recurring jobs or workflowsshould-have
  • ai-native userVersion, review, and roll back my automationsnice-to-have

Deployment modes — stories about deployment modes in this arenaDeployment modes

Stories about deployment modes in this arena

  • developerRun the agent against a local browser on my own machine for development, without any cloud accountshould-have

Framework model support — stories about framework model support in this arenaFramework model support

Stories about framework model support in this arena

  • developerPlug the browser layer into agent frameworks (Claude Agent SDK, Vercel AI SDK, LangChain, CrewAI) through documented adaptersshould-have
  • developerBring my own LLM provider — the framework is model-agnostic rather than locked to one vendor's modelsshould-have

Nl task execution — stories about nl task execution in this arenaNl task execution

Stories about nl task execution in this arena

  • developerHand the product a natural-language goal and it completes a multi-step web task end to end — navigating, filling forms, and clicking through flowsmust-have
  • ai agentSubmit a browser task over a hosted HTTP API and receive the result by polling or webhook, without managing any browser myselfshould-have
  • automation-engineerCompose repeatable multi-step workflows with loops, conditionals, and parameters instead of one-shot promptsshould-have

Openness — open source, data portability, and self-hosting storiesOpenness

Open source, data portability, and self-hosting stories

  • ai-native userExport all of my data in open formats and leavemust-have
  • ai-native userSelf-host the core productmust-have
  • ai-native userDo everything through the API that I can do in the UIshould-have
  • ai-native userRead the product's source under an open licenseshould-have

Pricing limits — free-tier ceilings, usage caps, and rate limits before you have to payPricing limits

Free-tier ceilings, usage caps, and rate limits before you have to pay

  • developerSee transparent per-task or per-browser-hour pricing and documented rate/concurrency limits before committingshould-have

Privacy posture — data-handling and privacy storiesPrivacy posture

Data-handling and privacy stories

  • ai-native userPrevent my data from being used to train AI modelsmust-have
  • ai-native userChoose where my data is stored (region/residency)should-have
  • ai-native userControl data retention and deletionshould-have
  • ai-native userOpt out of telemetry and usage trackingshould-have

Replay debugging — stories about replay debugging in this arenaReplay debugging

Stories about replay debugging in this arena

  • automation-engineerWatch a session live and take human control mid-run when the agent gets stuckshould-have
  • automation-engineerDebug a failed agent run from recorded replays — video, screenshots, step-by-step action timelinesshould-have

Scale parallelism — running many jobs at once — concurrency, fleets, queueingScale parallelism

Running many jobs at once — concurrency, fleets, queueing

  • automation-engineerRun a fleet of concurrent browser sessions with documented concurrency limits and programmatic session managementshould-have
  • developerGet webhook notifications when tasks and sessions finish instead of polling for statusnice-to-have

Stealth captcha — stories about stealth captcha in this arenaStealth captcha

Stories about stealth captcha in this arena

  • automation-engineerRely on a documented captcha stance — automatic solving, human fallback, or explicit non-support — instead of silent task failuresshould-have
  • automation-engineerEnable stealth fingerprinting and residential or geo-targeted proxies so legitimate automations aren't blocked as botsshould-have
  • automation-engineerPoint to the vendor's published acceptable-use and anti-abuse posture governing what its stealth and automation features may be used fornice-to-have

Structured extraction — stories about structured extraction in this arenaStructured extraction

Stories about structured extraction in this arena

  • developerExtract typed, schema-validated data (Zod/Pydantic-style) from pages the agent visits, not just raw textmust-have
  • developerMy agent can download files from and upload files to the sites it operates, with the artifacts retrievable afterwardsnice-to-have

Appendix: recorded probes

Hands-on probe recordings — transcripts/videos a human can replay, the strongest evidence tier. Watch them at https://ultrametric.ai/productarena/proofs

  • Browser Useecho '<jsonrpc initialize>' | uvx --from 'browser-use[cli]' browser-use --mcpterminal · recorded 2026-09-05 · exit 0
  • Browser Usemktemp -d && uv venv && uv pip install browser-use && python -c "import browser_use; print('PA_PROBE_OK browser-use', version('browser-use'))"terminal · recorded 2026-09-05 · exit 0
  • Hyperbrowserecho '<jsonrpc initialize>' | npx -y hyperbrowser-mcpterminal · recorded 2026-09-05 · exit 0
  • Hyperbrowsermktemp -d && npm install @hyperbrowser/sdk && node -e "console.log('PA_PROBE_OK Hyperbrowser export:', typeof require('@hyperbrowser/sdk').Hyperbrowser)"terminal · recorded 2026-09-05 · exit 0
  • Nottecurl -s https://docs.notte.cc/llms.txt | head -6terminal · recorded 2026-09-10 · exit 0
  • Nottecurl -si -X POST https://api.notte.cc/mcp/ -H 'Content-Type: application/json' -d '<jsonrpc initialize>'terminal · recorded 2026-09-10 · exit 0
  • Skyverncurl -si -X POST https://api.skyvern.com/mcp -H 'Content-Type: application/json' -d '<jsonrpc initialize>'terminal · recorded 2026-09-05 · exit 0
  • Skyvernmktemp -d && uv venv && uv pip install skyvern && skyvern --helpterminal · recorded 2026-09-05 · exit 0
  • Smoothcurl -s https://docs.smooth.sh/llms.txt | head -6terminal · recorded 2026-09-14 · exit 0
  • Smoothcurl -s https://docs.smooth.sh/api-reference/openapi.json | head -c 200terminal · recorded 2026-09-14 · exit 0
  • Smoothmktemp -d && uv venv && uv pip install smooth-py && python -c "print('PA_PROBE_OK smooth-py', version('smooth-py'))" && smooth --helpterminal · recorded 2026-09-14 · exit 0
  • Stagehandecho '<jsonrpc initialize>' | npx -y @browserbasehq/mcpterminal · recorded 2026-09-05 · exit 0
  • Stagehandmktemp -d && npm install @browserbasehq/stagehand && node -e "import('@browserbasehq/stagehand').then(m=>console.log('PA_PROBE_OK Stagehand export:', typeof m.Stagehand))"terminal · recorded 2026-09-05 · exit 0
  • SteelHOME=$(mktemp -d) sh -c 'curl -sSf https://setup.steel.dev | sh -s -- --non-interactive && ~/.steel/bin/steel --version'terminal · recorded 2026-09-05 · exit 0
  • Steeldocker run -d --name pa-steel-probe -p 13000:3000 ghcr.io/steel-dev/steel-browser && curl localhost:13000/v1/health && curl -X POST localhost:13000/v1/sessions -d '{}' && curl localhost:13000/v1/sessionsterminal · recorded 2026-09-05 · exit 0
  • Steelmktemp -d && npm install steel-sdk && node -e "console.log('PA_PROBE_OK Steel export:', typeof require('steel-sdk').Steel)"terminal · recorded 2026-09-05 · exit 0

Cite as: ProductArena by Ultrametric Inc, Browser Automation for Agents arena, rankings as of 2026-09-16 — https://ultrametric.ai/productarena/arena/browser-agents

License: © 2026 Ultrametric Inc. Brief quotation of individual verdicts, scores, or evidence excerpts is permitted with attribution to "ProductArena by Ultrametric Inc (ultrametric.ai/productarena)", as is use of the data to evaluate, contest, or contribute corrections. Bulk copying, redistribution, or use to build competing datasets requires prior written permission (see DATA-LICENSE in the repository).

No liability: rankings, verdicts, and scores are research outputs derived from the cited evidence at a point in time, provided "as is", without warranties. Ultrametric Inc accepts no responsibility for procurement, purchasing, or other decisions made in reliance on them — verify against the cited evidence before acting (https://ultrametric.ai/productarena/terms).