ScrapingBee vs Riveter
ScrapingBee
ScrapingBee
ScrapingBee wins · 28–25 (40 drawn)
Agenticness — how well agents can access and operate the productAgenticness
How well agents can access and operate the product
Agent access
ai-native userPoint an agent at llms.txt or agent-oriented docs
weight 2 · round to ScrapingBeeScrapingBee has a confirmed live llms.txt at the documented URL (HTTP 200) serving an index of its documentation, directly satisfying the story of pointing an agent at agent-oriented docs; it also has an official MCP server further supporting agentic access. Missing for 10: a per-page markdown/docs.md mirror (probe shows 404) and an OpenAPI spec discoverable at standard paths, which would round out machine-readable doc coverage.
- [probe] “PROBE llms.txt: HTTP 200 at https://www.scrapingbee.com/llms.txt # ScrapingBee Documentation > Official documentation index for ScrapingBee…”
- [probe] “PROBE docs-md: HTTP 404 at https://www.scrapingbee.com/documentation.md”
- [probe] “official MCP server documented at https://mcp.scrapingbee.com/”
Riveternone0/10Direct probes show llms.txt returns 404 and no OpenAPI spec is discoverable at any standard path, and no evidence pack item claims an agent-oriented docs format exists; while MCP integration is mentioned, that's a separate capability from machine-readable docs for pointing an agent at.
ai-native userRun the product headlessly / in CI for automation
weight 2 · round to ScrapingBeeScrapingBee is a pure API service (API key + HTTP calls) designed for programmatic use, and it ships an official CLI (scrapingbee-cli) enabling scripted/headless invocation suitable for CI pipelines. Missing for 10: explicit CI/CD pipeline examples (e.g., GitHub Actions), and no independent hands-on confirmation of CLI use in automation contexts.
- [claimed-docs] “To scrape a web page, you only need two things: Your API key... The encoded web page URL you want to scrape”
- [probe] “official CLI documented at https://github.com/ScrapingBee/scrapingbee-cli”
Riveter exposes a full API with SDKs (Go example shown), webhooks for async completion, dry_run/max_credits safety controls, and scheduling for recurring automation — all of which support headless, non-interactive use in a pipeline. However, there is no explicit CI/CD example, GitHub Actions integration, or CLI documentation demonstrating a documented headless workflow. Missing for 10: explicit CI/CD or pipeline integration guide, CLI headless invocation docs, independent confirmation of automated/scripted runs.
- [claimed-docs] “Pass webhook_url in the JSON body when starting a run and Riveter POSTs the full results to your URL when it finishes (events: run.completed…”
- [claimed-docs] “dry_run: true — validate the request and return a credit estimate without creating or charging anything.”
- [claimed-docs] “max_credits — a ceiling. When the estimate's maximum exceeds it, the request is refused with 422 credit_cap_exceeded and nothing is charged.”
- [claimed-docs] “the SDKs handle auth, retries (429s and transient GET failures), the wait long-poll, polling until a run finishes (wait_for_result), and pag…”
- [claimed-docs] “run, err := client.Enrich(ctx, riveter.EnrichParams{ Prompt: "Research each company", Attributes: []string{"CEO", "Employee Count"},”
- [claimed-docs] “Schedule any project to monitor for changes and keep your data fresh.”
ai-native userConnect an agent via an official MCP server
weight 3 · round to RiveterScrapingBee is a scraping API/service (not itself an agent), so publishing an official MCP server is a valid axis; a probe confirms an official MCP server is documented at mcp.scrapingbee.com. However, there is no first-party documentation detail on the MCP server's tool set, setup, or independent/community corroboration of it working. Missing for 10: detailed official docs on MCP server capabilities/setup, independent hands-on confirmation.
- [probe] “official MCP server documented at https://mcp.scrapingbee.com/”
Docs explicitly describe connecting Riveter to Claude, ChatGPT, Cursor, or any MCP-compatible assistant via two connection methods, including a local Node.js-based server option, indicating an official MCP server offering. Missing for 10: no independent/hands-on corroboration of the MCP server working, and no detail on the remote/hosted connection method's implementation.
- [claimed-docs] “Use Riveter from Claude, ChatGPT, Cursor, or any MCP-compatible AI assistant. Pick one of the two ways to connect.”
- [claimed-docs] “Runs on your machine and needs Node.js and an API key. Use it when your client cannot reach remote servers.”
ai-native userUse an official CLI
weight 2 · round to ScrapingBeeEvidence shows an official CLI repo published by ScrapingBee (scrapingbee-cli on GitHub), directly satisfying the story. However, there's no documentation excerpt describing CLI features, installation, or usage details, nor independent community corroboration of its use. Missing for 10: detailed CLI documentation/usage examples, independent/hands-on validation, and confirmation of active maintenance.
- [probe] “official CLI documented at https://github.com/ScrapingBee/scrapingbee-cli”
Riveternone0/10Evidence shows SDKs (Go), a local MCP server requiring Node.js, and REST API features, but no mention of an official CLI tool for running enrichments or managing the product. The docs and probes (llms.txt, openapi) surface no CLI reference, so this applicable axis is unmet.
- [claimed-docs] “Use Riveter from Claude, ChatGPT, Cursor, or any MCP-compatible AI assistant. Pick one of the two ways to connect.”
- [claimed-docs] “Runs on your machine and needs Node.js and an API key. Use it when your client cannot reach remote servers.”
- [claimed-docs] “the SDKs handle auth, retries (429s and transient GET failures), the wait long-poll, polling until a run finishes (wait_for_result), and pag…”
- [claimed-docs] “run, err := client.Enrich(ctx, riveter.EnrichParams{ Prompt: "Research each company", Attributes: []string{"CEO", "Employee Count"},”
ai-native userDrive the product through a documented public API
weight 3 · round to ScrapingBeeScrapingBee's entire product is a documented public REST API with extensive parameter documentation (docs-1 to docs-15) and an llms.txt index for AI discoverability (probe-1), plus official CLI and MCP server (probe-4, probe-5) enabling agentic access. Missing for 10: a machine-readable OpenAPI/Swagger spec (probe-2 and probe-3 both 404) and independent hands-on confirmation of API integration ease.
- [claimed-docs] “To scrape a web page, you only need two things: Your API key... The encoded web page URL you want to scrape”
- [claimed-docs] “extract_rules [stringified JSON] ("") Data extraction from CSS selectors”
- [claimed-docs] “ai_query [string] ("") The information you want to extract from the webpage using AI”
- [probe] “PROBE llms.txt: HTTP 200 at https://www.scrapingbee.com/llms.txt # ScrapingBee Documentation > Official documentation index for ScrapingBee…”
- [probe] “official MCP server documented at https://mcp.scrapingbee.com/”
- [probe] “official CLI documented at https://github.com/ScrapingBee/scrapingbee-cli”
- [probe] “PROBE openapi: all candidate paths 404 (https://www.scrapingbee.com/openapi.json, https://www.scrapingbee.com/swagger.json, https://www.scra…”
Docs describe concrete API mechanics (webhook_url, dry_run, max_credits, SDK auth/retry/pagination handling, Go SDK code sample) showing a real documented public API surface for driving runs programmatically, and MCP/remote-server integration is documented. However, probes for a formal machine-readable spec (openapi.json/swagger.json) and llms.txt all returned 404, so there's no discoverable canonical API reference, undermining full 'documented public API' claims. missing for 10: a public OpenAPI/swagger spec or llms.txt confirming a fully machine-readable API contract, independent third-party confirmation of API usage.
- [claimed-docs] “Pass webhook_url in the JSON body when starting a run and Riveter POSTs the full results to your URL when it finishes (events: run.completed…”
- [claimed-docs] “dry_run: true — validate the request and return a credit estimate without creating or charging anything.”
- [claimed-docs] “max_credits — a ceiling. When the estimate's maximum exceeds it, the request is refused with 422 credit_cap_exceeded and nothing is charged.”
- [claimed-docs] “the SDKs handle auth, retries (429s and transient GET failures), the wait long-poll, polling until a run finishes (wait_for_result), and pag…”
- [claimed-docs] “run, err := client.Enrich(ctx, riveter.EnrichParams{ Prompt: "Research each company", Attributes: []string{"CEO", "Employee Count"},”
- [claimed-docs] “Use Riveter from Claude, ChatGPT, Cursor, or any MCP-compatible AI assistant. Pick one of the two ways to connect.”
- [probe] “PROBE llms.txt: HTTP 404 at https://docs.riveterhq.com/llms.txt”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.riveterhq.com/openapi.json, https://docs.riveterhq.com/swagger.json, https://docs.rivet…”
ai-native userIssue scoped/least-privilege API credentials for an agent
weight 2 · round drawnScrapingBeenone0/10Evidence only shows a single API key model for authentication with no mention of scoped, restricted-permission, or per-agent credential issuance; the community note about using two API keys does not indicate least-privilege scoping. Missing for 10: any documentation of scoped/restricted API keys, role-based permissions, or credential issuance mechanisms for agents.
- [claimed-docs] “To scrape a web page, you only need two things: Your API key... The encoded web page URL you want to scrape”
- [community] “cool idea, but I don't like how I need to use two separate API keys to connect to this API (which relies on ScrapingBee for scraping) - can …”
Riveternone0/10Riveter's evidence covers a single API key model, credit caps, and dry-run cost estimation, but there is no mention of scoped or least-privilege credentials, per-agent tokens, or permission scoping for agents. missing for 10: scoped/least-privilege credential issuance, per-agent API key scoping, role/permission-based access control.
ai-native userBuild against official SDKs
weight 2 · round to RiveterScrapingBeenone0/10The evidence pack documents ScrapingBee's REST API parameters, an official CLI, and an MCP server, but contains no mention of official SDKs (e.g., Python, Node.js, PHP client libraries) that AI-native developers could build against. Absence of evidence for this applicable capability warrants a 'none' verdict.
Riveter ships an official Go SDK (riveterhq/riveter-go) with documented client code (riveter.EnrichParams), and docs describe SDK-level handling of auth, retries, long-polling, and pagination, indicating a first-party SDK layer built for AI-native workflows. Missing for 10: confirmation of additional language SDKs (e.g., Python/JS) beyond Go, and independent/hands-on corroboration of SDK reliability.
- [claimed-docs] “the SDKs handle auth, retries (429s and transient GET failures), the wait long-poll, polling until a run finishes (wait_for_result), and pag…”
- [claimed-docs] “run, err := client.Enrich(ctx, riveter.EnrichParams{ Prompt: "Research each company", Attributes: []string{"CEO", "Employee Count"},”
ai-native userSubscribe to events via webhooks
weight 2 · round to RiveterScrapingBeenone0/10No evidence of any webhook subscription or event notification system in ScrapingBee's documentation; all evidence covers synchronous scraping API parameters, proxies, and rendering options with no mention of webhooks or event-driven callbacks.
Riveter supports webhooks by passing a webhook_url when starting a run, with Riveter POSTing results back on run.completed, run.stopped, and run.finished events — a real event-notification mechanism for agentic workflows. However this is scoped to a single run's lifecycle rather than a general subscription model (no persistent webhook registration/management endpoint, no broader event catalog, no signature/security details). Missing for 10: a dedicated webhook subscription/management API, documentation of additional event types beyond run lifecycle, and payload signing/verification details.
- [claimed-docs] “Pass webhook_url in the JSON body when starting a run and Riveter POSTs the full results to your URL when it finishes (events: run.completed…”
Agentic features
ai-native userGet AI-generated insights and suggestions from my data inside the product
weight 2 · round to RiveterScrapingBee's `ai_query` parameter lets users ask AI to extract specific information from scraped pages, which is a limited AI capability applied to data the product handles, but it's user-directed extraction rather than proactive AI-generated insights or suggestions surfaced inside the product. Missing for 10: evidence of automated insight generation, trend/anomaly detection, or suggestion features beyond on-demand query-based extraction.
- [claimed-docs] “ai_query [string] ("") The information you want to extract from the webpage using AI”
Riveter's core enrichment feature fills columns using AI agents, web search/scrape, and other tools to generate insights directly on user data, and search_agent provides ad hoc AI-researched answers within the product. missing for 10: independent/hands-on corroboration of insight quality, no example of proactive/unprompted suggestions (only prompt-driven enrichment), and no dashboard-level 'insights' UI evidence beyond API/SDK docs.
- [claimed-docs] “An enrichment takes rows of input data and fills in new columns using AI, web searches, web scrapes, and other tools.”
- [claimed-docs] “You can build one from a natural-language prompt or a structured spec, and Riveter will generate the rows for you.”
- [claimed-docs] “A search_agent call asks one question and gets one AI-researched answer back — the same agent loop that fills a single enrichment cell, with…”
- [claimed-docs] “Riveter uses AI agents that interpret pages the way a person would, so the same configuration keeps working after a redesign.”
- [claimed-docs] “It reads PDFs and images, calls third party APIs as part of a workflow, and combines those results with data pulled from the web in a single…”
ai-native userSet up automations that run autonomously in the background
weight 2 · round to RiveterScrapingBeenone0/10ScrapingBee is an on-demand scraping API/CLI/MCP server; evidence shows only synchronous request-response scraping calls, with no scheduling, triggers, or background job/automation orchestration features documented.
Riveter supports scheduling projects to run on a cadence ('every minute' for fast-moving data) and webhook notifications on run completion, which enables autonomous background execution without manual triggering. However, there's no evidence of broader automation orchestration (e.g., conditional triggers, chaining multiple actions, or a dedicated automation/workflow builder) beyond scheduled data refresh. Missing for 10: evidence of multi-step autonomous workflows beyond scheduled enrichment refresh, independent/hands-on confirmation that scheduling works reliably in production, and any automation trigger types beyond time-based schedules.
- [claimed-docs] “Schedule any project to monitor for changes and keep your data fresh.”
- [claimed-docs] “For fast moving data like scores or election results, you can refresh as often as every minute.”
- [claimed-docs] “Pass webhook_url in the JSON body when starting a run and Riveter POSTs the full results to your URL when it finishes (events: run.completed…”
ai-native userDelegate tasks to a built-in AI assistant inside the product
weight 3 · round to RiveterScrapingBeenone0/10The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)
Riveter ships an internal 'agent loop' (search_agent, enrichment AI) that autonomously researches, scrapes, and fills data on request, which functions as a built-in AI assistant for delegated research tasks rather than a conversational general-purpose assistant. Missing for 10: evidence of a general chat/task interface for arbitrary delegation, independent hands-on validation, and clarity on how broadly the agent can handle tasks beyond enrichment/search/scrape.
- [claimed-docs] “An enrichment takes rows of input data and fills in new columns using AI, web searches, web scrapes, and other tools.”
- [claimed-docs] “A search_agent call asks one question and gets one AI-researched answer back — the same agent loop that fills a single enrichment cell, with…”
- [claimed-docs] “Riveter uses AI agents that interpret pages the way a person would, so the same configuration keeps working after a redesign.”
- [claimed-docs] “It reads PDFs and images, calls third party APIs as part of a workflow, and combines those results with data pulled from the web in a single…”
ai-native userOperate the product with natural-language commands
weight 2 · round to RiveterScrapingBee's `ai_query` parameter lets users specify what to extract from a page using natural language, and there is a documented official MCP server (mcp.scrapingbee.com) that would let AI agents invoke ScrapingBee via natural-language tool calls. However, the core product interface remains a structured REST API with many typed parameters, not a natural-language command interface itself. Missing for 10: evidence of a chat/NL interface for configuring scrapes beyond ai_query, and independent confirmation the MCP server supports full natural-language operation.
- [claimed-docs] “ai_query [string] ("") The information you want to extract from the webpage using AI”
- [probe] “official MCP server documented at https://mcp.scrapingbee.com/”
Riveter explicitly supports building enrichments from natural-language prompts (riveter-docs-2), offers a search_agent that answers questions in natural language without setup (riveter-docs-5), and can be operated via MCP-compatible AI assistants like Claude, ChatGPT, and Cursor (riveter-docs-9), which is the core mechanism for natural-language control. Missing for 10: independent/hands-on confirmation of NL command reliability, and no evidence of a broader NL command surface beyond enrichment/search (e.g., NL-driven scheduling or config changes).
- [claimed-docs] “You can build one from a natural-language prompt or a structured spec, and Riveter will generate the rows for you.”
- [claimed-docs] “A search_agent call asks one question and gets one AI-researched answer back — the same agent loop that fills a single enrichment cell, with…”
- [claimed-docs] “Use Riveter from Claude, ChatGPT, Cursor, or any MCP-compatible AI assistant. Pick one of the two ways to connect.”
- [claimed-docs] “run, err := client.Enrich(ctx, riveter.EnrichParams{ Prompt: "Research each company", Attributes: []string{"CEO", "Employee Count"},”
ai-native userApply a preset configuration tuned for research agents that returns structured, citable output
weight 2 · round to ScrapingBeeScrapingBee offers markdown output (return_page_markdown), AI-driven extraction (ai_query), and structured extraction (extract_rules) which can produce citable, structured output usable by research agents, plus an MCP server for agentic integration. However, there is no evidence of a dedicated 'preset configuration tuned for research agents' — no named research-agent mode, no citation metadata, and no documentation bundling these features into a single agent-oriented preset. missing for 10: a documented research-agent preset/mode, citation/source-attribution output, and evidence of agent-specific tuning beyond generic AI extraction params.
- [claimed-docs] “ai_query [string] ("") The information you want to extract from the webpage using AI”
- [claimed-docs] “return_page_markdown [boolean] (false) Return the page content in markdown format”
- [claimed-docs] “extract_rules [stringified JSON] ("") Data extraction from CSS selectors”
- [probe] “official MCP server documented at https://mcp.scrapingbee.com/”
Riveternone0/10Riveter offers enrichment, search_agent, and scrape tools with structured outputs, but there is no evidence of a preset/template configuration specifically tuned for research agents or citable output formatting; missing for 10: a named preset or template targeting research-agent workflows, citation/source-attribution formatting in outputs, and any documentation referencing 'research agent' presets.
Api quality
ai-native userExplore an interactive API reference with runnable examples
weight 2 · round drawnScrapingBeenone0/10The evidence pack shows only static parameter documentation and no mention of an interactive API reference, live 'try it' console, or runnable code examples; probes even show no OpenAPI/swagger spec and a 404 on a machine-readable docs endpoint, suggesting no interactive explorer exists.
- [claimed-docs] “To scrape a web page, you only need two things: Your API key... The encoded web page URL you want to scrape”
- [probe] “PROBE docs-md: HTTP 404 at https://www.scrapingbee.com/documentation.md”
- [probe] “PROBE openapi: all candidate paths 404 (https://www.scrapingbee.com/openapi.json, https://www.scrapingbee.com/swagger.json, https://www.scra…”
Riveternone0/10No evidence of an interactive API reference or runnable examples; probes for llms.txt and OpenAPI/Swagger specs both returned 404s, and docs snippets are static text/code examples only, not interactive/runnable.
ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)
weight 2 · round drawnScrapingBeenone0/10Direct probes for an OpenAPI/Swagger spec at all standard locations returned 404, and no evidence pack item shows a downloadable machine-readable API spec being offered elsewhere; only an llms.txt index and human-readable docs exist.
Riveternone0/10Probes for llms.txt and OpenAPI/swagger spec files all returned 404s, and no documentation mentions a downloadable machine-readable API spec despite having a REST API and SDKs.
ai-native userTest against a sandbox environment without touching production data
weight 1 · round to RiveterScrapingBeenone0/10The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)
Riveter offers a dry_run mode that validates a request and returns a credit estimate without creating or charging anything, and a max_credits cap that blocks runs before they execute — both function like a lightweight 'test without side effects' capability. However, there's no explicit documentation of a separate sandbox environment or synthetic/test dataset distinct from production data sources (Riveter always operates against live web/data sources when actually run). Missing for 10: a documented sandbox/staging environment, sample or mock datasets, and explicit guidance on testing enrichments without touching real production data sources.
- [claimed-docs] “dry_run: true — validate the request and return a credit estimate without creating or charging anything.”
- [claimed-docs] “max_credits — a ceiling. When the estimate's maximum exceeds it, the request is refused with 422 credit_cap_exceeded and nothing is charged.”
ai-native userRely on versioned APIs with a documented deprecation policy
weight 2 · round drawnScrapingBeenone0/10No evidence of API versioning scheme or a documented deprecation policy; OpenAPI spec probes 404 and docs don't mention versioning/deprecation terms at all.
Riveternone0/10No evidence of API versioning scheme or a documented deprecation policy; probes for OpenAPI/spec discovery returned 404s, and docs mention SDKs/features but nothing about version numbers or deprecation guarantees. Missing for 10: versioned endpoint scheme (e.g., /v1/), a published deprecation/sunset policy, changelog or migration guides.
data-engineerThe documented rate limit (requests per second or minute) enforced on my API key before throttling kicks in
weight 3 · round drawnScrapingBeenone0/10No evidence pack item mentions a documented rate limit (requests per second/minute) or throttling behavior for API keys; documentation excerpts cover scraping parameters and features but not concurrency/rate-limit thresholds.
Riveternone0/10There is a mention of SDKs handling retries on 429s, implying rate limiting exists, but no documented numeric rate limit (requests per second/minute) is given anywhere in the evidence pack, and probes for API spec/docs return 404s.
- [claimed-docs] “the SDKs handle auth, retries (429s and transient GET failures), the wait long-poll, polling until a run finishes (wait_for_result), and pag…”
- [probe] “PROBE llms.txt: HTTP 404 at https://docs.riveterhq.com/llms.txt”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.riveterhq.com/openapi.json, https://docs.riveterhq.com/swagger.json, https://docs.rivet…”
Anti bot — getting past bot defenses — CAPTCHAs, fingerprinting, blocksAnti bot
Getting past bot defenses — CAPTCHAs, fingerprinting, blocks
Block evasion
ai-native userHave an agent automatically get past a CAPTCHA, login, or form wall without my manual intervention
weight 2 · round to ScrapingBeeScrapingBee provides premium proxies to bypass hard-to-scrape sites and JS 'scenario' scripting to interact with pages (e.g., click/fill forms), which could support login flows, but there is no explicit claim or evidence of automatic CAPTCHA solving or a documented login-automation workflow that removes manual intervention entirely. missing for 10: explicit CAPTCHA-solving mechanism, documented login/form-wall bypass workflow, and independent evidence of successful autonomous bypass.
- [claimed-docs] “If you want to interact with pages you want to scrape before we return your the HTML you can add JavaScript scenario to your API call.”
- [claimed-docs] “premium_proxy [boolean] (false) Use premium proxies to bypass difficult to scrape websites”
- [claimed-docs] “session_id [integer] ("") Route multiple API requests through the same IP address”
data-engineerAutomatically retry through a chain of different proxies when anti-bot detection blocks a request
weight 2 · round to ScrapingBeeScrapingBee offers premium_proxy and country_code parameters and an 'auto' mode that picks the cheapest configuration that succeeds, implying some automatic fallback/retry logic, but there's no explicit documentation of a chained multi-proxy retry mechanism specifically triggered by anti-bot detection. missing for 10: explicit documentation of automatic retry chains across multiple proxies upon anti-bot block detection, and independent verification of this retry behavior.
- [claimed-docs] “premium_proxy [boolean] (false) Use premium proxies to bypass difficult to scrape websites”
- [claimed-docs] “country_code [string] ("") Premium proxy geolocation”
- [claimed-docs] “mode [string] ("") Let ScrapingBee pick the cheapest configuration that succeeds. Only value is auto”
developerUse an undetected browser mode to bypass sophisticated bot detection systems
weight 3 · round to ScrapingBeeScrapingBee's docs mention `premium_proxy` explicitly for bypassing 'difficult to scrape websites' and headless browser rendering with JS scenarios, which implies anti-bot capability, but the evidence never uses 'undetected browser' or 'stealth mode' terminology or details specific bot-detection bypass techniques (fingerprint spoofing, TLS/JA3 evasion, etc.). Missing for 10: explicit stealth/undetected-mode documentation, technical detail on fingerprint evasion, and independent verification that it defeats sophisticated bot detection.
- [claimed-docs] “premium_proxy [boolean] (false) Use premium proxies to bypass difficult to scrape websites”
- [claimed-docs] “If you want to interact with pages you want to scrape before we return your the HTML you can add JavaScript scenario to your API call.”
- [claimed-docs] “Our headless browsers will wait for the CSS / Xpath selector passed in the parameter before returning the HTML.”
Riveternone0/10No evidence mentions undetected browser mode, bot-detection bypass, proxies, or stealth automation features; Riveter's evidence only covers enrichment, scraping, and search tooling. Missing for 10: any mention of anti-bot/stealth browser capabilities, CAPTCHA handling, or evasion of bot detection.
Proxy rotation
developerRequest a proxy from a specific country to get geolocation-appropriate content
weight 2 · round to ScrapingBeeScrapingBee's docs explicitly document a `country_code` parameter for premium proxy geolocation, directly enabling country-specific proxy requests, alongside `premium_proxy` to enable this feature. missing for 10: independent/hands-on confirmation of geolocation accuracy and no list of supported countries in the evidence.
- [claimed-docs] “premium_proxy [boolean] (false) Use premium proxies to bypass difficult to scrape websites”
- [claimed-docs] “country_code [string] ("") Premium proxy geolocation”
developerUse premium residential or datacenter proxies to bypass sites that are hard to scrape
weight 3 · round to ScrapingBeeDocs explicitly document `premium_proxy` for bypassing hard-to-scrape sites, plus `country_code` for geolocation and `session_id` for sticky IP sessions, directly matching the story. Missing for 10: explicit distinction/documentation of residential vs datacenter proxy types and independent third-party validation of bypass success rates.
- [claimed-docs] “premium_proxy [boolean] (false) Use premium proxies to bypass difficult to scrape websites”
- [claimed-docs] “country_code [string] ("") Premium proxy geolocation”
- [claimed-docs] “session_id [integer] ("") Route multiple API requests through the same IP address”
developerRoute requests through a rotating pool of proxy IPs to avoid blocks
weight 3 · round to ScrapingBeeDocs confirm premium/rotating proxy usage (premium_proxy, country_code) to bypass blocks, plus session_id to pin a single IP when needed, indicating an underlying rotating proxy pool by default with control options. Missing for 10: no independent/hands-on evidence confirming rotation effectiveness against real anti-bot defenses, and no explicit documentation describing pool size or rotation algorithm.
- [claimed-docs] “premium_proxy [boolean] (false) Use premium proxies to bypass difficult to scrape websites”
- [claimed-docs] “country_code [string] ("") Premium proxy geolocation”
- [claimed-docs] “session_id [integer] ("") Route multiple API requests through the same IP address”
developerRoute multiple requests through the same proxy IP using a session identifier to maintain a consistent identity
weight 2 · round to ScrapingBeeOfficial docs explicitly document a `session_id` parameter to route multiple API requests through the same proxy IP, directly matching the story. Missing for 10: independent/hands-on corroboration of session persistence behavior beyond first-party docs.
- [claimed-docs] “session_id [integer] ("") Route multiple API requests through the same IP address”
Automation depth — how much of the product can run unattendedAutomation depth
How much of the product can run unattended
ai-native userPerform bulk operations across many items at once
weight 2 · round to RiveterScrapingBeenone0/10The evidence pack documents single-page scraping parameters (JS scenarios, extraction rules, proxies, screenshots) but never mentions a batch/bulk API endpoint, concurrent job submission, or a mechanism to process many URLs/items in one call.
Riveter's core enrichment model operates on many rows at once (bulk input data with AI-filled columns), supports batch generation from a prompt/spec, scheduling for ongoing refresh, and examples like pulling every dentist from every practice in a city in one request. Missing for 10: independent/hands-on verification of large-scale bulk runs and no explicit documentation of per-run item limits or throughput benchmarks.
- [claimed-docs] “An enrichment takes rows of input data and fills in new columns using AI, web searches, web scrapes, and other tools.”
- [claimed-docs] “You can build one from a natural-language prompt or a structured spec, and Riveter will generate the rows for you.”
- [claimed-docs] “Schedule any project to monitor for changes and keep your data fresh.”
- [claimed-docs] “It can find every dental practice in a city, then pull every dentist from each one, in a single request.”
- [claimed-docs] “For fast moving data like scores or election results, you can refresh as often as every minute.”
- [claimed-docs] “It reads PDFs and images, calls third party APIs as part of a workflow, and combines those results with data pulled from the web in a single…”
ai-native userDefine rules that trigger actions automatically on events
weight 3 · round to RiveterScrapingBeenone0/10The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)
Riveter supports scheduled refresh of projects (time-based automation) and webhook events (run.completed/stopped/finished) that can notify external systems, giving some automation-on-events capability, but there is no evidence of a rules/condition engine that lets users define arbitrary triggers (e.g., 'if data matches X, then do Y') beyond scheduling and run-completion notifications. missing for 10: conditional rule definitions, event-driven branching logic, multi-condition triggers, and any UI/API for building custom automations beyond schedule+webhook.
- [claimed-docs] “Pass webhook_url in the JSON body when starting a run and Riveter POSTs the full results to your URL when it finishes (events: run.completed…”
- [claimed-docs] “Schedule any project to monitor for changes and keep your data fresh.”
- [claimed-docs] “For fast moving data like scores or election results, you can refresh as often as every minute.”
ai-native userSchedule recurring jobs or workflows
weight 2 · round to RiveterScrapingBeenone0/10ScrapingBee's evidence pack covers API scraping parameters, JS rendering, proxies, and extraction, but contains no mention of scheduling, recurring jobs, cron-like triggers, or workflow orchestration features.
Docs state you can 'schedule any project to monitor for changes and keep your data fresh' and refresh as often as every minute, indicating recurring job/workflow scheduling support. However, details are thin — no documentation on schedule configuration (cron-like syntax, timezone, pause/resume), no UI/API endpoint specifics for managing schedules, and no independent or hands-on corroboration. Missing for 10: scheduling API/UI details, configuration options, independent verification of reliability at scale.
- [claimed-docs] “Schedule any project to monitor for changes and keep your data fresh.”
- [claimed-docs] “For fast moving data like scores or election results, you can refresh as often as every minute.”
Dev experience — day-to-day developer experience — setup friction, docs, debugging, iteration speedDev experience
Day-to-day developer experience — setup friction, docs, debugging, iteration speed
Collaboration
developerShare scrapers with teammates and manage organizations and role-based permissions
weight 2 · round drawnScrapingBeenone0/10No evidence in the pack addresses team/organization management, sharing scrapers, or role-based permissions; the documentation excerpts focus entirely on API scraping parameters (JS rendering, proxies, extraction rules), not collaboration or account administration features.
Deployment flexibility
developerBuild and deploy custom serverless scraping scripts on the platform without managing my own infrastructure
weight 2 · round to RiveterScrapingBee is a managed scraping API (no infrastructure to manage) and supports JS 'scenarios' for custom page interaction plus extraction rules, which is a lightweight form of custom scraping logic. However, there is no evidence of a true serverless scripting/deployment platform (e.g., custom code upload, scheduled jobs, or a scripting runtime) — the story's 'build and deploy custom scripts' aspect is only partially matched by parameterized API calls. Missing for 10: evidence of a script/job deployment mechanism, scheduling, or custom code execution beyond JS scenario snippets, and independent developer confirmation of this workflow.
- [claimed-docs] “To scrape a web page, you only need two things: Your API key... The encoded web page URL you want to scrape”
- [claimed-docs] “If you want to interact with pages you want to scrape before we return your the HTML you can add JavaScript scenario to your API call.”
- [claimed-docs] “extract_rules [stringified JSON] ("") Data extraction from CSS selectors”
Riveter's docs show fully managed, serverless-style capabilities (enrichments, scrapes, quick_search, search_agent) that developers configure via natural-language prompts or structured specs and trigger via API/SDK/webhooks with no server management (riveter-docs-1,2,3,4,5,6,11,12). However, this is closer to configuring built-in AI-driven tools than deploying arbitrary custom scraping code/scripts — there's no evidence of a code-upload or custom-script execution environment. Missing for 10: evidence of arbitrary custom code/script deployment (vs. prompt/spec-based enrichment configuration), and independent confirmation of the serverless execution model.
- [claimed-docs] “An enrichment takes rows of input data and fills in new columns using AI, web searches, web scrapes, and other tools.”
- [claimed-docs] “You can build one from a natural-language prompt or a structured spec, and Riveter will generate the rows for you.”
- [claimed-docs] “A scrape lets you turn a URL into easily parseable text.”
- [claimed-docs] “A quick_search lets you quickly web search a query, and pull structured results with urls, titles, and snippets — synchronously, in one requ…”
- [claimed-docs] “A search_agent call asks one question and gets one AI-researched answer back — the same agent loop that fills a single enrichment cell, with…”
- [claimed-docs] “Pass webhook_url in the JSON body when starting a run and Riveter POSTs the full results to your URL when it finishes (events: run.completed…”
- [claimed-docs] “the SDKs handle auth, retries (429s and transient GET failures), the wait long-poll, polling until a run finishes (wait_for_result), and pag…”
- [claimed-docs] “run, err := client.Enrich(ctx, riveter.EnrichParams{ Prompt: "Research each company", Attributes: []string{"CEO", "Employee Count"},”
developerDeploy the scraping service via a Docker container for production use
weight 2 · round drawnScrapingBeenone0/10The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)
developerSelf-host an open-source version of the scraper instead of relying on a hosted cloud service
weight 2 · round drawnScrapingBeenone0/10ScrapingBee is a hosted cloud scraping API with no evidence of an open-source, self-hostable version; community comments explicitly ask about open-sourcing the stack, confirming none exists.
- [community] “Any plans on open sourcing any part of your stack instead of relying on paid services like ScrapingBee? What does your SaaS setup look like?”
- [community] “Have you looked at running something locally instead of paying for ScrapingBee? I'm using Laravel and considering Dusk to retrieve page cont…”
Riveternone0/10Riveter is presented as a hosted API/service (with a local MCP connector for client access to the remote service), but there is no evidence of an open-source, self-hostable version of the scraper itself; docs only describe running a local MCP bridge that still relies on the remote API key.
- [claimed-docs] “Use Riveter from Claude, ChatGPT, Cursor, or any MCP-compatible AI assistant. Pick one of the two ways to connect.”
- [claimed-docs] “Runs on your machine and needs Node.js and an API key. Use it when your client cannot reach remote servers.”
Integrations
developerConnect the scraping API to no-code automation platforms like n8n or Zapier through a prebuilt connector
weight 2 · round drawnScrapingBeenone0/10No evidence of a prebuilt n8n or Zapier connector; the docs cover API parameters, an MCP server, and a CLI, but nothing about no-code automation platform integrations.
Library compatibility
developerBuild scrapers using popular open-source automation libraries like Playwright, Puppeteer, Selenium, or Scrapy
weight 2 · round drawnScrapingBeenone0/10The evidence pack describes ScrapingBee's own API parameters (JS scenario, screenshots, extraction rules, proxies) but contains no mention of official integrations, SDKs, or middleware for Playwright, Puppeteer, Selenium, or Scrapy. No documentation, probe, or community evidence shows developers can plug ScrapingBee into these specific open-source automation libraries.
Migration lock in
developerExport my scraped data and job configurations in a portable format to migrate to another provider without lock-in
weight 3 · round drawnScrapingBeenone0/10No evidence of any export/migration tooling for scraped data or job configs in a portable format; the docs cover API parameters and scraping features but nothing about data portability or provider migration. missing for 10: export format documentation, job/config export mechanism, migration guides or tooling, any mention of avoiding vendor lock-in.
Quickstart
developerPublish my custom scraper to a public marketplace and earn revenue when others use it
weight 1 · round drawnScrapingBeenone0/10The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)
developerRun a ready-made scraper from a marketplace instead of building one from scratch
weight 2 · round drawnScrapingBeenone0/10Evidence shows only API parameters/docs for building custom scraping requests; there is no marketplace of pre-built, ready-made scrapers a developer could pick and run instead of building their own.
developerStart building immediately using a library of ready-made project templates
weight 1 · round drawnScrapingBeenone0/10No evidence of ready-made project templates or scaffolding to jumpstart development; documentation only covers API parameters and usage, not starter templates or boilerplate projects.
Extraction quality — how faithfully content is extracted — structure, fidelity, edge casesExtraction quality
How faithfully content is extracted — structure, fidelity, edge cases
Ai extraction
developerExtract structured data from a page using natural language instructions instead of writing selectors
weight 3 · round drawnScrapingBee's ai_query parameter lets developers specify in natural language the information they want extracted from a webpage, avoiding manual CSS/XPath selectors, as an alternative to the selector-based extract_rules feature. missing for 10: independent/hands-on validation of AI extraction accuracy, and details on structured output schema/reliability beyond the docs blurb.
- [claimed-docs] “ai_query [string] ("") The information you want to extract from the webpage using AI”
- [claimed-docs] “extract_rules [stringified JSON] ("") Data extraction from CSS selectors”
Riveter's core enrichment feature lets developers build extraction jobs from a natural-language prompt with target attributes instead of writing selectors, and AI agents interpret pages semantically so configs survive redesigns, directly matching the story. missing for 10: independent/hands-on verification of extraction accuracy and no live API schema (openapi/llms.txt probes 404) to confirm behavior beyond vendor docs.
- [claimed-docs] “You can build one from a natural-language prompt or a structured spec, and Riveter will generate the rows for you.”
- [claimed-docs] “run, err := client.Enrich(ctx, riveter.EnrichParams{ Prompt: "Research each company", Attributes: []string{"CEO", "Employee Count"},”
- [claimed-docs] “Riveter uses AI agents that interpret pages the way a person would, so the same configuration keeps working after a redesign.”
- [claimed-docs] “An enrichment takes rows of input data and fills in new columns using AI, web searches, web scrapes, and other tools.”
developerPass a JSON schema so the API returns structured data matching that schema
weight 2 · round to ScrapingBeeScrapingBee offers extract_rules (CSS-selector based structured extraction) and ai_query (AI-driven extraction), which let developers get structured data, but there is no evidence of accepting a formal JSON Schema definition that the API validates/conforms output to — extract_rules is a custom stringified JSON of selectors, not a schema spec. missing for 10: explicit JSON Schema input support, schema validation/conformance guarantee, examples of schema-driven structured output.
- [claimed-docs] “extract_rules [stringified JSON] ("") Data extraction from CSS selectors”
- [claimed-docs] “ai_query [string] ("") The information you want to extract from the webpage using AI”
Riveter lets you define enrichments via a natural-language prompt or a 'structured spec' with named attributes/columns (riveter-docs-2, riveter-docs-12), which produces structured output, but there is no documented mechanism for passing an arbitrary JSON Schema that the API validates/returns against. missing for 10: explicit JSON Schema input parameter, schema validation of output, and any example showing schema-conformant responses.
- [claimed-docs] “You can build one from a natural-language prompt or a structured spec, and Riveter will generate the rows for you.”
- [claimed-docs] “run, err := client.Enrich(ctx, riveter.EnrichParams{ Prompt: "Research each company", Attributes: []string{"CEO", "Employee Count"},”
- [claimed-docs] “An enrichment takes rows of input data and fills in new columns using AI, web searches, web scrapes, and other tools.”
ai-native userHave an LLM read a page and decide what structured fields to pull out without pre-written selectors
weight 2 · round to RiveterScrapingBee has an ai_query parameter that lets an LLM extract requested information from a page without pre-written CSS/XPath selectors, directly matching the story's intent, but this is described only in a single doc line rather than deeply documented with examples of dynamic field discovery. missing for 10: no documentation showing the AI deciding on its own what structured fields/schema to output (vs. a user-specified query), no independent/hands-on evidence of extraction quality or reliability, and no example of full structured JSON field inference without any query guidance.
- [claimed-docs] “ai_query [string] ("") The information you want to extract from the webpage using AI”
- [claimed-docs] “extract_rules [stringified JSON] ("") Data extraction from CSS selectors”
Docs describe enrichments where AI agents interpret pages and fill arbitrary attribute columns from a natural-language prompt or structured spec (no selectors), with scraping/search tools feeding an AI agent loop that adapts to page structure and redesigns. This directly matches the story of an LLM reading a page and deciding what fields to extract without pre-written selectors. Missing for 10: independent hands-on verification of extraction accuracy and no example showing the LLM's field-selection reasoning in practice.
- [claimed-docs] “An enrichment takes rows of input data and fills in new columns using AI, web searches, web scrapes, and other tools.”
- [claimed-docs] “You can build one from a natural-language prompt or a structured spec, and Riveter will generate the rows for you.”
- [claimed-docs] “A scrape lets you turn a URL into easily parseable text.”
- [claimed-docs] “Riveter uses AI agents that interpret pages the way a person would, so the same configuration keeps working after a redesign.”
- [claimed-docs] “It can find every dental practice in a city, then pull every dentist from each one, in a single request.”
- [claimed-docs] “run, err := client.Enrich(ctx, riveter.EnrichParams{ Prompt: "Research each company", Attributes: []string{"CEO", "Employee Count"},”
developerPlug in a local or self-hosted LLM as the extraction backend instead of a cloud-only model
weight 2 · round drawnScrapingBeenone0/10ScrapingBee's AI extraction (ai_query) uses its own cloud-based AI backend with no documented option to plug in a local or self-hosted LLM; evidence shows only a fixed AI extraction parameter, not a configurable backend.
- [claimed-docs] “ai_query [string] ("") The information you want to extract from the webpage using AI”
Riveternone0/10No evidence anywhere in the docs suggests Riveter allows swapping in a local or self-hosted LLM as the extraction engine; the product is presented as a cloud-only enrichment/extraction service with API keys, credits, and hosted agents. Missing for 10: any mention of local model support, self-hosted backend configuration, or BYO-model options.
- [claimed-docs] “An enrichment takes rows of input data and fills in new columns using AI, web searches, web scrapes, and other tools.”
- [claimed-docs] “Use Riveter from Claude, ChatGPT, Cursor, or any MCP-compatible AI assistant. Pick one of the two ways to connect.”
- [claimed-docs] “Runs on your machine and needs Node.js and an API key. Use it when your client cannot reach remote servers.”
Basic scraping
developerScrape a web page with a single API call and get its raw HTML back
weight 3 · round to ScrapingBeeDocs confirm a single API call with just an API key and target URL returns the page's HTML, with straightforward defaults (docs-1) and no complex setup required. Additional options (JS rendering, wait selectors, markdown/extract_rules) show this basic case is well-supported and flexible, though there's no independent hands-on confirmation of raw HTML fidelity. Missing for 10: independent/community verification of raw HTML output quality.
- [claimed-docs] “To scrape a web page, you only need two things: Your API key... The encoded web page URL you want to scrape”
- [claimed-docs] “If you want to interact with pages you want to scrape before we return your the HTML you can add JavaScript scenario to your API call.”
- [claimed-docs] “Our headless browsers will wait for the CSS / Xpath selector passed in the parameter before returning the HTML.”
Riveternone0/10Riveter's scrape endpoint explicitly returns 'easily parseable text' from a URL, not raw HTML — the opposite of what this story asks for, and no evidence shows an option to retrieve unprocessed HTML.
- [claimed-docs] “A scrape lets you turn a URL into easily parseable text.”
Data safety
data-engineerAutomatically detect and filter personally identifiable information out of scraped content before it reaches storage
weight 2 · round drawnScrapingBeenone0/10No evidence of any PII detection, redaction, or filtering feature in ScrapingBee's documentation or capabilities; the product offers extraction rules and AI query tools but nothing about identifying or stripping personal data before storage.
Riveternone0/10No evidence anywhere in the pack mentions PII detection, filtering, redaction, or compliance controls for scraped/enriched data; Riveter's documented features cover scraping, enrichment, search, and workflow orchestration but nothing about identifying or removing personal data before storage.
Document extraction
data-engineerExtract text content from PDFs, Word, Excel, and PowerPoint files without hosting them myself
weight 2 · round to RiveterScrapingBeenone0/10The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)
Riveter is delivered as a hosted API/SaaS (no self-hosting required) and docs state it 'reads PDFs and images' as part of enrichment workflows, but there is no evidence it extracts text from Word, Excel, or PowerPoint files specifically. missing for 10: explicit support for .docx/.xlsx/.pptx extraction, any extraction-quality benchmarks or examples for Office file formats.
- [claimed-docs] “It reads PDFs and images, calls third party APIs as part of a workflow, and combines those results with data pulled from the web in a single…”
Multimodal extraction
ai-native userGet automatic captions for images on a page so a text-only model can reason about visual content
weight 2 · round to RiveterScrapingBeenone0/10No evidence of an image-captioning or alt-text generation feature; ScrapingBee's AI features (ai_query) extract structured data from page text/HTML, not image captions for visual content, and by default it blocks images entirely. Missing for 10: any documented image captioning/vision-to-text capability, alt-text generation, or multimodal image description output.
- [claimed-docs] “By default, and to speed up requests, ScrapingBee blocks all images and CSS in the scraped page, but to scrape them, use `block_resources=fa…”
- [claimed-docs] “ai_query [string] ("") The information you want to extract from the webpage using AI”
Riveter's docs mention it 'reads PDFs and images' and combines results with web data (riveter-docs-18), implying some visual-content ingestion, but there is no explicit description of generating captions or text descriptions of images for downstream reasoning by a text-only model. Missing for 10: explicit captioning/description output format, example enrichment showing image-to-text extraction, and any confirmation this text is usable standalone by a text-only model.
- [claimed-docs] “It reads PDFs and images, calls third party APIs as part of a workflow, and combines those results with data pulled from the web in a single…”
Search integration
developerSearch the web and get full page content from results in a single call instead of just links and snippets
weight 3 · round drawnScrapingBeenone0/10Evidence shows ScrapingBee scrapes a given URL (with JS rendering, markdown output, extract_rules, ai_query) but nothing indicates a single API call that performs a web search and returns full page content for each result — the llms.txt probe mentions 'search' only in passing with no supporting detail. Missing for 10: any documented search endpoint, example combining query+results with full page bodies, or independent confirmation of this workflow.
- [claimed-docs] “To scrape a web page, you only need two things: Your API key... The encoded web page URL you want to scrape”
- [probe] “PROBE llms.txt: HTTP 200 at https://www.scrapingbee.com/llms.txt # ScrapingBee Documentation > Official documentation index for ScrapingBee…”
Riveternone0/10Riveter's quick_search explicitly returns only urls, titles, and snippets (not full page content), and its scrape tool requires a specific URL rather than combining search+content in one call. search_agent returns a single synthesized answer, not full page content per search result, so no evidenced single-call capability matches the story's exact requirement.
- [claimed-docs] “A scrape lets you turn a URL into easily parseable text.”
- [claimed-docs] “A quick_search lets you quickly web search a query, and pull structured results with urls, titles, and snippets — synchronously, in one requ…”
- [claimed-docs] “A search_agent call asks one question and gets one AI-researched answer back — the same agent loop that fills a single enrichment cell, with…”
Selector extraction
developerExtract specific fields from a page using CSS or XPath selector rules
weight 3 · round to ScrapingBeeDocs explicitly document extract_rules for CSS-based field extraction and confirm the headless browser waits on CSS/XPath selectors, directly supporting structured field extraction. missing for 10: no independent/hands-on corroboration of extraction accuracy or XPath-specific examples.
- [claimed-docs] “extract_rules [stringified JSON] ("") Data extraction from CSS selectors”
- [claimed-docs] “Our headless browsers will wait for the CSS / Xpath selector passed in the parameter before returning the HTML.”
Riveternone0/10Riveter's docs describe AI-driven page interpretation and scraping (turning URLs into parseable text, agents reading pages 'the way a person would') rather than CSS/XPath selector rules; no evidence pack item mentions selector-based extraction at all, and one item explicitly frames the AI approach as an alternative to fragile configuration that would break on redesign, which is the kind of setup selectors typically require.
- [claimed-docs] “A scrape lets you turn a URL into easily parseable text.”
- [claimed-docs] “Riveter uses AI agents that interpret pages the way a person would, so the same configuration keeps working after a redesign.”
Structured data handling
data-engineerExtract data from very large tables using intelligent chunking so it fits within processing limits
weight 1 · round drawnScrapingBeenone0/10ScrapingBee's evidence covers web scraping features (JS rendering, proxies, extraction rules, AI queries) but nothing addresses handling very large tables, chunking data to fit processing/token limits, or pagination strategies for oversized datasets. Missing for 10: any mention of table extraction, chunking mechanism, size-limit handling, or pagination/splitting of large data outputs.
Riveternone0/10Riveter's evidence covers enrichment, scraping, search, and workflow automation, but there is no mention of chunking large tables, row batching, pagination for extraction limits, or handling of very large datasets to fit processing constraints. missing for 10: any mention of chunking strategy, table size limits, batching large extractions, or row-splitting logic.
Js rendering — handling JavaScript-heavy pages — rendering, waiting, dynamic contentJs rendering
Handling JavaScript-heavy pages — rendering, waiting, dynamic content
Headless rendering
developerRender JavaScript-heavy single-page applications and get the fully rendered HTML
weight 3 · round to ScrapingBeeScrapingBee's docs explicitly describe headless-browser rendering of JS-heavy SPAs built with React/Angular/Vue/JQuery, with support for waiting on selectors and running JS scenarios before returning fully rendered HTML. This directly matches the story's core capability, though evidence lacks independent hands-on corroboration of rendering fidelity. Missing for 10: independent/hands-on verification of rendered output quality, benchmarks against specific SPA frameworks.
- [claimed-docs] “This can be useful for scraping a Single Page Application built with frameworks such as React.js, Angular.js, JQuery or Vue.”
- [claimed-docs] “If you want to interact with pages you want to scrape before we return your the HTML you can add JavaScript scenario to your API call.”
- [claimed-docs] “Our headless browsers will wait for the CSS / Xpath selector passed in the parameter before returning the HTML.”
- [claimed-docs] “To scrape a web page, you only need two things: Your API key... The encoded web page URL you want to scrape”
Riveternone0/10Riveter is a data-enrichment/scraping/AI-agent tool focused on turning URLs into text and filling data columns; there is no evidence it renders JS-heavy SPAs into fully rendered HTML (e.g., headless browser rendering, DOM snapshot output). The 'scrape' feature converts URLs to 'easily parseable text', not full rendered HTML, so this capability is unevidenced.
- [claimed-docs] “A scrape lets you turn a URL into easily parseable text.”
developerHave the API wait for a specific selector to appear before returning the rendered page
weight 2 · round to ScrapingBeeScrapingBee's docs explicitly state headless browsers wait for a CSS/XPath selector before returning HTML, directly matching the story. Missing for 10: independent/hands-on confirmation of this specific wait-for-selector behavior beyond vendor docs, and example code showing the parameter in use.
- [claimed-docs] “Our headless browsers will wait for the CSS / Xpath selector passed in the parameter before returning the HTML.”
- [claimed-docs] “If you want to interact with pages you want to scrape before we return your the HTML you can add JavaScript scenario to your API call.”
Interactive automation
developerAccess a managed remote browser sandbox for interactive, manual browsing workflows
weight 2 · round drawnScrapingBeenone0/10ScrapingBee's documentation describes a headless browser API for automated scraping (JS scenarios, screenshots, extraction rules) but no evidence of an interactive, manual remote-browser sandbox session a developer could drive by hand.
- [claimed-docs] “If you want to interact with pages you want to scrape before we return your the HTML you can add JavaScript scenario to your API call.”
- [claimed-docs] “Our headless browsers will wait for the CSS / Xpath selector passed in the parameter before returning the HTML.”
- [claimed-docs] “If you need to change the dimension of the browser's viewport (window) when scraping the target page you can use the `window_width` and `win…”
developerKeep interacting with an already-scraped page, clicking and filling forms to reach content behind a login wall
weight 2 · round to ScrapingBeeScrapingBee's JS 'scenario' feature lets you script click/fill actions before the page HTML is returned, and session_id lets you reuse the same IP across multiple API calls to preserve login state — enabling a login-wall workflow. However, evidence shows only a stateless-per-request model (scenario executed once, then HTML returned) rather than a persistent, continuously interactive browser session across multiple later calls. Missing for 10: documentation of a true persistent/interactive session object you can repeatedly command, and any hands-on confirmation this pattern reliably defeats login walls.
- [claimed-docs] “If you want to interact with pages you want to scrape before we return your the HTML you can add JavaScript scenario to your API call.”
- [claimed-docs] “Our headless browsers will wait for the CSS / Xpath selector passed in the parameter before returning the HTML.”
- [claimed-docs] “session_id [integer] ("") Route multiple API requests through the same IP address”
developerScript page interactions like clicking, filling inputs, and scrolling before content is returned
weight 3 · round to ScrapingBeeScrapingBee's docs explicitly describe a 'JavaScript scenario' feature to interact with pages (click, fill, scroll, etc.) before HTML is returned, plus wait-for-selector support to ensure content loads after interactions. This directly matches the story of scripting interactions before content is returned, though evidence lacks a full list of supported actions or independent hands-on confirmation. Missing for 10: detailed enumeration of supported interaction commands (click/fill/scroll) beyond generic 'JavaScript scenario' mention, and independent/community validation of this specific feature.
- [claimed-docs] “If you want to interact with pages you want to scrape before we return your the HTML you can add JavaScript scenario to your API call.”
- [claimed-docs] “Our headless browsers will wait for the CSS / Xpath selector passed in the parameter before returning the HTML.”
- [claimed-docs] “This can be useful for scraping a Single Page Application built with frameworks such as React.js, Angular.js, JQuery or Vue.”
Render configuration
developerControl the browser viewport width and height when rendering a page
weight 1 · round to ScrapingBeeOfficial docs explicitly state window_width and window_height parameters let developers change the browser viewport dimensions when rendering the target page. missing for 10: no independent/hands-on corroboration beyond first-party docs.
- [claimed-docs] “If you need to change the dimension of the browser's viewport (window) when scraping the target page you can use the `window_width` and `win…”
Session persistence
developerPass my own session cookies so the API fetches pages requiring authentication
weight 2 · round drawnScrapingBeenone0/10No evidence pack item mentions passing custom cookies or headers for authenticated sessions; docs cover JS rendering, proxies, extraction, screenshots, but nothing about supplying session cookies for authenticated page fetches.
developerReuse a persistent browser profile with saved cookies and login state across multiple requests
weight 2 · round drawnScrapingBeenone0/10ScrapingBee's docs mention session_id only for routing requests through the same IP address, not for persisting cookies or login state across requests; no evidence of a saved browser profile or session state reuse mechanism.
- [claimed-docs] “session_id [integer] ("") Route multiple API requests through the same IP address”
Openness — open source, data portability, and self-hosting storiesOpenness
Open source, data portability, and self-hosting stories
ai-native userDo everything through the API that I can do in the UI
weight 2 · round drawnScrapingBee is API-first, and the docs show an extensive, feature-rich API surface (JS rendering, screenshots, extraction rules, AI query, proxies, session control) covering essentially all scraping functionality (scrapingbee-docs-1 through 15). However, there is no explicit statement comparing the API's capabilities to what's available in ScrapingBee's dashboard/UI, so full parity can't be confirmed from evidence. Missing for 10: explicit UI-vs-API feature parity documentation, confirmation that dashboard-only tools (e.g. request builder, account settings) have no capabilities absent from the API.
- [claimed-docs] “To scrape a web page, you only need two things: Your API key... The encoded web page URL you want to scrape”
- [claimed-docs] “If you want to interact with pages you want to scrape before we return your the HTML you can add JavaScript scenario to your API call.”
- [claimed-docs] “extract_rules [stringified JSON] ("") Data extraction from CSS selectors”
- [claimed-docs] “ai_query [string] ("") The information you want to extract from the webpage using AI”
- [claimed-docs] “screenshot_selector [string] ("") Return a screenshot of a particular area of the page, targeted by a CSS selector”
Docs show many core capabilities (building enrichments via prompt/spec, scraping, quick_search, search_agent, webhooks, dry_run) are all API-accessible, suggesting broad parity, but there is no explicit statement of full UI/API parity and some UI-highlighted features like scheduling refresh (riveter-docs-13, riveter-docs-17) aren't confirmed as API-exposed. Additionally, probes show no discoverable OpenAPI spec (riveter-probe-2) or llms.txt (riveter-probe-1), undermining confidence that the API surface is fully documented/openly specified. missing for 10: explicit parity statement, API access to scheduling/monitoring feature, published OpenAPI spec for verification.
- [claimed-docs] “You can build one from a natural-language prompt or a structured spec, and Riveter will generate the rows for you.”
- [claimed-docs] “A scrape lets you turn a URL into easily parseable text.”
- [claimed-docs] “A quick_search lets you quickly web search a query, and pull structured results with urls, titles, and snippets — synchronously, in one requ…”
- [claimed-docs] “A search_agent call asks one question and gets one AI-researched answer back — the same agent loop that fills a single enrichment cell, with…”
- [claimed-docs] “Pass webhook_url in the JSON body when starting a run and Riveter POSTs the full results to your URL when it finishes (events: run.completed…”
- [claimed-docs] “dry_run: true — validate the request and return a credit estimate without creating or charging anything.”
- [claimed-docs] “Schedule any project to monitor for changes and keep your data fresh.”
- [claimed-docs] “For fast moving data like scores or election results, you can refresh as often as every minute.”
- [probe] “PROBE llms.txt: HTTP 404 at https://docs.riveterhq.com/llms.txt”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.riveterhq.com/openapi.json, https://docs.riveterhq.com/swagger.json, https://docs.rivet…”
ai-native userExport all of my data in open formats and leave
weight 3 · round to ScrapingBeeScrapingBee returns scraped content in open formats such as raw HTML, JSON (extract_rules) and Markdown (return_page_markdown), so output data is not locked into a proprietary format. However, there is no evidence of account-level data export, no mention of stored user data portability, and no explicit 'leave anytime with your data' commitment—since it's a stateless scraping API, the 'export and leave' framing only partially applies. Missing for 10: account/usage data export tooling, explicit data-portability statement, independent confirmation of format openness beyond docs.
- [claimed-docs] “extract_rules [stringified JSON] ("") Data extraction from CSS selectors”
- [claimed-docs] “return_page_markdown [boolean] (false) Return the page content in markdown format”
- [claimed-docs] “ai_query [string] ("") The information you want to extract from the webpage using AI”
Riveternone0/10No evidence of any data export feature or open-format export capability; the evidence only covers enrichment, scraping, search, and API integration features, with no mention of exporting data or portability guarantees. missing for 10: export functionality documentation, supported open formats (CSV/JSON/etc), any data-portability or account-closure workflow.
ai-native userRead the product's source under an open license
weight 2 · round drawnScrapingBeenone0/10No evidence ScrapingBee's core product source is available under an open license; it is a closed, paid SaaS API. A community comment even asks whether the vendor plans to open source any part of their stack, implying it currently is not.
- [community] “Any plans on open sourcing any part of your stack instead of relying on paid services like ScrapingBee? What does your SaaS setup look like?”
ai-native userSelf-host the core product
weight 3 · round drawnScrapingBeenone0/10ScrapingBee is a hosted SaaS API; no evidence of any self-hostable core product, on-premise deployment option, or open-source release. Community comment explicitly asks whether ScrapingBee plans to open-source its stack, with no vendor response indicating such an offering exists.
- [community] “Any plans on open sourcing any part of your stack instead of relying on paid services like ScrapingBee? What does your SaaS setup look like?”
- [community] “Have you looked at running something locally instead of paying for ScrapingBee? I'm using Laravel and considering Dusk to retrieve page cont…”
Riveternone0/10Riveter is presented as a hosted API/SaaS product (with local MCP server option only for connecting AI clients, not for self-hosting the core enrichment engine); no evidence of open-source code, self-hosting instructions, or a downloadable core product exists in the pack.
- [claimed-docs] “Use Riveter from Claude, ChatGPT, Cursor, or any MCP-compatible AI assistant. Pick one of the two ways to connect.”
- [claimed-docs] “Runs on your machine and needs Node.js and an API key. Use it when your client cannot reach remote servers.”
Output formats — stories about output formats in this arenaOutput formats
Stories about output formats in this arena
Content formats
developerReceive scraped content as clean markdown instead of raw HTML
weight 3 · round to ScrapingBeeScrapingBee's docs explicitly offer a `return_page_markdown` parameter to return page content as markdown instead of raw HTML, directly matching the story. Missing for 10: independent/hands-on confirmation of markdown output quality and any community corroboration of this specific feature.
- [claimed-docs] “return_page_markdown [boolean] (false) Return the page content in markdown format”
Docs state a scrape 'turns a URL into easily parseable text,' implying cleaned output rather than raw HTML, but there's no explicit mention of markdown formatting or output schema. Missing for 10: explicit confirmation that scrape output is markdown-formatted, example output showing markdown structure, independent verification of output cleanliness.
- [claimed-docs] “A scrape lets you turn a URL into easily parseable text.”
developerChoose exactly which output format is returned, such as markdown, HTML, text, or frontmatter
weight 2 · round to ScrapingBeeDocs confirm HTML is the default output and a dedicated `return_page_markdown` parameter lets developers get markdown instead, but there's no documented option for a plain-text-only extraction or a frontmatter output format, and extract_rules/ai_query only allow custom JSON-style extraction, not those specific formats. Missing for 10: explicit plain-text output mode, frontmatter output support, independent confirmation of format switching.
- [claimed-docs] “To scrape a web page, you only need two things: Your API key... The encoded web page URL you want to scrape”
- [claimed-docs] “return_page_markdown [boolean] (false) Return the page content in markdown format”
- [claimed-docs] “extract_rules [stringified JSON] ("") Data extraction from CSS selectors”
developerReceive scraped content as structured JSON
weight 3 · round to RiveterScrapingBee offers extract_rules for CSS-selector-based structured data extraction and ai_query for AI-driven extraction, plus return_page_markdown for markdown output, indicating structured output beyond raw HTML. However, there's no explicit documented 'return as JSON' toggle or example showing a full JSON schema response, and no independent/community confirmation of structured JSON output quality. Missing for 10: explicit JSON output examples/schema, independent verification of structured JSON extraction reliability.
- [claimed-docs] “extract_rules [stringified JSON] ("") Data extraction from CSS selectors”
- [claimed-docs] “ai_query [string] ("") The information you want to extract from the webpage using AI”
- [claimed-docs] “return_page_markdown [boolean] (false) Return the page content in markdown format”
Riveter's enrichments and scrapes explicitly return structured, parseable data (columns, urls/titles/snippets, webhook payloads of 'full results'), and SDK examples show structured attribute objects returned from calls, indicating outputs are consumable as structured JSON rather than raw text. missing for 10: an explicit statement of JSON schema/response format in docs, and independent/hands-on confirmation of the JSON structure (API docs endpoints 404 in probes).
- [claimed-docs] “An enrichment takes rows of input data and fills in new columns using AI, web searches, web scrapes, and other tools.”
- [claimed-docs] “A scrape lets you turn a URL into easily parseable text.”
- [claimed-docs] “A quick_search lets you quickly web search a query, and pull structured results with urls, titles, and snippets — synchronously, in one requ…”
- [claimed-docs] “Pass webhook_url in the JSON body when starting a run and Riveter POSTs the full results to your URL when it finishes (events: run.completed…”
- [claimed-docs] “the SDKs handle auth, retries (429s and transient GET failures), the wait long-poll, polling until a run finishes (wait_for_result), and pag…”
- [claimed-docs] “run, err := client.Enrich(ctx, riveter.EnrichParams{ Prompt: "Research each company", Attributes: []string{"CEO", "Employee Count"},”
Llm ready output
ai-native userGet clean LLM-ready text directly instead of dealing with blocking, rendering, and messy HTML myself
weight 3 · round to ScrapingBeeScrapingBee offers return_page_markdown to get markdown output plus ai_query for AI-driven extraction and premium proxies/JS rendering to avoid blocking, directly addressing the LLM-ready text need. However, evidence doesn't show a dedicated 'clean text extraction' mode beyond markdown/extract_rules, and no independent benchmarks confirm output quality for LLM consumption. missing for 10: independent validation of markdown/text cleanliness, dedicated boilerplate-removal/reader-mode feature, hands-on confirmation from users of LLM-ready output.
- [claimed-docs] “return_page_markdown [boolean] (false) Return the page content in markdown format”
- [claimed-docs] “ai_query [string] ("") The information you want to extract from the webpage using AI”
- [claimed-docs] “premium_proxy [boolean] (false) Use premium proxies to bypass difficult to scrape websites”
- [claimed-docs] “Our headless browsers will wait for the CSS / Xpath selector passed in the parameter before returning the HTML.”
Docs claim a scrape converts any URL into 'easily parseable text' and that AI agents interpret pages 'the way a person would', directly addressing the ask for clean, LLM-ready text instead of raw HTML. However, all evidence is vendor documentation with no independent hands-on verification of output cleanliness, no example output shown, and no explicit mention of handling JS rendering/blocking obstacles beyond the general claim. Missing for 10: independent corroboration of scrape text quality, concrete example output, and explicit handling of anti-bot/rendering blockers.
- [claimed-docs] “A scrape lets you turn a URL into easily parseable text.”
- [claimed-docs] “Riveter uses AI agents that interpret pages the way a person would, so the same configuration keeps working after a redesign.”
- [claimed-docs] “It reads PDFs and images, calls third party APIs as part of a workflow, and combines those results with data pulled from the web in a single…”
- [claimed-docs] “A quick_search lets you quickly web search a query, and pull structured results with urls, titles, and snippets — synchronously, in one requ…”
ai-native userRequest semantically chunked output instead of one large content blob, so it feeds cleanly into a retrieval pipeline
weight 2 · round drawnScrapingBeenone0/10ScrapingBee offers markdown conversion, CSS-based extraction rules, and AI query extraction, but no evidence of semantic/chunked output splitting content into retrieval-ready segments. The docs list output options (HTML, markdown, screenshots, extract_rules) but never mention chunking or segmenting content for RAG pipelines.
- [claimed-docs] “extract_rules [stringified JSON] ("") Data extraction from CSS selectors”
- [claimed-docs] “ai_query [string] ("") The information you want to extract from the webpage using AI”
- [claimed-docs] “return_page_markdown [boolean] (false) Return the page content in markdown format”
Riveternone0/10Riveter's evidence describes enrichments, scrapes, searches, and structured row outputs, but nothing indicates a semantic-chunking output mode designed for retrieval pipelines (e.g., configurable chunk size/overlap, chunk metadata). Structured rows/columns are not the same as semantic chunking for RAG ingestion, and no such feature is documented.
- [claimed-docs] “An enrichment takes rows of input data and fills in new columns using AI, web searches, web scrapes, and other tools.”
- [claimed-docs] “A scrape lets you turn a URL into easily parseable text.”
- [claimed-docs] “A quick_search lets you quickly web search a query, and pull structured results with urls, titles, and snippets — synchronously, in one requ…”
Visual capture
developerCapture a screenshot of a full page or a specific selected area
weight 2 · round to ScrapingBeeDocs confirm a `screenshot_selector` parameter for capturing a specific CSS-selected area of a page, directly supporting selected-area screenshots. However, no evidence explicitly documents a full-page screenshot parameter or option, so only half the story is substantiated. Missing for 10: explicit full-page screenshot parameter/documentation, independent/hands-on confirmation of screenshot output quality.
- [claimed-docs] “screenshot_selector [string] ("") Return a screenshot of a particular area of the page, targeted by a CSS selector”
Pricing limits — free-tier ceilings, usage caps, and rate limits before you have to payPricing limits
Free-tier ceilings, usage caps, and rate limits before you have to pay
Cost optimization
developerLet the API automatically pick the cheapest configuration that still succeeds
weight 2 · round to ScrapingBeeScrapingBee's docs explicitly describe a `mode=auto` parameter that lets the API pick the cheapest configuration that still succeeds, directly matching the story. This is first-party documented evidence, though there's no independent/hands-on corroboration of its effectiveness. Missing for 10: independent verification that auto mode reliably picks the cheapest successful config in practice.
- [claimed-docs] “mode [string] ("") Let ScrapingBee pick the cheapest configuration that succeeds. Only value is auto”
Riveternone0/10Riveter offers cost controls like dry_run estimates and max_credits caps that refuse overpriced requests, but there is no evidence the API automatically searches for or selects the cheapest configuration that still succeeds — it only estimates/caps, it doesn't auto-optimize. Missing for 10: any documentation of automatic configuration search/optimization for cost, fallback logic that retries cheaper options, or an API parameter that lets Riveter choose the minimal successful config itself.
- [claimed-docs] “dry_run: true — validate the request and return a credit estimate without creating or charging anything.”
- [claimed-docs] “max_credits — a ceiling. When the estimate's maximum exceeds it, the request is refused with 422 credit_cap_exceeded and nothing is charged.”
developerBlock ads on the target page to speed up scraping requests
weight 1 · round to ScrapingBeeOfficial docs explicitly document the `block_ads=true` parameter to prevent ad loading and speed up scraping requests, directly matching the story. Missing for 10: independent/hands-on corroboration of the speed benefit and no third-party benchmark confirming the claim.
- [claimed-docs] “By default, ScrapingBee does not block ads. To avoid scraping them (e.g.,to speed up your request), use `block_ads=true`”
developerBlock images and CSS resources by default to reduce bandwidth and speed up requests
weight 1 · round to ScrapingBeeOfficial docs explicitly state ScrapingBee blocks all images and CSS by default to speed up requests, with an opt-out via block_resources=false, directly matching the story. Missing for 10: independent/hands-on corroboration beyond vendor docs.
- [claimed-docs] “By default, and to speed up requests, ScrapingBee blocks all images and CSS in the scraped page, but to scrape them, use `block_resources=fa…”
ai-native userSet how much reasoning effort an autonomous agent spends on a data-gathering task (low, medium, high)
weight 2 · round drawnScrapingBeenone0/10The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)
Cost transparency
developerWhether exceeding my plan's monthly credit or request quota triggers overage charges or a hard cutoff
weight 3 · round drawnScrapingBeenone0/10No evidence in the pack addresses billing behavior when exceeding plan credits/requests—no mention of overage charges, hard cutoffs, or quota enforcement policy.
Riveternone0/10The evidence describes credit estimation, dry_run, and max_credits cap that refuses requests at 422 before charging, but there is no mention of plan-level monthly credit/request quotas, nor whether exceeding them triggers overage billing or a hard cutoff. missing for 10: any documentation of monthly plan quotas, overage billing policy, or hard-cutoff behavior when a subscription limit is exceeded.
- [claimed-docs] “dry_run: true — validate the request and return a credit estimate without creating or charging anything.”
- [claimed-docs] “max_credits — a ceiling. When the estimate's maximum exceeds it, the request is refused with 422 credit_cap_exceeded and nothing is charged.”
developerWhether failed, blocked, or empty-result requests still consume my billing quota
weight 2 · round drawnScrapingBeenone0/10The evidence pack contains no documentation or discussion of billing behavior for failed, blocked, or empty-result requests—no mention of credit refunds, only-charge-on-success policies, or how failed/blocked scrapes affect quota consumption. Community comments discuss cost/pricing generally but not this specific billing mechanic.
Riveternone0/10The docs describe dry_run cost estimation and max_credits caps that prevent overage, but nothing states whether a failed, blocked, or empty-result run still consumes credits. Missing for 10: explicit policy on billing for failed/empty/blocked runs, any refund or non-charge guarantee for zero-result enrichments.
- [claimed-docs] “dry_run: true — validate the request and return a credit estimate without creating or charging anything.”
- [claimed-docs] “max_credits — a ceiling. When the estimate's maximum exceeds it, the request is refused with 422 credit_cap_exceeded and nothing is charged.”
developerSet a spending cap or usage alert so proxy/credit consumption doesn't silently blow past my budget
weight 3 · round to RiveterScrapingBeenone0/10No evidence of any spending cap, usage alert, or budget notification feature in ScrapingBee's docs or community reports; community comments even highlight cost as a pain point without mentioning any budget-control tooling.
- [community] “Using ScrapingBee is expensive; I've brought the cost of a CRM creation down to about 1.5 cents (+3 cents for a custom cover image) by looki…”
Riveter offers per-request cost control via dry_run (credit estimate before charging) and max_credits (hard ceiling that returns 422 credit_cap_exceeded with nothing charged), which directly prevents a single run from blowing past a set budget. However, there's no evidence of an account-wide spending cap, recurring usage alerts, or a dashboard/notification system for cumulative consumption across runs. Missing for 10: account/org-level budget cap, proactive usage alerts/notifications, historical spend tracking dashboard.
- [claimed-docs] “dry_run: true — validate the request and return a credit estimate without creating or charging anything.”
- [claimed-docs] “max_credits — a ceiling. When the estimate's maximum exceeds it, the request is refused with 422 credit_cap_exceeded and nothing is charged.”
Performance tuning
developerTrade off latency against completeness by controlling exactly when content is returned
weight 1 · round to RiveterScrapingBee lets developers control timing/completeness tradeoffs via wait-for-selector, JS scenarios, block_ads/block_resources flags, and an 'auto' mode that picks the cheapest successful configuration, giving direct levers over latency vs. completeness. However, this is all documented capability with no independent benchmarking or hands-on confirmation of actual latency impact. Missing for 10: independent/hands-on verification of latency-completeness tradeoffs, explicit 'wait' or timeout parameter documentation, and real-world performance data beyond vendor docs.
- [claimed-docs] “If you want to interact with pages you want to scrape before we return your the HTML you can add JavaScript scenario to your API call.”
- [claimed-docs] “Our headless browsers will wait for the CSS / Xpath selector passed in the parameter before returning the HTML.”
- [claimed-docs] “By default, ScrapingBee does not block ads. To avoid scraping them (e.g.,to speed up your request), use `block_ads=true`”
- [claimed-docs] “By default, and to speed up requests, ScrapingBee blocks all images and CSS in the scraped page, but to scrape them, use `block_resources=fa…”
- [claimed-docs] “mode [string] ("") Let ScrapingBee pick the cheapest configuration that succeeds. Only value is auto”
Riveter explicitly exposes multiple latency/completeness tradeoffs: quick_search returns fast synchronous structured snippets, search_agent runs a fuller AI research loop for one question, and full enrichments can be tracked via wait_for_result long-polling or async webhook callbacks — giving a developer direct control over when and how complete the returned content is. missing for 10: no independent/hands-on benchmarks or third-party confirmation of actual latency differences between these modes.
- [claimed-docs] “A quick_search lets you quickly web search a query, and pull structured results with urls, titles, and snippets — synchronously, in one requ…”
- [claimed-docs] “A search_agent call asks one question and gets one AI-researched answer back — the same agent loop that fills a single enrichment cell, with…”
- [claimed-docs] “Pass webhook_url in the JSON body when starting a run and Riveter POSTs the full results to your URL when it finishes (events: run.completed…”
- [claimed-docs] “the SDKs handle auth, retries (429s and transient GET failures), the wait long-poll, polling until a run finishes (wait_for_result), and pag…”
Plan scale limits
data-engineerThe maximum concurrent sessions or requests allowed on my pricing tier and the cost to raise that cap
weight 2 · round drawnScrapingBeenone0/10No evidence in the pack specifies concurrent session/request limits per pricing tier or the cost to increase that cap; documentation snippets cover feature parameters (JS scenario, proxies, extraction) but not concurrency caps or upgrade pricing.
Privacy posture — data-handling and privacy storiesPrivacy posture
Data-handling and privacy stories
ai-native userChoose where my data is stored (region/residency)
weight 2 · round drawnScrapingBeenone0/10Evidence covers proxy geolocation for scraping targets (country_code) but no mention of data residency or storage region controls for ScrapingBee's own data handling/storage; no privacy/compliance documentation is present. Missing for 10: any documentation of data storage regions, residency options, or compliance certifications (e.g., EU data hosting).
- [claimed-docs] “country_code [string] ("") Premium proxy geolocation”
Riveternone0/10No evidence in the pack mentions data residency, regional storage options, or compliance controls for where data is stored; the docs focus entirely on enrichment features and API mechanics. Missing for 10: any mention of region selection, data residency options, or storage location controls.
ai-native userControl data retention and deletion
weight 2 · round drawnScrapingBeenone0/10No evidence in the pack addresses data retention policies, deletion controls, or privacy/data lifecycle management for ScrapingBee; documentation excerpts focus solely on scraping features and API parameters. Missing for 10: any mention of data retention windows, deletion APIs/requests, privacy policy details, or compliance certifications.
ai-native userOpt out of telemetry and usage tracking
weight 2 · round drawnScrapingBeenone0/10No evidence pack mentions telemetry, usage tracking, or an opt-out mechanism for ScrapingBee's own product usage; documentation excerpts focus solely on scraping API parameters.
Scale reliability — behavior under load — scaling limits, uptime, failure handlingScale reliability
Behavior under load — scaling limits, uptime, failure handling
Ai driven crawling
ai-native userRely on adaptive crawling that automatically stops once enough information has been gathered to answer my query
weight 2 · round to RiveterScrapingBeenone0/10ScrapingBee's docs describe single-page scraping, AI-based extraction (ai_query), and cost-optimizing 'auto' mode, but there is no evidence of adaptive multi-step crawling that dynamically decides when enough information has been gathered to stop. No crawling/agentic loop or stopping-criteria feature is documented.
- [claimed-docs] “ai_query [string] ("") The information you want to extract from the webpage using AI”
- [claimed-docs] “mode [string] ("") Let ScrapingBee pick the cheapest configuration that succeeds. Only value is auto”
Riveter's search_agent and enrichment agent loop imply some autonomous research process that fills a cell with an AI-researched answer, suggesting the agent decides when it has enough data, but there is no explicit documentation of stopping criteria or adaptive crawling behavior tied to query sufficiency. missing for 10: explicit description of adaptive stopping/crawling logic, evidence of how the agent determines 'enough information', independent confirmation of this behavior in practice.
- [claimed-docs] “A search_agent call asks one question and gets one AI-researched answer back — the same agent loop that fills a single enrichment cell, with…”
- [claimed-docs] “An enrichment takes rows of input data and fills in new columns using AI, web searches, web scrapes, and other tools.”
Batch processing
data-engineerBatch scrape thousands of URLs asynchronously
weight 3 · round to RiveterScrapingBeenone0/10The evidence pack only documents single-URL synchronous scraping API parameters (JS rendering, extraction rules, proxies, screenshots) with no mention of batch job submission, async processing, concurrency limits, or a queue/webhook system for handling thousands of URLs at scale.
Riveter's enrichment engine explicitly processes rows of URLs with scraping, runs asynchronously (webhook_url on completion), and SDKs handle retries, long-polling, and pagination — all core pieces for async batch scraping. However, there's no explicit documentation of scale limits, concurrency handling, or a tested example at thousands-of-URLs volume. Missing for 10: explicit large-scale (thousands of URLs) benchmarks or case studies, concurrency/rate-limit guidance for very large batches.
- [claimed-docs] “An enrichment takes rows of input data and fills in new columns using AI, web searches, web scrapes, and other tools.”
- [claimed-docs] “A scrape lets you turn a URL into easily parseable text.”
- [claimed-docs] “Pass webhook_url in the JSON body when starting a run and Riveter POSTs the full results to your URL when it finishes (events: run.completed…”
- [claimed-docs] “the SDKs handle auth, retries (429s and transient GET failures), the wait long-poll, polling until a run finishes (wait_for_result), and pag…”
- [claimed-docs] “It can find every dental practice in a city, then pull every dentist from each one, in a single request.”
developerApply different crawl configurations to different URL patterns within a single batch job
weight 1 · round drawnScrapingBeenone0/10ScrapingBee's API is per-URL request based with configuration parameters set per call; there is no evidence of a 'batch job' concept or a way to define per-URL-pattern rules within a single job. The docs describe single-page scraping options (JS scenario, extract_rules, proxies, etc.) but nothing about batch jobs with pattern-based configuration.
Riveternone0/10No evidence describes applying different crawl configurations per URL pattern within one batch/enrichment job; docs mention scraping, searching, and enrichment generally but not per-pattern configuration rules. missing for 10: any mention of per-URL-pattern rules or configuration scoping within a single job, examples or docs showing mixed crawl settings in one batch.
Concurrency
data-engineerSpin up many concurrent scraping sessions to gather data at scale
weight 3 · round to RiveterScrapingBee is inherently an API you can call many times, and docs mention session_id for routing multiple requests through the same IP, but the evidence pack contains no explicit documentation of concurrency limits, parallel-request quotas, or scaling architecture for high-volume data-engineering workloads. Missing for 10: explicit concurrency/rate-limit specs, documented plan-based concurrent request caps, and independent evidence of successful large-scale concurrent scraping.
- [claimed-docs] “session_id [integer] ("") Route multiple API requests through the same IP address”
- [claimed-docs] “premium_proxy [boolean] (false) Use premium proxies to bypass difficult to scrape websites”
- [claimed-docs] “To scrape a web page, you only need two things: Your API key... The encoded web page URL you want to scrape”
Riveter's enrichment engine processes many rows in a single run and can chain scrapes/searches (e.g., finding every dental practice then every dentist in one request), implying built-in batch/bulk scraping at scale, and SDKs handle retries/pagination for large jobs. However, there is no explicit documentation of concurrency limits, parallel session management, or throughput guarantees for scraping specifically. Missing for 10: explicit concurrency/session limits, performance benchmarks, and independent evidence of scaling to many simultaneous scrape sessions.
- [claimed-docs] “An enrichment takes rows of input data and fills in new columns using AI, web searches, web scrapes, and other tools.”
- [claimed-docs] “A scrape lets you turn a URL into easily parseable text.”
- [claimed-docs] “It can find every dental practice in a city, then pull every dentist from each one, in a single request.”
- [claimed-docs] “the SDKs handle auth, retries (429s and transient GET failures), the wait long-poll, polling until a run finishes (wait_for_result), and pag…”
- [claimed-docs] “Schedule any project to monitor for changes and keep your data fresh.”
Crawl compliance
data-engineerConfigure the crawler to respect robots.txt rules and target-site rate limits automatically
weight 2 · round drawnScrapingBeenone0/10No evidence that ScrapingBee offers robots.txt compliance settings or automatic rate-limit throttling per target site; docs cover proxies, JS rendering, extraction, and viewport settings but nothing about robots.txt or rate-limiting configuration.
Riveternone0/10No evidence anywhere in the docs mentions robots.txt compliance or rate-limit configuration; the pack only covers scraping features, retries, credits, and MCP integration. This is a fair axis for a web-scraping/crawling product, but absence of evidence means it cannot be credited as delivered.
Fault tolerance
data-engineerResume a crashed deep crawl from a saved checkpoint instead of restarting from scratch
weight 2 · round drawnScrapingBeenone0/10No evidence of any crawl checkpoint/resume feature; ScrapingBee's docs describe single-page API requests, sessions, and proxy parameters but nothing about deep crawl state persistence or resuming crashed crawls.
Operational transparency
data-engineerCheck a public status page showing uptime history and past incident postmortems before committing to the service
weight 2 · round drawnScrapingBeenone0/10No evidence of a public status page, uptime history, or incident postmortems anywhere in the evidence pack; docs focus on API features and community items discuss cost/alternatives, not reliability transparency.
Scheduling monitoring
data-engineerMonitor target pages for content changes, such as price or listing updates, and get notified as they happen
weight 2 · round to RiveterScrapingBeenone0/10ScrapingBee is an on-demand scraping API (fetch a page, extract data, render JS) with no evidence of scheduled monitoring, change-detection, diffing, or notification/webhook features for tracking content changes over time. The evidence pack only covers single-request scraping parameters, proxies, and rendering options, not continuous monitoring or alerting.
Riveter explicitly supports scheduling projects to monitor for changes, refreshing as often as every minute, and can POST results to a webhook_url when a run finishes, which together deliver change-monitoring plus notification. However, the webhook fires on run completion rather than a dedicated 'content changed' diff event, and there's no independent/hands-on evidence of this workflow in production. Missing for 10: independent corroboration of the schedule+webhook pipeline in practice, and explicit diff/change-detection logic distinguishing 'changed' vs 'unchanged' pages.
- [claimed-docs] “Schedule any project to monitor for changes and keep your data fresh.”
- [claimed-docs] “For fast moving data like scores or election results, you can refresh as often as every minute.”
- [claimed-docs] “Pass webhook_url in the JSON body when starting a run and Riveter POSTs the full results to your URL when it finishes (events: run.completed…”
data-engineerMonitor job performance, validate data quality, and receive alerts when something fails
weight 2 · round to RiveterScrapingBeenone0/10Evidence covers scraping features (JS rendering, extraction, proxies) but nothing about job monitoring dashboards, data quality validation, or failure alerting mechanisms; community comments focus on cost/alternatives, not reliability tooling.
Riveter supports webhook alerts on run completion/stop/finish events and scheduled monitoring for data freshness, giving some job-status alerting and monitoring capability, but there is no explicit data-quality validation feature (e.g., schema/anomaly checks) or job performance dashboards described. missing for 10: explicit data quality validation tooling, job performance metrics/dashboard, and independent confirmation of alerting reliability.
- [claimed-docs] “Pass webhook_url in the JSON body when starting a run and Riveter POSTs the full results to your URL when it finishes (events: run.completed…”
- [claimed-docs] “Schedule any project to monitor for changes and keep your data fresh.”
- [claimed-docs] “dry_run: true — validate the request and return a credit estimate without creating or charging anything.”
- [claimed-docs] “max_credits — a ceiling. When the estimate's maximum exceeds it, the request is refused with 422 credit_cap_exceeded and nothing is charged.”
developerMonitor live system metrics and worker/browser pool status through a real-time dashboard
weight 1 · round drawnScrapingBeenone0/10The evidence pack covers API parameters, docs, and community discussion but contains no mention of a real-time dashboard for monitoring system metrics or worker/browser pool status; ScrapingBee's dashboard (if any) is not documented here.
developerSchedule scraping jobs to run automatically at specific times
weight 2 · round to RiveterScrapingBeenone0/10ScrapingBee's evidence describes only on-demand API scraping (parameters, JS rendering, proxies, extraction) with no mention of a scheduling feature, cron-like triggers, or job scheduler UI. No evidence supports automated, time-based recurring scraping jobs.
Riveter supports scheduling projects to monitor for changes and refresh data as often as every minute, which implies automatic recurring scraping jobs, but there's no detail on specifying exact times/cron-like scheduling, timezone control, or a documented scheduling API/UI. missing for 10: explicit scheduling configuration details (time-of-day, cron syntax, timezone), independent/hands-on confirmation of scheduling reliability, and API endpoint documentation for creating/managing schedules.
- [claimed-docs] “Schedule any project to monitor for changes and keep your data fresh.”
- [claimed-docs] “For fast moving data like scores or election results, you can refresh as often as every minute.”
Site crawling
data-engineerRun a deep crawl using a breadth-first strategy with a configurable maximum page limit
weight 2 · round drawnScrapingBeenone0/10ScrapingBee's documented API is per-page scraping (single URL requests with rendering, extraction, proxy options) with no evidence of a crawl orchestration feature supporting breadth-first traversal or a configurable max-page limit for multi-page crawls.
- [claimed-docs] “To scrape a web page, you only need two things: Your API key... The encoded web page URL you want to scrape”
- [claimed-docs] “extract_rules [stringified JSON] ("") Data extraction from CSS selectors”
developerCrawl an entire website and get content from all its pages with one request
weight 3 · round to RiveterScrapingBeenone0/10ScrapingBee's API is designed for single-page scraping requests (one URL per call); the evidence shows no crawler feature that follows links across a domain or aggregates content from multiple pages in one request. No mention of a 'crawl' endpoint, sitemap traversal, or multi-page job in a single API call.
- [claimed-docs] “To scrape a web page, you only need two things: Your API key... The encoded web page URL you want to scrape”
- [claimed-docs] “This can be useful for scraping a Single Page Application built with frameworks such as React.js, Angular.js, JQuery or Vue.”
Riveter's docs describe single-URL 'scrape' and 'quick_search' calls, but the marketing example of finding every dental practice in a city and pulling data from each one in a single request shows it can aggregate content across multiple pages/sources in one enrichment run, which approximates whole-site crawling. There is no explicit sitemap-style 'crawl entire website' feature or evidence of full-domain page enumeration. missing for 10: explicit full-site/sitemap crawl feature, evidence of automatically discovering and traversing all pages of a single domain, independent confirmation of multi-page crawl behavior.
- [claimed-docs] “A scrape lets you turn a URL into easily parseable text.”
- [claimed-docs] “It can find every dental practice in a city, then pull every dentist from each one, in a single request.”
- [claimed-docs] “It reads PDFs and images, calls third party APIs as part of a workflow, and combines those results with data pulled from the web in a single…”
developerInstantly discover all URLs on a website without fully crawling it
weight 2 · round drawnScrapingBeenone0/10ScrapingBee's evidence covers page scraping, JS rendering, extraction rules, proxies, and AI queries, but nothing describes a sitemap/URL-discovery feature that lists all URLs on a site without crawling each page. No sitemap parsing, URL enumeration, or site-mapping endpoint is documented.
Not comparable on these axes
ai-native userPlug MCP servers into this product so it can use their tools
weight 3 · not comparableScrapingBeen/aScrapingBee is a web-scraping API/SaaS product, not an agent or orchestration platform that would itself consume other MCP servers' tools; evidence only shows it exposes its own MCP server (mcp.scrapingbee.com), i.e., it is the tool provider, not a tool consumer. Plugging external MCP servers into ScrapingBee to gain their tools is a category error for this kind of product.
- [probe] “official MCP server documented at https://mcp.scrapingbee.com/”
Riveternone0/10All MCP-related evidence describes Riveter acting as an MCP *server* that other AI assistants (Claude, ChatGPT, Cursor) connect to, not Riveter acting as a client that can plug in and consume external MCP servers' tools. No evidence shows Riveter itself importing/connecting to third-party MCP servers to extend its own tool use.
- [claimed-docs] “Use Riveter from Claude, ChatGPT, Cursor, or any MCP-compatible AI assistant. Pick one of the two ways to connect.”
- [claimed-docs] “Runs on your machine and needs Node.js and an API key. Use it when your client cannot reach remote servers.”
ai-native userVersion, review, and roll back my automations
weight 1 · not comparableScrapingBeen/aScrapingBee is a web scraping API/proxy service, not an automation-building platform with workflows to version or roll back; versioning/review/rollback of automations is a category error for this product type.
Riveternone0/10No evidence of version history, review workflows, or rollback capability for automations/enrichments; the pack only covers run execution, credit control, and data enrichment features. Missing for 10: versioning of automation configs, review/approval workflow, rollback/undo mechanism.
ai-native userPrevent my data from being used to train AI models
weight 3 · not comparableScrapingBeenone0/10No evidence in the pack addresses data-use, training-data opt-out, or AI-training privacy policies for ScrapingBee's service; nothing documents a mechanism to prevent scraped/customer data from being used to train AI models.
Rivetern/aRiveter is a data enrichment/web-scraping tool, not an AI model provider or chat assistant whose data usage/training policies would be relevant here; the evidence contains no mention of AI training data policy or opt-out controls, and this axis is a category error for the product type.