Document Extraction APIs Arena
LlamaParse vs Extend
Extend wins · 11–25 (15 drawn)
Agenticness — how well agents can access and operate the productAgenticness
How well agents can access and operate the product
Agent access
ai-native userPoint an agent at llms.txt or agent-oriented docs
weight 2 · round to ExtendAn llms.txt file is confirmed live via probe (HTTP 200), and LlamaIndex also documents an explicit 'for-agents' page with an MCP server and agent-oriented docs tools (search_docs, grep_docs, read_doc) designed for agents to consume documentation programmatically. missing for 10: independent/community confirmation that agents actually use llms.txt successfully in practice, and the docs-md alternate format probe returned 404 showing incomplete agent-doc surface.
- [probe] “PROBE llms.txt: HTTP 200 at https://developers.llamaindex.ai/llms.txt # LlamaIndex Documentation > LlamaIndex is a framework for building L…”
- [claimed-docs] “My agent should be able to search these docs”
- [claimed-docs] “The server ships with the following tools: 1. `search_docs` — a basic lexical search using BM25 2. `grep_docs` — exact search using regex 3.…”
- [claimed-docs] “search_docs — a basic lexical search using BM25 2. grep_docs — exact search using regex 3. read_doc — provides an interface to read the enti…”
- [probe] “official MCP server documented at https://developers.llamaindex.ai/for-agents/mcp/”
- [probe] “PROBE docs-md: HTTP 404 at https://developers.llamaindex.ai/llamaparse/.md”
Extend has a confirmed live llms.txt at docs.extend.ai/llms.txt (HTTP 200) providing agent-oriented documentation, alongside agent-oriented docs like the Agent Quickstart and MCP integration docs that an AI agent could consume directly. Missing for 10: no independent/community confirmation of an agent actually consuming llms.txt successfully in practice.
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.extend.ai/llms.txt # Extend > Extend is a platform for building, iterating on, evaluating, and dep…”
- [claimed-docs] “Agent Quickstart | Extend Documentation”
- [claimed-docs] “Using Extend via MCP | Extend Documentation”
ai-native userRun the product headlessly / in CI for automation
weight 2 · round to LlamaParseLlamaParse offers a REST API, CLI, and SDKs (Python/TypeScript/Go/Java) suitable for scripted/CI use, plus webhooks for async job notification instead of polling, which supports headless automation workflows. Missing for 10: explicit CI/CD examples (GitHub Actions, Docker recipes), documented non-interactive auth/service-account flow for pipelines, and independent hands-on confirmation of headless CI usage.
- [claimed-docs] “Get your first parse job running in under a minute—whether you prefer Python, TypeScript, Go, Java, the CLI, the REST API, or the Web UI.”
- [claimed-docs] “Instead of continuously polling for status updates, you can configure webhook endpoints to be notified immediately when jobs complete, fail,…”
- [claimed-docs] “result = client.parsing.parse( file_id=file.id, tier="agentic", version="latest", expand=["markdown"],)”
Extend offers a documented CLI, Python SDK (pip install), and REST API with sync/async modes and a test API key mode, all of which support headless/CI-driven automation of document processing. However, there is no explicit CI/CD integration guide, GitHub Actions example, or independent hands-on report of running it in a CI pipeline. missing for 10: explicit CI pipeline examples/integration guide, independent hands-on verification of headless CI usage, and details on exit codes/CLI scripting ergonomics for automation.
- [claimed-docs] “pip install extend-ai”
- [claimed-docs] “CLI | Extend Documentation”
- [claimed-docs] “SDKs | Extend Documentation”
- [claimed-docs] “API Quickstart | Extend Documentation”
- [claimed-docs] “Every processing endpoint in Extend (extract, classify, split, parse, and edit) supports both a synchronous and asynchronous mode.”
- [claimed-docs] “Runs made with a test API key never touch production data and never fire production webhooks”
- [probe] “official CLI documented at https://docs.extend.ai/cli”
ai-native userConnect an agent via an official MCP server
weight 3 · round drawnFirst-party docs explicitly describe an official MCP endpoint (https://mcp.llamaindex.ai/mcp) that exposes Parse, Classify, Extract, Split, and Index as callable tools for any MCP client, and a dedicated docs page details this server. Missing for 10: independent/hands-on confirmation that the MCP server works as documented and clarity on how it relates to the separate docs-search MCP tools (search_docs/grep_docs/read_doc) mentioned elsewhere.
- [claimed-docs] “https://mcp.llamaindex.ai/mcp is the endpoint that exposes the platform's document processing — Parse, Classify, Extract, Split, and Index —…”
- [claimed-docs] “`https://mcp.llamaindex.ai/mcp` is the endpoint that exposes the platform's document processing — Parse, Classify, Extract, Split, and Index…”
- [probe] “official MCP server documented at https://developers.llamaindex.ai/for-agents/mcp/”
- [claimed-docs] “The server ships with the following tools: 1. `search_docs` — a basic lexical search using BM25 2. `grep_docs` — exact search using regex 3.…”
Extend is a document-processing platform (not an agent itself), and it documents an official MCP server ('Using Extend via MCP' at docs.extend.ai/mcp), confirmed by probe as live documentation, allowing AI agents to connect via MCP. missing for 10: no independent/hands-on confirmation of the MCP server working in practice, and limited detail on setup specifics beyond the doc title.
- [claimed-docs] “Using Extend via MCP | Extend Documentation”
- [probe] “official MCP server documented at https://docs.extend.ai/mcp”
ai-native userUse an official CLI
weight 2 · round to ExtendDocs mention the CLI as one of several supported interfaces to run parse jobs ('whether you prefer Python, TypeScript, Go, Java, the CLI, the REST API, or the Web UI') but no dedicated CLI reference, install instructions, or command examples are provided in the evidence pack. Missing for 10: CLI installation/usage docs, command examples, independent/community confirmation of CLI usage.
- [claimed-docs] “Get your first parse job running in under a minute—whether you prefer Python, TypeScript, Go, Java, the CLI, the REST API, or the Web UI.”
Extend documents an official CLI at docs.extend.ai/cli, alongside SDKs and MCP support, indicating a first-party CLI exists for AI-native workflows. Missing for 10: no independent/hands-on confirmation of CLI usage or specific command examples beyond the docs listing page.
- [claimed-docs] “CLI | Extend Documentation”
- [probe] “official CLI documented at https://docs.extend.ai/cli”
ai-native userDrive the product through a documented public API
weight 3 · round to ExtendLlamaParse offers a documented REST API plus SDKs (Python, TypeScript, Go, Java) and a CLI, with example code snippets for calling the parse endpoint, and webhook support for async job notification. Independent probes could not find a public OpenAPI spec, which slightly weakens strict machine-readability of the API. missing for 10: publicly discoverable OpenAPI/swagger spec, independent third-party confirmation of API robustness.
- [claimed-docs] “Get your first parse job running in under a minute—whether you prefer Python, TypeScript, Go, Java, the CLI, the REST API, or the Web UI.”
- [claimed-docs] “result = client.parsing.parse( file_id=file.id, tier="agentic", version="latest", expand=["markdown"],)”
- [claimed-docs] “Instead of continuously polling for status updates, you can configure webhook endpoints to be notified immediately when jobs complete, fail,…”
- [claimed-docs] “One API key, one SDK, and five composable products: Parse (agentic OCR), Extract (structured data), Classify, Split, and Index.”
- [probe] “PROBE openapi: all candidate paths 404 (https://developers.llamaindex.ai/openapi.json, https://developers.llamaindex.ai/swagger.json, https:…”
Extend provides a documented public API with authentication, endpoints (upload, extract, classify, split, parse), sync/async modes, SDKs, CLI, and a dedicated API Quickstart, giving AI-native users a clear path to drive the product programmatically. Community feedback confirms real-world usage of the product's data extraction capabilities. missing for 10: independent third-party validation specifically of API robustness/rate limits beyond docs.
- [claimed-docs] “Authenticate a request”
- [claimed-docs] “SDKs | Extend Documentation”
- [claimed-docs] “API Quickstart | Extend Documentation”
- [claimed-docs] “This endpoint accepts file contents and registers them as a File in Extend, which can be used for running workflows, creating evaluation set…”
- [claimed-docs] “Every processing endpoint in Extend (extract, classify, split, parse, and edit) supports both a synchronous and asynchronous mode.”
- [claimed-docs] “CLI | Extend Documentation”
- [community] “We've been using Extend for over a year and have been super happy with the product and accuracy of the data extraction.”
ai-native userIssue scoped/least-privilege API credentials for an agent
weight 2 · round drawnLlamaParsenone0/10Docs mention SSO and role-based access controls for managing org/project access (llamaparse-docs-9, llamaparse-docs-18), but there is no evidence of scoped or least-privilege API key/credential issuance specifically for agents (e.g., per-key permission scopes, agent-specific tokens). This is a fair axis for an API platform, so absence of evidence yields 'none' rather than 'na'.
- [claimed-docs] “Identity and access. Use SSO and role-based access controls to manage access to your organization and projects.”
- [claimed-docs] “Use SSO and role-based access controls to manage access to your organization and projects.”
Extendnone0/10Evidence only shows basic API-key authentication (extend-docs-4) and a separate test-vs-production API key for sandboxing (extend-docs-38), but nothing about issuing scoped, role/permission-limited, or least-privilege credentials for individual agents (e.g., granular scopes, RBAC, per-agent key restrictions).
ai-native userBuild against official SDKs
weight 2 · round drawnLlamaParse provides official SDKs across multiple languages (Python, TypeScript, Go, Java) plus CLI and REST API, documented with code examples (e.g., client.parsing.parse), and maintained as a public GitHub package (llama-cloud) with migration guidance. This directly supports building AI-native apps against official first-party SDKs rather than reverse-engineered clients. missing for 10: independent third-party benchmarking of SDK reliability/versioning, and no OpenAPI spec found (probe-3) to corroborate REST completeness
- [claimed-docs] “Get your first parse job running in under a minute—whether you prefer Python, TypeScript, Go, Java, the CLI, the REST API, or the Web UI.”
- [claimed-docs] “result = client.parsing.parse( file_id=file.id, tier="agentic", version="latest", expand=["markdown"],)”
- [github] “Please migrate to the new packages: - **Python**: `pip install llama-cloud>=1.0`”
- [claimed-docs] “One API key, one SDK, and five composable products: Parse (agentic OCR), Extract (structured data), Classify, Split, and Index.”
Extend documents an official Python SDK (pip install extend-ai), a dedicated SDKs docs page, API quickstart, authentication docs, and API reference endpoints, giving AI-native developers a clear official SDK path alongside CLI and MCP integrations. missing for 10: independent/hands-on developer corroboration of SDK usage beyond first-party docs, and multi-language SDK coverage details.
- [claimed-docs] “pip install extend-ai”
- [claimed-docs] “SDKs | Extend Documentation”
- [claimed-docs] “API Quickstart | Extend Documentation”
- [claimed-docs] “Authenticate a request”
- [claimed-docs] “This endpoint accepts file contents and registers them as a File in Extend, which can be used for running workflows, creating evaluation set…”
ai-native userSubscribe to events via webhooks
weight 2 · round drawnDocs explicitly describe configuring webhook endpoints to be notified when parse jobs complete, fail, or reach other states, directly matching the story. Missing for 10: independent/hands-on confirmation of webhook reliability, payload schema/signature verification details, and community corroboration of real-world webhook usage.
- [claimed-docs] “Instead of continuously polling for status updates, you can configure webhook endpoints to be notified immediately when jobs complete, fail,…”
Extend has a dedicated 'Webhook Configuration' documentation page and separately confirms webhooks fire on production runs (and are suppressed in test mode), showing an established webhook subscription mechanism for async events. Missing for 10: specifics on supported event types, payload schema, and independent/hands-on verification of webhook reliability.
- [claimed-docs] “Webhook Configuration | Extend Documentation”
- [claimed-docs] “Runs made with a test API key never touch production data and never fire production webhooks”
- [claimed-docs] “Async Processing | Extend Documentation”
Agentic features
ai-native userGet AI-generated insights and suggestions from my data inside the product
weight 2 · round to ExtendLlamaParsenone0/10LlamaParse/LlamaCloud is documented as a document parsing, extraction, classification, and indexing pipeline driven by user-defined schemas and rules, not a product that surfaces its own AI-generated insights or suggestions about the user's data inside a UI. Evidence covers extraction, classification, splitting, and RAG-indexing capabilities, but none describe an insights/suggestion feature (e.g., auto-generated summaries, anomaly flags, or recommendations) exposed to the user. missing for 10: any documented insights/suggestions UI or feature, evidence of autonomous analysis surfaced to users, independent confirmation of such a capability.
- [claimed-docs] “LlamaExtract provides a simple API for extracting structured data from unstructured documents like PDFs, text files, and images.”
- [claimed-docs] “Just define your Extraction Configuration (schema and settings), drag and drop any supported document into LlamaParse and extract data from …”
- [claimed-docs] “Generate extraction schemas with a prompt”
- [claimed-docs] “One API key, one SDK, and five composable products: Parse (agentic OCR), Extract (structured data), Classify, Split, and Index.”
Extend's core AI extraction/classification pipeline generates structured data, per-field confidence scores that suggest which values to trust vs. route to human review, and citations pointing back to source locations — a form of AI-generated insight/suggestion from document data. However, there's no evidence of broader analytical insights, trend/summary generation, or proactive suggestions beyond the extraction/review-routing use case. Missing for 10: dedicated 'insights' or summary-generation feature, evidence of proactive suggestions beyond confidence-based review routing, and independent corroboration of this specific capability.
- [claimed-docs] “Use them to trust high-confidence values automatically and route the rest to human review.”
- [claimed-docs] “plus per-field confidence scores and citations that point back to the exact spot on the page”
- [claimed-docs] “Agentic OCR”
- [claimed-docs] “Parse, extract, and split your hardest documents with unmatched accuracy. Ship reliable document agents in minutes, not months.”
- [claimed-docs] “Confidence Scores | Extend Documentation”
ai-native userSet up automations that run autonomously in the background
weight 2 · round to ExtendLlamaParse supports asynchronous job processing with webhook notifications instead of polling, which enables integration into automated pipelines that run without manual monitoring, and MCP tool exposure lets agents trigger parsing autonomously as part of larger workflows. However, there's no evidence of native scheduling, cron-like triggers, or persistent background jobs that run independently of an external caller invoking them. Missing for 10: scheduled/triggered automation, autonomous recurring job execution, evidence of agents independently initiating parse jobs on a schedule.
- [claimed-docs] “Instead of continuously polling for status updates, you can configure webhook endpoints to be notified immediately when jobs complete, fail,…”
- [claimed-docs] “https://mcp.llamaindex.ai/mcp is the endpoint that exposes the platform's document processing — Parse, Classify, Extract, Split, and Index —…”
- [claimed-docs] “`https://mcp.llamaindex.ai/mcp` is the endpoint that exposes the platform's document processing — Parse, Classify, Extract, Split, and Index…”
- [probe] “official MCP server documented at https://developers.llamaindex.ai/for-agents/mcp/”
Extend's Workflows feature (versioned, deployable pipelines) combined with async processing and webhook callbacks lets document-processing tasks run without a user waiting live, which is the core of 'background automation' — evidenced by extend-docs-19, extend-docs-12, extend-docs-13, extend-docs-37, extend-docs-43. However there's no explicit mention of scheduling/triggers (e.g., cron, event subscriptions beyond webhooks) or autonomous multi-step agent loops running unattended over time, so the 'autonomous' framing is only partially substantiated. Missing for 10: explicit scheduling/trigger mechanisms, evidence of long-running unattended agent loops, and independent confirmation that workflows execute fully autonomously without manual intervention.
- [claimed-docs] “Workflows”
- [claimed-docs] “Webhook Configuration | Extend Documentation”
- [claimed-docs] “Async Processing | Extend Documentation”
- [claimed-docs] “Every processing endpoint in Extend (extract, classify, split, parse, and edit) supports both a synchronous and asynchronous mode.”
- [claimed-docs] “A workflow has one editable draft and any number of immutable deployed versions.”
ai-native userDelegate tasks to a built-in AI assistant inside the product
weight 3 · round to ExtendLlamaParsenone0/10LlamaParse's evidence describes it as a document parsing/extraction API (Parse, Extract, Classify, Split, Index) callable via SDKs, CLI, REST, or exposed to external agents via an MCP server — but there is no mention of a built-in AI assistant inside the product itself that a user could converse with or delegate tasks to. The MCP endpoint (llamaparse-docs-8/16/27) lets external agents call LlamaParse's tools, which is the reverse direction of this story.
Extend markets built-in agentic features like "Composer & Review Agent," "Agentic OCR," and "Agent form filling," plus an "Agent Quickstart" guide, suggesting users can delegate document-processing tasks to built-in AI agents. However, there's no clear evidence of a general-purpose conversational AI assistant inside the product UI for delegating arbitrary tasks — the agentic features are narrowly scoped to document extraction/review workflows. Missing for 10: a documented chat/assistant interface, examples of delegating broader tasks beyond document processing, and independent hands-on confirmation of these agent features in use.
- [claimed-docs] “Composer & Review Agent”
- [claimed-docs] “Agentic OCR”
- [claimed-docs] “Agent form filling”
- [claimed-docs] “Agent Quickstart | Extend Documentation”
- [claimed-docs] “Ship reliable document agents in minutes, not months.”
ai-native userOperate the product with natural-language commands
weight 2 · round drawnLlamaParse supports several natural-language-driven controls: Classify uses natural-language rules, Extract can generate schemas from a prompt, and Parse output can be steered with custom natural-language prompts (e.g., focusing only on prices in a receipt). It also exposes an MCP endpoint so agents can invoke Parse/Classify/Extract/Split/Index via tool calls, which is agent-friendly but not itself a natural-language command interface. Missing for 10: a unified conversational/chat interface for operating the whole platform, and independent hands-on evidence confirming natural-language commands reliably control parsing behavior end-to-end.
- [claimed-docs] “Classify lets you automatically categorize documents into types you define (for example: invoice, receipt, contract) using natural-language …”
- [claimed-docs] “Generate extraction schemas with a prompt”
- [claimed-docs] “Guide parsing output with custom prompts. Extract specific data from a McDonald's receipt by instructing the parser to focus only on prices …”
- [claimed-docs] “https://mcp.llamaindex.ai/mcp is the endpoint that exposes the platform's document processing — Parse, Classify, Extract, Split, and Index —…”
- [claimed-docs] “`https://mcp.llamaindex.ai/mcp` is the endpoint that exposes the platform's document processing — Parse, Classify, Extract, Split, and Index…”
Extend documents an official MCP server and Agent Quickstart, which allow AI agents/assistants to drive the platform via natural-language tool calls rather than raw API calls, and it also ships a CLI for programmatic control. However, there is no first-party example or independent corroboration showing an end-user issuing free-form natural-language commands and Extend interpreting/executing them directly. Missing for 10: concrete examples of natural-language command execution, independent/hands-on validation of the MCP/agent workflow.
- [claimed-docs] “Using Extend via MCP | Extend Documentation”
- [claimed-docs] “Agent Quickstart | Extend Documentation”
- [probe] “official MCP server documented at https://docs.extend.ai/mcp”
- [probe] “official CLI documented at https://docs.extend.ai/cli”
- [claimed-docs] “CLI | Extend Documentation”
Api quality
ai-native userExplore an interactive API reference with runnable examples
weight 2 · round to ExtendLlamaParsenone0/10Docs show many static code snippets/examples (Python calls, curl-like usage) but there is no evidence of an interactive, runnable API reference (e.g., Swagger/OpenAPI explorer or live code sandbox); explicit probes for OpenAPI/Swagger endpoints returned 404s, indicating no such interactive reference exists.
- [probe] “PROBE openapi: all candidate paths 404 (https://developers.llamaindex.ai/openapi.json, https://developers.llamaindex.ai/swagger.json, https:…”
- [claimed-docs] “result = client.parsing.parse( file_id=file.id, tier="agentic", version="latest", expand=["markdown"],)”
- [claimed-docs] “pull every table out into pandas DataFrames with source-page provenance, and turn on Cost Optimizer to keep costs down on long mixed-complex…”
- [claimed-docs] “Guide parsing output with custom prompts. Extract specific data from a McDonald's receipt by instructing the parser to focus only on prices …”
Extend has a documented API reference (authentication, endpoints like upload-file, quickstarts) and SDKs/CLI, but there's no evidence of an interactive 'try it' console or runnable code examples embedded in the docs. missing for 10: explicit interactive API playground, in-browser runnable examples, live request/response testing.
- [claimed-docs] “Authenticate a request”
- [claimed-docs] “API Quickstart | Extend Documentation”
- [claimed-docs] “This endpoint accepts file contents and registers them as a File in Extend, which can be used for running workflows, creating evaluation set…”
- [claimed-docs] “SDKs | Extend Documentation”
- [claimed-docs] “CLI | Extend Documentation”
ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)
weight 2 · round drawnLlamaParsenone0/10LlamaParse exposes a REST API, but there is no evidence of a downloadable OpenAPI/Swagger spec; explicit probes for common OpenAPI endpoints (openapi.json, swagger.json, etc.) all returned 404.
- [probe] “PROBE openapi: all candidate paths 404 (https://developers.llamaindex.ai/openapi.json, https://developers.llamaindex.ai/swagger.json, https:…”
- [claimed-docs] “Get your first parse job running in under a minute—whether you prefer Python, TypeScript, Go, Java, the CLI, the REST API, or the Web UI.”
Extendnone0/10Extend documents a full API reference, authentication, and multiple endpoints, but no evidence pack item mentions a downloadable OpenAPI/Swagger spec or any machine-readable API definition file. missing for 10: OpenAPI/Swagger spec file, documented spec download link, machine-readable API schema.
ai-native userTest against a sandbox environment without touching production data
weight 1 · round to ExtendLlamaParsenone0/10No evidence describes a dedicated sandbox/test environment separate from production for LlamaParse; free credits and self-hosting/BYOC options are mentioned but do not constitute a documented sandbox mode for testing without touching production data.
Extend explicitly documents a test environment where runs made with a test API key never touch production data and never fire production webhooks, directly matching the sandbox story. missing for 10: no independent/hands-on corroboration of the test-key isolation beyond first-party docs, and no detail on how thoroughly sandbox data mirrors production for realistic testing.
- [claimed-docs] “Runs made with a test API key never touch production data and never fire production webhooks”
ai-native userRely on versioned APIs with a documented deprecation policy
weight 2 · round to LlamaParseThere is a version parameter in the parse API (version="latest") and a GitHub migration notice telling users to move to llama-cloud>=1.0, showing some versioning/migration practice, but no explicit documented deprecation policy, versioning scheme, or changelog is provided. missing for 10: a published API versioning scheme, a formal deprecation policy/timeline, changelog or release notes, and independent confirmation of stability guarantees.
- [claimed-docs] “result = client.parsing.parse( file_id=file.id, tier="agentic", version="latest", expand=["markdown"],)”
- [github] “Please migrate to the new packages: - **Python**: `pip install llama-cloud>=1.0`”
Extendnone0/10Evidence shows versioning for internal artifacts (processors, workflows, base models) but no documentation of API endpoint versioning (e.g., v1/v2 paths) or any deprecation/sunset policy for the REST API/SDK itself.
- [claimed-docs] “the thing you version and iterate on in Extend Studio, and the unit you evaluate and optimize”
- [claimed-docs] “A workflow has one editable draft and any number of immutable deployed versions.”
- [claimed-docs] “Base model versions are Extend's underlying model releases, selected with the baseVersion config field.”
- [claimed-docs] “Authenticate a request”
- [claimed-docs] “API Quickstart | Extend Documentation”
Automation depth — how much of the product can run unattendedAutomation depth
How much of the product can run unattended
ai-native userPerform bulk operations across many items at once
weight 2 · round drawnLlamaParse exposes a programmatic API/SDK plus webhooks so jobs can be automated at scale, and Classify lets rules be saved and referenced across multiple jobs, suggesting some bulk workflow support. However there is no explicit documentation of a batch/bulk submission endpoint, multi-file job queue, or bulk status/management UI for large sets of items. Missing for 10: explicit batch-submit API, bulk job dashboard/management, and independent evidence of large-scale bulk usage in production.
- [claimed-docs] “Instead of continuously polling for status updates, you can configure webhook endpoints to be notified immediately when jobs complete, fail,…”
- [claimed-docs] “Save reusable classification rules as a configuration and reference them by ID across multiple jobs.”
- [claimed-docs] “Get your first parse job running in under a minute—whether you prefer Python, TypeScript, Go, Java, the CLI, the REST API, or the Web UI.”
- [claimed-docs] “result = client.parsing.parse( file_id=file.id, tier="agentic", version="latest", expand=["markdown"],)”
Extend supports batch-like operations via multifile extraction (run over a collection of files with shared context), async processing endpoints for extract/classify/split/parse/edit, and evaluation set runs that score a processor against every item in a set — all pointing to bulk/batch capability across many documents. However, there's no explicit documentation of a bulk API endpoint for arbitrary large-scale item batches, rate limits/throughput guarantees, or dedicated bulk-operations tooling beyond these features. missing for 10: dedicated bulk-operation API/endpoint documentation, throughput/scale guarantees, independent hands-on evidence of large-scale bulk runs.
- [claimed-docs] “Multifile extraction lets you run a single extraction over a collection of files with a shared context.”
- [claimed-docs] “Every processing endpoint in Extend (extract, classify, split, parse, and edit) supports both a synchronous and asynchronous mode.”
- [claimed-docs] “An evaluation set run scores one processor version against every item in an evaluation set and returns aggregate metrics”
- [claimed-docs] “Async Processing | Extend Documentation”
ai-native userDefine rules that trigger actions automatically on events
weight 3 · round to ExtendLlamaParse supports webhooks to notify on job completion/failure and saved reusable classification rule configs, which are event-trigger-like building blocks, but there is no evidence of a general rules engine where users define arbitrary conditions/triggers that automatically fire actions across the platform. missing for 10: a documented rules/automation engine, conditional trigger definitions beyond job status, and evidence of chained automated actions (not just notification via webhook).
- [claimed-docs] “Instead of continuously polling for status updates, you can configure webhook endpoints to be notified immediately when jobs complete, fail,…”
- [claimed-docs] “Save reusable classification rules as a configuration and reference them by ID across multiple jobs.”
Extend supports webhooks to fire on processing events and has a Workflows system with versioning, which lets users react to events (e.g., file processed, extraction complete) with automated actions, but there's no documented general-purpose rule engine for defining custom trigger-condition-action logic across arbitrary events. Missing for 10: an explicit rules/conditions builder beyond simple webhook firing, and evidence of triggering on non-document events.
- [claimed-docs] “Webhook Configuration | Extend Documentation”
- [claimed-docs] “Workflows”
- [claimed-docs] “Every processing endpoint in Extend (extract, classify, split, parse, and edit) supports both a synchronous and asynchronous mode.”
- [claimed-docs] “A workflow has one editable draft and any number of immutable deployed versions.”
ai-native userSchedule recurring jobs or workflows
weight 2 · round drawnLlamaParsenone0/10LlamaParse's evidence covers parsing, extraction, classification, splitting, webhooks for job status, self-hosting, and MCP tool exposure, but nothing describes native scheduling of recurring jobs or workflows (e.g., cron-like triggers or recurring pipeline runs). Webhooks only cover async notification for one-off jobs, not recurrence.
- [claimed-docs] “Instead of continuously polling for status updates, you can configure webhook endpoints to be notified immediately when jobs complete, fail,…”
- [claimed-docs] “One API key, one SDK, and five composable products: Parse (agentic OCR), Extract (structured data), Classify, Split, and Index.”
Extendnone0/10Extend documents workflows, webhooks, and async processing, but no evidence of a scheduling mechanism (cron-like triggers, recurring runs) for workflows or jobs. Missing for 10: any mention of scheduled/recurring workflow triggers, cron configuration, or time-based automation.
- [claimed-docs] “Workflows”
- [claimed-docs] “Webhook Configuration | Extend Documentation”
- [claimed-docs] “Async Processing | Extend Documentation”
- [claimed-docs] “A workflow has one editable draft and any number of immutable deployed versions.”
ai-native userVersion, review, and roll back my automations
weight 1 · round to ExtendLlamaParsenone0/10No evidence of versioning, review workflows, or rollback for LlamaParse automations/configurations (e.g., parsing jobs, extraction schemas, classification rules). Only reusable configs via ID are mentioned (llamaparse-docs-39), but no version history, diff/review, or rollback capability is documented.
Extend documents concrete versioning for workflows ("one editable draft and any number of immutable deployed versions"), processors ("the thing you version and iterate on in Extend Studio"), and evaluation sets scoring specific processor versions — this covers version and review well. However, there is no explicit evidence of a rollback mechanism (reverting a deployed workflow/processor to a prior version) or of review workflows like diffs/approval gates beyond the Composer & Review Agent mention. missing for 10: explicit rollback/revert capability, diff/audit view between versions, independent confirmation of these versioning features in practice.
- [claimed-docs] “the thing you version and iterate on in Extend Studio, and the unit you evaluate and optimize”
- [claimed-docs] “A workflow has one editable draft and any number of immutable deployed versions.”
- [claimed-docs] “Evaluation sets let you test the accuracy of your document processors reliably and repeatedly.”
- [claimed-docs] “An evaluation set run scores one processor version against every item in an evaluation set and returns aggregate metrics”
- [claimed-docs] “you save that config once as a processor, give it a stable id, and reference it from then on”
- [claimed-docs] “Composer & Review Agent”
Deployment compliance — stories about deployment compliance in this arenaDeployment compliance
Stories about deployment compliance in this arena
Compliance
data engineerUploaded documents get zero-retention handling with SOC 2 and HIPAA options, so I can process contracts and medical records
weight 3 · round to ExtendFirst-party docs explicitly claim SOC 2 Type II completion and a HIPAA-compliant pipeline with BAA for Enterprise customers, plus BYOC/self-hosting and regional endpoints for data residency, directly addressing contract/medical-record processing needs. However, zero-retention handling specifically is not explicitly documented, HIPAA is gated behind 'Enterprise' tier, and there is no independent/third-party audit corroboration or hands-on confirmation of these compliance claims. Missing for 10: explicit zero-retention policy documentation, independent verification of SOC2/HIPAA claims, and clarity on whether zero-retention applies outside Enterprise tier.
- [claimed-docs] “Enterprise readiness: "LlamaParse Platform has completed a SOC 2 Type II audit. The report and subprocessor list are available through the T…”
- [claimed-docs] “Enterprise readiness, deployment & data residency: "managed SaaS, single-tenant, BYOC, self-hosting, and regional endpoints" — the platform …”
- [claimed-docs] “LlamaParse Platform has completed a SOC 2 Type II audit. The report and subprocessor list are available through the Trust Center. A HIPAA-co…”
- [claimed-docs] “Deploy LlamaCloud in your own cloud infrastructure with full self-hosting and BYOC support. Run LlamaParse, LlamaExtract, and the complete L…”
- [claimed-docs] “Identity and access. Use SSO and role-based access controls to manage access to your organization and projects.”
Extend's docs explicitly document SOC 2 Type II alignment, HIPAA (with BAA add-on) and GDPR compliance, plus configurable zero data retention (including for AI subprocessors) at the workspace level, directly matching the data-engineer's need to process contracts and medical records securely. Missing for 10: independent third-party audit confirmation or hands-on customer verification of ZDR/HIPAA in production use beyond vendor docs.
- [claimed-docs] “Compliance: "We maintain controls aligned with SOC 2 Type II requirements. Reports and security documentation are available in the Trust Cen…”
- [claimed-docs] “Data handling, Zero Data retention: "We maintain automatic data-retention policies that vary in default length by billing tier and can be co…”
- [claimed-docs] “Compliance | Extend Documentation”
- [claimed-docs] “Data Handling | Extend Documentation”
- [claimed-docs] “Zero data retention”
- [claimed-docs] “HIPAA Compliance and BAA add-on”
Deployment
data engineerRun the extraction stack in my own VPC or fully self-hosted when documents can't leave my infrastructure
weight 2 · round to LlamaParseDocs explicitly describe BYOC/self-hosted deployment of the full LlamaCloud platform (Parse, Extract) on Kubernetes via Helm charts across AWS/Azure/GCP, keeping data and infrastructure under customer control, plus enterprise deployment options including single-tenant and regional endpoints. Missing for 10: independent/hands-on verification of self-hosted deployment success and more detail on operational limitations of self-hosting vs managed SaaS.
- [claimed-docs] “LlamaCloud can be deployed entirely within your own infrastructure. Self-hosted deployments — also known as BYOC (Bring Your Own Cloud) — gi…”
- [claimed-docs] “Deploy LlamaCloud in your own cloud infrastructure with full self-hosting and BYOC support. Run LlamaParse, LlamaExtract, and the complete L…”
- [claimed-docs] “Enterprise readiness, deployment & data residency: "managed SaaS, single-tenant, BYOC, self-hosting, and regional endpoints" — the platform …”
- [claimed-docs] “LlamaCloud can be deployed entirely within your own infrastructure... give you the full capabilities of the LlamaCloud platform while keepin…”
Pricing page explicitly lists 'BYOC (your VPC) and Hybrid deployments' as an offering, indicating self-hosted/VPC deployment options exist, but there is no dedicated documentation detailing setup, architecture, or infrastructure requirements for such deployments, and no independent/hands-on confirmation. missing for 10: detailed self-hosting/VPC deployment docs, architecture guides, independent confirmation of BYOC actually working, clarity on which components run in customer VPC vs SaaS.
- [claimed-docs] “BYOC (your VPC) and Hybrid deployments”
Format coverage — stories about format coverage in this arenaFormat coverage
Stories about format coverage in this arena
Formats
developerOne API handles my whole document mix — PDF, DOCX, PPTX, XLSX, HTML, images, email — without per-format plumbing
weight 2 · round to LlamaParseDocs explicitly claim one unified API/SDK supporting 130+ file formats across categories (PDF, DOCX, PPTX, XLSX, HTML, images, email implied) via a single 'Parse' product with one API key/SDK, and examples show handling PDFs, Excel, scans, images, and forms without separate per-format code paths. Community feedback corroborates strong parsing on complex/mixed documents but focuses mainly on PDFs, and no independent evidence tests DOCX/PPTX/email specifically. Missing for 10: independent hands-on verification across the full non-PDF format set (DOCX, PPTX, XLSX, email) and explicit example code showing format-agnostic single-call usage.
- [claimed-docs] “LlamaParse supports 130+ file formats across four categories.”
- [claimed-docs] “One API key, one SDK, and five composable products: Parse (agentic OCR), Extract (structured data), Classify, Split, and Index.”
- [claimed-docs] “Parse is an agentic document parser built for LLM pipelines—layout-aware OCR that turns PDFs, scans, tables, and charts into clean markdown,…”
- [claimed-docs] “Parse Excel files and build a RAG app to answer questions about spreadsheet dat”
- [community] “I tried LlamaParse and was impressed by the result on a document with complex layout. None of the open-source parsers provided results even …”
Extend documents a single API surface (extract, classify, split, parse, edit) that ingests uploaded files and auto-converts Word/PowerPoint to PDF, and maintains a dedicated 'Supported File Types' doc, showing a unified pipeline for mixed document types. However, the evidence never explicitly confirms XLSX, HTML, or email format support, or shows a worked example spanning all these formats without extra plumbing. Missing for 10: explicit confirmation of XLSX/HTML/email support, and hands-on proof of one-call handling across all listed formats.
- [claimed-docs] “Supported File Types | Extend Documentation”
- [claimed-docs] “This endpoint accepts file contents and registers them as a File in Extend, which can be used for running workflows, creating evaluation set…”
- [claimed-docs] “If an uploaded file is detected as a Word or PowerPoint document, it will be automatically converted to a PDF.”
- [claimed-docs] “Every processing endpoint in Extend (extract, classify, split, parse, and edit) supports both a synchronous and asynchronous mode.”
Scale limits
data engineerThousand-page documents and multi-gigabyte files process reliably without timeouts or silent truncation
weight 2 · round to ExtendDocs mention async webhooks (avoiding polling/timeout issues) and a 'Cost Optimizer' for 'long mixed-complexity documents,' suggesting some design accommodation for large files, but there is no explicit claim, benchmark, or size/page limit documentation addressing thousand-page or multi-gigabyte files specifically, and no community evidence of large-file reliability. Missing for 10: documented size/page limits, explicit large-document benchmarks or case studies, and independent confirmation of no timeouts/truncation on very large files.
- [claimed-docs] “Instead of continuously polling for status updates, you can configure webhook endpoints to be notified immediately when jobs complete, fail,…”
- [claimed-docs] “pull every table out into pandas DataFrames with source-page provenance, and turn on Cost Optimizer to keep costs down on long mixed-complex…”
- [claimed-docs] “turn on Cost Optimizer to keep costs down on long mixed-complexity documents”
Extend explicitly advertises 2,000+ page support, async processing for all endpoints (to avoid timeouts on long jobs), and automatic Word/PPT-to-PDF conversion, plus a test environment guide—strong first-party evidence targeting large-document reliability. However, there's no explicit mention of multi-gigabyte file size limits, chunking/truncation safeguards for extremely large files, or independent/hands-on confirmation that thousand-page or multi-GB documents process without silent truncation. missing for 10: independent verification of large-file handling, explicit multi-GB file size limits/support, evidence of no silent truncation on very large inputs.
- [claimed-docs] “2,000+ page support”
- [claimed-docs] “Every processing endpoint in Extend (extract, classify, split, parse, and edit) supports both a synchronous and asynchronous mode.”
- [claimed-docs] “If an uploaded file is detected as a Word or PowerPoint document, it will be automatically converted to a PDF.”
- [claimed-docs] “Runs made with a test API key never touch production data and never fire production webhooks”
- [claimed-docs] “Async Processing | Extend Documentation”
Ocr multilingual — stories about ocr multilingual in this arenaOcr multilingual
Stories about ocr multilingual in this arena
Languages
developerNon-English documents — including CJK and right-to-left scripts — parse with the same fidelity as English
weight 2 · round drawnLlamaParsenone0/10No evidence in the pack addresses multilingual or CJK/RTL parsing fidelity; docs mention layout-aware OCR, format support, and general parsing quality but nothing about language-specific handling or non-English script accuracy.
Ocr
developerHandwritten fields and annotations are recognized and extracted, flagged with confidence when uncertain
weight 1 · round to ExtendDocs claim handwriting recognition as part of layout-aware OCR ('turn even the most complex layouts, tables, charts, handwriting, checkboxes, and images into clean markdown') and an enriched forms pass returns field values, checkbox states, and bounding boxes as structured JSON, but there is no documented confidence scoring or uncertainty flagging for handwritten fields/annotations specifically. Missing for 10: explicit confidence/uncertainty scores per extracted field, independent validation of handwriting accuracy, and any mention of flagging low-confidence handwritten extractions.
- [claimed-docs] “Turn even the most complex layouts, tables, charts, handwriting, checkboxes, and images into clean markdown”
- [claimed-docs] “Turn on the enriched forms pass to get each form page as structured JSON — fields with values, checkbox states, and bounding boxes — and rea…”
Extend explicitly advertises handwriting detection ('Detect tables, checkboxes, images, and handwriting') and 'Agentic OCR' as pricing features, plus a documented Confidence Scores mechanism that lets low-confidence extractions route to human review. However, there is no direct documentation showing handwritten field values are specifically flagged with lower confidence or examples of handwriting-specific extraction accuracy/behavior. Missing for 10: dedicated documentation or examples of handwriting-specific extraction accuracy, confidence flagging tied specifically to handwritten content, and independent/hands-on verification of handwriting OCR quality.
- [claimed-docs] “Detect tables, checkboxes, images, and handwriting”
- [claimed-docs] “Agentic OCR”
- [claimed-docs] “Confidence Scores | Extend Documentation”
- [claimed-docs] “Use them to trust high-confidence values automatically and route the rest to human review.”
- [claimed-docs] “plus per-field confidence scores and citations that point back to the exact spot on the page”
developerScanned and photographed documents OCR accurately — skewed pages, stamps, low quality scans included
weight 3 · round to ExtendLlamaParse's docs claim strong OCR for scans/photos, complex layouts, handwriting, checkboxes and images (llamaparse-docs-25, llamaparse-docs-12, llamaparse-docs-2), and community feedback confirms it handles complex layouts better than open-source parsers (llamaparse-comm-1). However, no evidence specifically addresses skewed-page correction, stamp detection, or degraded/low-quality scan handling, and community reports flag inconsistent table extraction and an independent benchmark showing lower faithfulness than claimed (llamaparse-comm-3, llamaparse-comm-6). missing for 10: explicit skew-correction handling, stamp/seal recognition, low-quality scan benchmarks, and independent hands-on validation of OCR accuracy on such edge cases.
- [claimed-docs] “Turn even the most complex layouts, tables, charts, handwriting, checkboxes, and images into clean markdown”
- [claimed-docs] “Turn PDFs, scans, or images into clean LLM-ready text”
- [claimed-docs] “Parse is an agentic document parser built for LLM pipelines—layout-aware OCR that turns PDFs, scans, tables, and charts into clean markdown,…”
- [community] “I tried LlamaParse and was impressed by the result on a document with complex layout. None of the open-source parsers provided results even …”
- [community] “I found the LlamaParse interface a bit confusing. It can extract numbers in tables, but the output isn't provided in tabular format—access t…”
- [community] “I ran the baseline PyPDF+Naive RAG benchmark myself and got a faithfulness score of 0.980, actually higher than the 0.667 the team reported …”
Extend markets itself as handling 'your hardest documents with unmatched accuracy' with agentic OCR, handwriting/table/checkbox detection, and confidence-based routing to human review, and a year-long HN user reports being happy with extraction accuracy. However no evidence specifically addresses skewed pages, stamps, or low-quality scan handling — these document conditions are never explicitly mentioned in docs or community feedback. Missing for 10: explicit documentation or benchmarks on skew correction, stamp/seal recognition, and degraded scan quality performance, plus independent hands-on testing on such documents.
- [claimed-docs] “Parse, extract, and split your hardest documents with unmatched accuracy. Ship reliable document agents in minutes, not months.”
- [claimed-docs] “Detect tables, checkboxes, images, and handwriting”
- [claimed-docs] “Agentic OCR”
- [claimed-docs] “Use them to trust high-confidence values automatically and route the rest to human review.”
- [claimed-docs] “plus per-field confidence scores and citations that point back to the exact spot on the page”
- [community] “We've been using Extend for over a year and have been super happy with the product and accuracy of the data extraction.”
Openness — open source, data portability, and self-hosting storiesOpenness
Open source, data portability, and self-hosting stories
ai-native userDo everything through the API that I can do in the UI
weight 2 · round drawnDocs show every core capability (Parse, Extract, Classify, Split, Index) is available via Python/TS/Go/Java SDKs, CLI and REST API alongside the Web UI, and webhooks let API users get async job notifications like the UI dashboard. However, admin/enterprise features (SSO, RBAC, project/org management, Trust Center reports) are described only in UI/dashboard terms with no evidence of equivalent API endpoints, and no OpenAPI spec was discoverable (probe 404s) to confirm full parity. Missing for 10: documented API endpoints for org/SSO/RBAC management, a public OpenAPI/swagger spec, and independent confirmation that all UI-configured settings (e.g., extraction schema builder) are fully API-driven.
- [claimed-docs] “Get your first parse job running in under a minute—whether you prefer Python, TypeScript, Go, Java, the CLI, the REST API, or the Web UI.”
- [claimed-docs] “Instead of continuously polling for status updates, you can configure webhook endpoints to be notified immediately when jobs complete, fail,…”
- [claimed-docs] “Identity and access. Use SSO and role-based access controls to manage access to your organization and projects.”
- [claimed-docs] “Just define your Extraction Configuration (schema and settings), drag and drop any supported document into LlamaParse and extract data from …”
- [claimed-docs] “One API key, one SDK, and five composable products: Parse (agentic OCR), Extract (structured data), Classify, Split, and Index.”
- [probe] “PROBE openapi: all candidate paths 404 (https://developers.llamaindex.ai/openapi.json, https://developers.llamaindex.ai/swagger.json, https:…”
Extend documents a broad API/SDK/CLI/MCP surface covering the same core objects seen in its UI (processors, workflows with versioning, evaluation sets, extraction/classification/splitting, file upload) suggesting close UI-API parity, but there is no explicit statement of full parity and UI-centric tools like Studio's interactive testing and the Composer & Review Agent (human-in-the-loop) are not shown to have a documented equivalent API path. missing for 10: explicit parity claim/documentation, API equivalents for Studio interactive iteration and Composer/Review Agent workflows, independent confirmation of full coverage.
- [claimed-docs] “CLI | Extend Documentation”
- [claimed-docs] “Using Extend via MCP | Extend Documentation”
- [claimed-docs] “SDKs | Extend Documentation”
- [claimed-docs] “API Quickstart | Extend Documentation”
- [claimed-docs] “Agent Quickstart | Extend Documentation”
- [claimed-docs] “you save that config once as a processor, give it a stable id, and reference it from then on”
- [claimed-docs] “the thing you version and iterate on in Extend Studio, and the unit you evaluate and optimize”
- [claimed-docs] “Evaluation sets let you test the accuracy of your document processors reliably and repeatedly.”
- [claimed-docs] “A workflow has one editable draft and any number of immutable deployed versions.”
- [claimed-docs] “Composer & Review Agent”
ai-native userExport all of my data in open formats and leave
weight 3 · round to LlamaParseLlamaParse's core outputs (markdown, text, JSON) are open, portable formats by design, and BYOC/self-hosting options mean users can run the entire platform on their own infrastructure without vendor lock-in. However, there's no documented explicit 'export all account data' feature or bulk data-portability tool beyond the per-job parse outputs. missing for 10: dedicated account-wide data export/deletion tooling, migration guides for leaving the platform, independent verification of format completeness.
- [claimed-docs] “Parse is an agentic document parser built for LLM pipelines—layout-aware OCR that turns PDFs, scans, tables, and charts into clean markdown,…”
- [claimed-docs] “Turn even the most complex layouts, tables, charts, handwriting, checkboxes, and images into clean markdown”
- [claimed-docs] “LlamaCloud can be deployed entirely within your own infrastructure. Self-hosted deployments — also known as BYOC (Bring Your Own Cloud) — gi…”
- [claimed-docs] “LlamaCloud can be deployed entirely within your own infrastructure... give you the full capabilities of the LlamaCloud platform while keepin…”
- [claimed-docs] “Deploy LlamaCloud in your own cloud infrastructure with full self-hosting and BYOC support. Run LlamaParse, LlamaExtract, and the complete L…”
Extendnone0/10Extend is a SaaS document-processing platform holding workflows, processors, evaluation sets and extracted data, so data-portability/export is a fair question, but the evidence pack contains no mention of a bulk data-export feature, open-format export of processed data/configs, or account-closure data dump — only retention/ZDR policies which describe deletion, not export.
- [claimed-docs] “Data handling, Zero Data retention: "We maintain automatic data-retention policies that vary in default length by billing tier and can be co…”
- [claimed-docs] “Data Handling | Extend Documentation”
ai-native userSelf-host the core product
weight 3 · round to LlamaParseDocs explicitly describe self-hosting/BYOC deployments giving full LlamaCloud capabilities within customer infrastructure, including Kubernetes/Helm deployment on AWS/Azure/GCP, plus enterprise deployment options (managed SaaS, single-tenant, BYOC, self-hosting, regional endpoints). Missing for 10: independent/hands-on confirmation of a self-hosted deployment actually running (no community reports), and details on licensing/open-source availability of the self-hosted code vs. it being a gated enterprise offering.
- [claimed-docs] “LlamaCloud can be deployed entirely within your own infrastructure. Self-hosted deployments — also known as BYOC (Bring Your Own Cloud) — gi…”
- [claimed-docs] “LlamaCloud can be deployed entirely within your own infrastructure... give you the full capabilities of the LlamaCloud platform while keepin…”
- [claimed-docs] “Deploy LlamaCloud in your own cloud infrastructure with full self-hosting and BYOC support. Run LlamaParse, LlamaExtract, and the complete L…”
- [claimed-docs] “Enterprise readiness, deployment & data residency: "managed SaaS, single-tenant, BYOC, self-hosting, and regional endpoints" — the platform …”
Extend is a closed SaaS platform; the only evidence bearing on self-hosting is a pricing-page bullet mentioning 'BYOC (your VPC) and Hybrid deployments' as an enterprise add-on, with no documentation of installation, source availability, or a true self-managed deployment path. Missing for 10: open-source or downloadable core product, self-hosting setup docs, infrastructure requirements, and independent confirmation that customers can run it fully outside Extend's cloud.
- [claimed-docs] “BYOC (your VPC) and Hybrid deployments”
Parse accuracy — stories about parse accuracy in this arenaParse accuracy
Stories about parse accuracy in this arena
Evals
ml engineerThe vendor publishes reproducible accuracy benchmarks and I can run my own evals before committing
weight 1 · round to ExtendLlamaParsenone0/10The evidence pack contains no vendor-published reproducible accuracy benchmarks (docs describe features, formats, and getting-started SDK usage, but no benchmark methodology, dataset, or published accuracy numbers) and no first-party guidance on running independent evals. A community report (comm-6) even claims a self-run baseline outperformed a previously reported LlamaParse pipeline number, but this is a single community claim referencing an external benchmark, not a vendor-published reproducible benchmark suite, so it cannot establish 'disputed' status. missing for 10: vendor-published benchmark methodology/dataset, published accuracy metrics, eval harness or reference scripts for ML engineers to reproduce results, any first-party corroboration of benchmark claims.
- [community] “I ran the baseline PyPDF+Naive RAG benchmark myself and got a faithfulness score of 0.980, actually higher than the 0.667 the team reported …”
Extend documents a full self-serve evaluation framework (evaluation sets, processors, versioning, aggregate metrics from evaluation-set runs) that lets an ML engineer run their own accuracy evals before committing, and community testimony corroborates strong extraction accuracy in practice. However there is no evidence of the vendor publishing its own reproducible benchmark numbers/datasets or accuracy reports for independent verification. Missing for 10: vendor-published benchmark results/methodology, third-party benchmark validation, public accuracy leaderboards or comparable datasets.
- [claimed-docs] “Evaluation sets let you test the accuracy of your document processors reliably and repeatedly.”
- [claimed-docs] “An evaluation set run scores one processor version against every item in an evaluation set and returns aggregate metrics”
- [claimed-docs] “you save that config once as a processor, give it a stable id, and reference it from then on”
- [claimed-docs] “the thing you version and iterate on in Extend Studio, and the unit you evaluate and optimize”
- [community] “We've been using Extend for over a year and have been super happy with the product and accuracy of the data extraction.”
Figures
ml engineerFigures and charts are extracted or described (VLM summaries, image crops) with positions traceable back to the source page
weight 2 · round drawnDocs confirm layout-aware parsing of charts/images into markdown and table extraction with source-page provenance (docs-15, docs-22, docs-25), and a community request for page numbers alongside markdown (comm-7) suggests this is not fully solved out-of-the-box. There's no explicit evidence of VLM-generated figure/chart summaries or image crop extraction with bounding boxes tied back to page position (only forms/checkbox bounding boxes are documented). missing for 10: explicit VLM-based figure/chart description or summarization feature, documented image-crop extraction with bounding-box/page coordinates for charts/figures specifically, and independent hands-on confirmation of this traceability.
- [claimed-docs] “pull every table out into pandas DataFrames with source-page provenance, and turn on Cost Optimizer to keep costs down on long mixed-complex…”
- [claimed-docs] “Turn on the enriched forms pass to get each form page as structured JSON — fields with values, checkbox states, and bounding boxes — and rea…”
- [claimed-docs] “Turn even the most complex layouts, tables, charts, handwriting, checkboxes, and images into clean markdown”
- [community] “LlamaParse looks nice. Is there a way to return page numbers also with the markdown? This is important for our use case.”
Extend documents citations/provenance pointing extracted field values back to exact page locations and detects tables/images/handwriting, but evidence never explicitly describes VLM-generated summaries of figures/charts or image crop extraction with bounding-box positions for visual elements specifically — citations are framed around extracted field values, not figure/chart description. missing for 10: explicit VLM-based figure/chart summarization, image crop extraction as an output artifact, and confirmation that citations/positions apply to figures/charts (not just text fields).
- [claimed-docs] “Detect tables, checkboxes, images, and handwriting”
- [claimed-docs] “Citations”
- [claimed-docs] “plus per-field confidence scores and citations that point back to the exact spot on the page”
- [claimed-docs] “Multifile extraction lets you run a single extraction over a collection of files with a shared context.”
- [claimed-docs] “Confidence Scores | Extend Documentation”
Layout
developerThe API parses complex real-world PDFs — multi-column layouts, headers, footers, footnotes — into clean, correctly ordered content
weight 3 · round to ExtendLlamaParsedisputedcontradicted5/10Docs claim agentic, layout-aware OCR that turns complex layouts, tables, charts, and handwriting into clean markdown (llamaparse-docs-2, llamaparse-docs-25), and one HN commenter found it markedly better than open-source parsers on complex layouts (llamaparse-comm-1). But other hands-on users report concrete accuracy gaps — table data extracted but not returned in usable tabular format (llamaparse-comm-3) and a benchmark run showing LlamaParse-based pipeline faithfulness (0.667) actually lower than a naive PyPDF baseline (0.980) (llamaparse-comm-6) — directly contradicting the 'clean, correctly ordered' claim for some real-world documents. missing for 10: independent benchmark specifically on multi-column/headers/footers/footnotes ordering, resolution of the conflicting community reports, and reproducible accuracy metrics.
- [claimed-docs] “Parse is an agentic document parser built for LLM pipelines—layout-aware OCR that turns PDFs, scans, tables, and charts into clean markdown,…”
- [claimed-docs] “Turn even the most complex layouts, tables, charts, handwriting, checkboxes, and images into clean markdown”
- [community] “I tried LlamaParse and was impressed by the result on a document with complex layout. None of the open-source parsers provided results even …”
- [community] “I found the LlamaParse interface a bit confusing. It can extract numbers in tables, but the output isn't provided in tabular format—access t…”
- [community] “I ran the baseline PyPDF+Naive RAG benchmark myself and got a faithfulness score of 0.980, actually higher than the 0.667 the team reported …”
Extend markets parsing as high-accuracy for 'hardest documents' with features like agentic OCR, table/checkbox/handwriting detection, and citations pointing to exact page locations, and one community user praises extraction accuracy over a year of use. However, no evidence specifically addresses multi-column layouts, headers/footers, or footnote handling/ordering. Missing for 10: explicit documentation or examples showing correct handling of multi-column reading order, header/footer exclusion, and footnote placement; independent benchmark or hands-on test of these specific layout challenges.
- [claimed-docs] “Parse, extract, and split your hardest documents with unmatched accuracy. Ship reliable document agents in minutes, not months.”
- [claimed-docs] “Detect tables, checkboxes, images, and handwriting”
- [claimed-docs] “Agentic OCR”
- [claimed-docs] “plus per-field confidence scores and citations that point back to the exact spot on the page”
- [community] “We've been using Extend for over a year and have been super happy with the product and accuracy of the data extraction.”
ml engineerParsed output preserves document hierarchy — headings, sections, reading order — so downstream LLMs see structure, not soup
weight 2 · round to LlamaParseDocs describe LlamaParse as 'layout-aware OCR' that converts complex layouts, tables, charts, and headings into 'clean markdown,' implying structural/hierarchy preservation, and a community reviewer praised its handling of complex layouts. However, another hands-on user reported that table data wasn't returned in proper tabular structure and was only accessible via Q&A, raising doubts about how faithfully structural elements are preserved, and no evidence explicitly confirms heading-level/section hierarchy or reading-order fidelity beyond general layout claims. Missing for 10: explicit documentation/examples of heading-level and section-order preservation, and independent benchmarking confirming structural fidelity beyond tables.
- [claimed-docs] “Parse is an agentic document parser built for LLM pipelines—layout-aware OCR that turns PDFs, scans, tables, and charts into clean markdown,…”
- [claimed-docs] “Turn even the most complex layouts, tables, charts, handwriting, checkboxes, and images into clean markdown”
- [claimed-docs] “Guide parsing output with custom prompts. Extract specific data from a McDonald's receipt by instructing the parser to focus only on prices …”
- [community] “I tried LlamaParse and was impressed by the result on a document with complex layout. None of the open-source parsers provided results even …”
- [community] “I found the LlamaParse interface a bit confusing. It can extract numbers in tables, but the output isn't provided in tabular format—access t…”
Extendnone0/10The evidence pack confirms Extend parses documents and extracts structured data (chunks, metadata) but contains no specific claim or demonstration that parsed output preserves heading hierarchy, section structure, or reading order for downstream LLM consumption.
- [claimed-docs] “Parse runs now include a **`metadata`** object on `output`, alongside `chunks`”
- [claimed-docs] “Detect tables, checkboxes, images, and handwriting”
Privacy posture — data-handling and privacy storiesPrivacy posture
Data-handling and privacy stories
ai-native userChoose where my data is stored (region/residency)
weight 2 · round to LlamaParseDocs state the platform offers 'managed SaaS, single-tenant, BYOC, self-hosting, and regional endpoints' for deployment and data residency, and self-hosting/BYOC explicitly lets customers keep data and infrastructure under their own control (including choice of cloud/region via Kubernetes on AWS/Azure/GCP). However, there is no detail on which specific regions are available for the managed SaaS regional-endpoint option, no UI/API mechanism shown for selecting a region, and no independent confirmation. Missing for 10: concrete list of supported regions/endpoints, documented API/UI control to select a region, and third-party or hands-on confirmation of data residency behavior.
- [claimed-docs] “Enterprise readiness, deployment & data residency: "managed SaaS, single-tenant, BYOC, self-hosting, and regional endpoints" — the platform …”
- [claimed-docs] “LlamaCloud can be deployed entirely within your own infrastructure. Self-hosted deployments — also known as BYOC (Bring Your Own Cloud) — gi…”
- [claimed-docs] “Deploy LlamaCloud in your own cloud infrastructure with full self-hosting and BYOC support. Run LlamaParse, LlamaExtract, and the complete L…”
Extendnone0/10Evidence covers compliance (SOC2, HIPAA, GDPR), data retention/ZDR, and BYOC/Hybrid deployment options, but nowhere mentions region selection or data residency choice for storage location. missing for 10: explicit region/residency selection controls, documentation naming specific data center regions, evidence of a residency configuration option.
- [claimed-docs] “Compliance: "We maintain controls aligned with SOC 2 Type II requirements. Reports and security documentation are available in the Trust Cen…”
- [claimed-docs] “Data handling, Zero Data retention: "We maintain automatic data-retention policies that vary in default length by billing tier and can be co…”
- [claimed-docs] “BYOC (your VPC) and Hybrid deployments”
ai-native userPrevent my data from being used to train AI models
weight 3 · round to ExtendLlamaParsenone0/10The evidence covers enterprise features like SOC2/HIPAA compliance, SSO/RBAC, and self-hosting/BYOC options, but nowhere states an explicit policy or toggle for preventing customer data from being used to train AI models. Self-hosting implies data stays on the customer's infra, but that is not the same as a documented no-training-use guarantee.
- [claimed-docs] “LlamaCloud can be deployed entirely within your own infrastructure. Self-hosted deployments — also known as BYOC (Bring Your Own Cloud) — gi…”
- [claimed-docs] “Identity and access. Use SSO and role-based access controls to manage access to your organization and projects.”
- [claimed-docs] “Enterprise readiness: "LlamaParse Platform has completed a SOC 2 Type II audit. The report and subprocessor list are available through the T…”
- [claimed-docs] “Enterprise readiness, deployment & data residency: "managed SaaS, single-tenant, BYOC, self-hosting, and regional endpoints" — the platform …”
Extend documents zero data retention (ZDR) options including for AI subprocessors, configurable at workspace level, plus SOC2/HIPAA/GDPR compliance controls, which addresses training-data concerns for enterprise customers. However, there's no explicit statement that customer data is never used to train Extend's own models by default, nor independent verification of this policy. Missing for 10: explicit default no-training-on-customer-data policy statement, independent audit/confirmation, and clarity on whether ZDR is opt-in/paid-tier only.
- [claimed-docs] “Zero data retention”
- [claimed-docs] “Data handling, Zero Data retention: "We maintain automatic data-retention policies that vary in default length by billing tier and can be co…”
- [claimed-docs] “Compliance | Extend Documentation”
- [claimed-docs] “Compliance: "We maintain controls aligned with SOC 2 Type II requirements. Reports and security documentation are available in the Trust Cen…”
ai-native userControl data retention and deletion
weight 2 · round to ExtendLlamaParse offers BYOC/self-hosting so customers can keep all data under their own infrastructure control, and cites SOC2 Type II/HIPAA compliance, which implies some data governance posture, but no evidence describes an explicit data retention policy, configurable retention windows, or a deletion API/mechanism for the managed SaaS offering. missing for 10: explicit retention period documentation, a data deletion API/endpoint, and confirmation of deletion behavior for managed (non-BYOC) deployments.
- [claimed-docs] “LlamaCloud can be deployed entirely within your own infrastructure. Self-hosted deployments — also known as BYOC (Bring Your Own Cloud) — gi…”
- [claimed-docs] “LlamaCloud can be deployed entirely within your own infrastructure... give you the full capabilities of the LlamaCloud platform while keepin…”
- [claimed-docs] “Enterprise readiness: "LlamaParse Platform has completed a SOC 2 Type II audit. The report and subprocessor list are available through the T…”
- [claimed-docs] “Enterprise readiness, deployment & data residency: "managed SaaS, single-tenant, BYOC, self-hosting, and regional endpoints" — the platform …”
- [claimed-docs] “Deploy LlamaCloud in your own cloud infrastructure with full self-hosting and BYOC support. Run LlamaParse, LlamaExtract, and the complete L…”
Extend's official docs explicitly describe configurable automatic data-retention policies and zero data retention (ZDR) options, settable at the workspace level, plus GDPR-related compliance documentation; the pricing page also advertises 'Zero data retention' as a feature tier. This directly addresses retention and deletion control for AI-native users. Missing for 10: no independent/hands-on confirmation that deletion requests are actually honored, and no explicit self-service deletion API/UI documented beyond retention policy configuration.
- [claimed-docs] “Data handling, Zero Data retention: "We maintain automatic data-retention policies that vary in default length by billing tier and can be co…”
- [claimed-docs] “Zero data retention”
- [claimed-docs] “Compliance | Extend Documentation”
- [claimed-docs] “Data Handling | Extend Documentation”
ai-native userOpt out of telemetry and usage tracking
weight 2 · round drawnLlamaParsenone0/10No evidence pack content addresses telemetry/usage-tracking opt-out settings for LlamaParse; only self-hosting/BYOC and enterprise compliance features (SOC2, HIPAA, SSO) are documented, none of which explicitly describe a telemetry opt-out control.
Extendnone0/10Extend documents data retention/ZDR and compliance policies, but there is no mention of telemetry or usage-tracking opt-out settings for the product itself (e.g., CLI/SDK analytics telemetry). missing for 10: any documentation of telemetry collection or an opt-out/disable-tracking mechanism.
Rag chunking — stories about rag chunking in this arenaRag chunking
Stories about rag chunking in this arena
Chunking
ai-native userOutput comes pre-chunked for RAG — semantic boundaries, metadata, embedding-ready segments — not a wall of text
weight 3 · round to ExtendLlamaParse's Index product offers a hosted vector-search/embedding pipeline (docs-19, docs-30, docs-31) and structured JSON/table extraction with page-level provenance (docs-15, docs-22), suggesting some RAG-ready segmentation, but the evidence never documents explicit semantic chunk boundaries or embedding-ready segment metadata as a first-class output of Parse itself, and a community question about even basic page-number metadata in markdown output (llamaparse-comm-7) went unanswered, indicating gaps in chunk-level metadata support. missing for 10: explicit chunking/boundary strategy documentation, embedding-ready segment schema, confirmed page/metadata tagging in default output.
- [claimed-docs] “Build a hosted vector search pipeline for RAG”
- [claimed-docs] “If you don’t want to set up and host a vector database, we offer a full-managed option in which we host the vector database for you. Alterna…”
- [claimed-docs] “We support a variety of embedding models that you can choose from”
- [claimed-docs] “pull every table out into pandas DataFrames with source-page provenance, and turn on Cost Optimizer to keep costs down on long mixed-complex…”
- [claimed-docs] “Turn on the enriched forms pass to get each form page as structured JSON — fields with values, checkbox states, and bounding boxes — and rea…”
- [community] “LlamaParse looks nice. Is there a way to return page numbers also with the markdown? This is important for our use case.”
Extend's Parse API outputs both `chunks` and a `metadata` object, and there's a dedicated Splitting feature for breaking documents into semantic segments, which aligns with the RAG-chunking need. However, there's no explicit documentation framing this as embedding-ready or optimized for RAG pipelines, nor independent corroboration of chunk quality for downstream embedding use. Missing for 10: explicit embedding-ready format specs, RAG-specific use-case documentation, and independent/hands-on validation of chunk semantic quality.
- [claimed-docs] “Parse runs now include a **`metadata`** object on `output`, alongside `chunks`”
- [claimed-docs] “Splitting Overview | Extend Documentation”
- [claimed-docs] “plus per-field confidence scores and citations that point back to the exact spot on the page”
Output
ai-native userI get clean markdown/JSON designed for LLM consumption, with noise like repeated headers and page furniture stripped
weight 2 · round to LlamaParseDocs strongly market LlamaParse as an agentic, layout-aware parser producing 'clean markdown, text, or JSON' from complex documents (docs-2, docs-12, docs-25) with markdown expansion options (docs-11), and early community feedback praised its handling of complex layouts versus open-source parsers (comm-1, comm-2). However, no evidence specifically addresses stripping of repeated headers/page furniture, and some hands-on community feedback flags real output-quality gaps (tables not returned in tabular form, disputed benchmark numbers) that temper the 'clean for LLM consumption' claim (comm-3, comm-6). Missing for 10: explicit documentation/example of header/footer/page-furniture deduplication, and independent verification of noise-free markdown output.
- [claimed-docs] “Parse is an agentic document parser built for LLM pipelines—layout-aware OCR that turns PDFs, scans, tables, and charts into clean markdown,…”
- [claimed-docs] “Turn PDFs, scans, or images into clean LLM-ready text”
- [claimed-docs] “Turn even the most complex layouts, tables, charts, handwriting, checkboxes, and images into clean markdown”
- [claimed-docs] “result = client.parsing.parse( file_id=file.id, tier="agentic", version="latest", expand=["markdown"],)”
- [community] “I tried LlamaParse and was impressed by the result on a document with complex layout. None of the open-source parsers provided results even …”
- [community] “I found the LlamaParse interface a bit confusing. It can extract numbers in tables, but the output isn't provided in tabular format—access t…”
- [community] “I ran the baseline PyPDF+Naive RAG benchmark myself and got a faithfulness score of 0.980, actually higher than the 0.667 the team reported …”
Extend's Parse feature outputs structured chunks plus a metadata object (extend-docs-3, extend-docs-9), suggesting some design toward LLM/RAG-ready output, but there is no explicit documentation claiming markdown/JSON output has repeated headers or page furniture stripped for clean LLM consumption. missing for 10: explicit claim/documentation of noise removal (headers, footers, page furniture) in parse output, and independent verification of clean output quality.
- [claimed-docs] “Parse runs now include a **`metadata`** object on `output`, alongside `chunks`”
- [claimed-docs] “Splitting Overview | Extend Documentation”
- [claimed-docs] “Every processing endpoint in Extend (extract, classify, split, parse, and edit) supports both a synchronous and asynchronous mode.”
Scale async — stories about scale async in this arenaScale async
Stories about scale async in this arena
Async
developerLong parses run as async jobs with status polling and completion webhooks, so my pipeline never blocks
weight 3 · round to LlamaParseDocs explicitly describe async job parsing with status polling replaceable by webhooks ('Instead of continuously polling for status updates, you can configure webhook endpoints to be notified immediately when jobs complete, fail, or reach other states') and the getting-started/parse job flow confirms async job submission. Missing for 10: independent/hands-on confirmation of webhook reliability at scale and detail on retry/backoff semantics for polling.
- [claimed-docs] “Instead of continuously polling for status updates, you can configure webhook endpoints to be notified immediately when jobs complete, fail,…”
- [claimed-docs] “result = client.parsing.parse( file_id=file.id, tier="agentic", version="latest", expand=["markdown"],)”
- [claimed-docs] “Get your first parse job running in under a minute—whether you prefer Python, TypeScript, Go, Java, the CLI, the REST API, or the Web UI.”
Extend's docs explicitly state every processing endpoint including parse supports both sync and async modes, and separate docs cover webhook configuration for completion notifications, matching the async-job + webhook pipeline pattern. Missing for 10: explicit mention of a status-polling endpoint/job-id retrieval flow and independent/hands-on confirmation that async+webhook flow works reliably in production.
- [claimed-docs] “Every processing endpoint in Extend (extract, classify, split, parse, and edit) supports both a synchronous and asynchronous mode.”
- [claimed-docs] “Async Processing | Extend Documentation”
- [claimed-docs] “Webhook Configuration | Extend Documentation”
Latency
developerA fast synchronous mode returns results in seconds for interactive apps, with latency documented per mode
weight 2 · round to ExtendLlamaParsenone0/10The evidence pack shows async-style job submission with polling or webhooks (llamaparse-docs-6) and tiered parsing modes (e.g., 'agentic' tier in llamaparse-docs-11), but there is no documentation of a fast synchronous mode returning results in seconds, nor any per-mode latency figures.
Docs confirm every processing endpoint supports both sync and async modes (extend-docs-37) and pricing page lists a 'Fast mode' option (extend-docs-25), implying a fast synchronous path exists, but no evidence quantifies actual latency (e.g., 'returns in X seconds') for either mode. Missing for 10: documented latency benchmarks per mode, explicit performance numbers for sync/fast mode, independent confirmation of response times.
- [claimed-docs] “Every processing endpoint in Extend (extract, classify, split, parse, and edit) supports both a synchronous and asynchronous mode.”
- [claimed-docs] “Async Processing | Extend Documentation”
- [claimed-docs] “Fast mode”
Scale
data engineerI push high-volume batches — millions of pages — with documented rate limits and predictable throughput
weight 2 · round drawnLlamaParse documents async job status via webhooks (avoiding polling) and offers self-hosted/BYOC deployment for scaling infrastructure, which supports high-volume async workflows, but there is no documented rate-limit table, batch-submission API, or throughput/SLA numbers for millions-of-pages workloads. Missing for 10: explicit rate-limit documentation, batch endpoint semantics, throughput benchmarks or SLAs for large-scale ingestion.
- [claimed-docs] “Instead of continuously polling for status updates, you can configure webhook endpoints to be notified immediately when jobs complete, fail,…”
- [claimed-docs] “LlamaCloud can be deployed entirely within your own infrastructure. Self-hosted deployments — also known as BYOC (Bring Your Own Cloud) — gi…”
- [claimed-docs] “LlamaCloud can be deployed entirely within your own infrastructure... give you the full capabilities of the LlamaCloud platform while keepin…”
- [claimed-docs] “Enterprise readiness, deployment & data residency: "managed SaaS, single-tenant, BYOC, self-hosting, and regional endpoints" — the platform …”
Extend documents async processing for all endpoints and a credit-based pricing model, which supports high-volume batch workflows, but there is no evidence of documented rate limits, throughput benchmarks, or guidance for processing millions of pages at scale. Missing for 10: explicit rate-limit documentation, throughput/SLA numbers, and batch-scale case studies or guarantees.
- [claimed-docs] “Async Processing | Extend Documentation”
- [claimed-docs] “Every processing endpoint in Extend (extract, classify, split, parse, and edit) supports both a synchronous and asynchronous mode.”
- [claimed-docs] “How Credits Work | Extend Documentation”
- [claimed-docs] “2,000+ page support”
Sdk dx — stories about sdk dx in this arenaSdk dx
Stories about sdk dx in this arena
Playground
developerI drag a document into a web playground and see parse/extract results before writing any code
weight 1 · round to LlamaParseDocs confirm a Web UI option alongside SDKs/CLI (llamaparse-docs-1) and explicitly describe dragging and dropping a document into LlamaParse to extract data (llamaparse-docs-14), and community evidence confirms an actual interface exists that users interacted with directly (llamaparse-comm-3, llamaparse-comm-4). However, evidence doesn't show a dedicated 'playground' branding, screenshots, or hands-on positive confirmation of the full parse-before-code workflow, and one community report calls the interface itself confusing. Missing for 10: explicit playground UI walkthrough/screenshots, independent hands-on confirmation of drag-and-drop parse+extract results without any code, and resolution of the UI confusion complaint.
- [claimed-docs] “Get your first parse job running in under a minute—whether you prefer Python, TypeScript, Go, Java, the CLI, the REST API, or the Web UI.”
- [claimed-docs] “Just define your Extraction Configuration (schema and settings), drag and drop any supported document into LlamaParse and extract data from …”
- [community] “I found the LlamaParse interface a bit confusing. It can extract numbers in tables, but the output isn't provided in tabular format—access t…”
- [community] “Signing up for LlamaParse was rough: email signup endlessly redirected, click link in email, asked to sign up again with email, click link a…”
Extend references a web-based "Studio" for iterating on processors (extend-docs-18, extend-docs-40) which implies a no-code interface for testing parse/extract, but no evidence describes a drag-and-drop upload flow or explicitly shows results appearing before any code is written. Missing for 10: explicit description or screenshot of drag-and-drop file upload in Studio, confirmation that parse/extract results appear instantly without any coding, and independent/hands-on confirmation of this playground UX.
- [claimed-docs] “Studio and Evals”
- [claimed-docs] “the thing you version and iterate on in Extend Studio, and the unit you evaluate and optimize”
- [claimed-docs] “you save that config once as a processor, give it a stable id, and reference it from then on”
Sdks
developerOfficial typed SDKs for Python and TypeScript cover the full API — parse, extract, jobs — with sensible defaults
weight 3 · round drawnDocs confirm official Python and TypeScript SDKs ("whether you prefer Python, TypeScript, Go, Java, the CLI, the REST API") and show a Python code sample for parsing with defaults (tier, version, expand), plus a webhook/job-status system, and GitHub shows an actively maintained llama-cloud Python package. However, there's no TypeScript code sample, no explicit documentation of a typed Extract or Jobs SDK surface, and no independent corroboration of SDK typing quality or defaults across both languages. Missing for 10: TypeScript SDK code examples, explicit typed Extract/Jobs API coverage in both SDKs, and third-party validation of SDK completeness.
- [claimed-docs] “Get your first parse job running in under a minute—whether you prefer Python, TypeScript, Go, Java, the CLI, the REST API, or the Web UI.”
- [claimed-docs] “result = client.parsing.parse( file_id=file.id, tier="agentic", version="latest", expand=["markdown"],)”
- [claimed-docs] “Instead of continuously polling for status updates, you can configure webhook endpoints to be notified immediately when jobs complete, fail,…”
- [github] “Please migrate to the new packages: - **Python**: `pip install llama-cloud>=1.0`”
- [claimed-docs] “LlamaExtract provides a simple API for extracting structured data from unstructured documents like PDFs, text files, and images.”
Extend has a dedicated SDKs docs page and a Python package (pip install extend-ai) referenced, plus full API coverage across parse/extract/split/classify/jobs implied by endpoint docs, suggesting SDKs wrap the full API. However, there's no explicit confirmation of a TypeScript SDK, no code samples showing typed usage or sensible defaults, and no independent/community corroboration of SDK quality or completeness. Missing for 10: explicit TypeScript SDK evidence, typed code examples demonstrating parse/extract/jobs coverage, and independent developer confirmation of SDK ergonomics/defaults.
- [claimed-docs] “SDKs | Extend Documentation”
- [claimed-docs] “pip install extend-ai”
- [claimed-docs] “API Quickstart | Extend Documentation”
- [claimed-docs] “Every processing endpoint in Extend (extract, classify, split, parse, and edit) supports both a synchronous and asynchronous mode.”
Structured extraction — stories about structured extraction in this arenaStructured extraction
Stories about structured extraction in this arena
Grounding
ai-native userEvery extracted field carries provenance — page number, bounding box, source snippet — so agents can cite and humans can verify
weight 2 · round to ExtendDocs show real provenance features—table extraction with source-page provenance, forms enrichment with bounding boxes and field values, and SEC-filing extraction with citations/verification against source—but these are scattered across specific modes (forms, tables, extract) rather than a universal guarantee that every extracted field always carries page+bbox+snippet. A community question explicitly asks whether page numbers can be returned with markdown output, suggesting this isn't uniformly available across the core Parse output. Missing for 10: a unified schema showing page number + bounding box + snippet attached to every field across all extraction modes, and independent confirmation that this works reliably in practice.
- [claimed-docs] “pull every table out into pandas DataFrames with source-page provenance, and turn on Cost Optimizer to keep costs down on long mixed-complex…”
- [claimed-docs] “Turn on the enriched forms pass to get each form page as structured JSON — fields with values, checkbox states, and bounding boxes — and rea…”
- [claimed-docs] “Extract structured data from SEC filings with citations. Verify values against the source document when extracting from complex financial do…”
- [community] “LlamaParse looks nice. Is there a way to return page numbers also with the markdown? This is important for our use case.”
Docs explicitly describe per-field citations pointing back to the exact spot on the page, confidence scores for trust/routing decisions, and multifile citations/file provenance, directly matching the provenance story (page/location + source snippet). missing for 10: no explicit mention of bounding-box coordinates format, and no independent/hands-on verification of citation accuracy beyond vendor docs.
- [claimed-docs] “plus per-field confidence scores and citations that point back to the exact spot on the page”
- [claimed-docs] “Use them to trust high-confidence values automatically and route the rest to human review.”
- [claimed-docs] “Multifile extraction lets you run a single extraction over a collection of files with a shared context.”
- [claimed-docs] “Confidence Scores | Extend Documentation”
Review
data engineerExtractions carry calibrated confidence scores with a human-in-the-loop review path for low-confidence fields
weight 2 · round to ExtendLlamaParsenone0/10No evidence of calibrated confidence scores per extracted field or any human-in-the-loop review workflow for low-confidence extractions. Docs mention citation/verification against source documents (llamaparse-docs-32) and bounding boxes for form fields (llamaparse-docs-22), but nothing about confidence scoring or a review queue/UI for flagged low-confidence fields.
- [claimed-docs] “Extract structured data from SEC filings with citations. Verify values against the source document when extracting from complex financial do…”
- [claimed-docs] “Turn on the enriched forms pass to get each form page as structured JSON — fields with values, checkbox states, and bounding boxes — and rea…”
- [claimed-docs] “Just define your Extraction Configuration (schema and settings), drag and drop any supported document into LlamaParse and extract data from …”
Extend has a dedicated Confidence Scores doc explicitly stating per-field confidence values let users 'trust high-confidence values automatically and route the rest to human review,' plus a Composer & Review Agent product feature and citations pointing to exact source locations for verification. This directly matches the calibrated confidence + human-in-the-loop review story. Missing for 10: independent/hands-on evidence validating that confidence scores are well-calibrated in practice and more detail on the review workflow UI itself.
- [claimed-docs] “Confidence Scores | Extend Documentation”
- [claimed-docs] “Use them to trust high-confidence values automatically and route the rest to human review.”
- [claimed-docs] “plus per-field confidence scores and citations that point back to the exact spot on the page”
- [claimed-docs] “Composer & Review Agent”
Schemas
developerI supply a JSON schema and get back validated structured fields extracted from the document
weight 3 · round to ExtendLlamaExtract (part of LlamaParse/LlamaCloud) explicitly supports defining an extraction schema/configuration and returning structured JSON matching that schema, including SEC filings extraction with citations and prompt-based schema generation. This directly matches the story, but evidence is entirely first-party docs with no independent/hands-on corroboration of schema validation accuracy for LlamaExtract specifically. Missing for 10: independent/community validation of extraction accuracy against schemas, explicit detail on validation/error-handling mechanics.
- [claimed-docs] “LlamaExtract provides a simple API for extracting structured data from unstructured documents like PDFs, text files, and images.”
- [claimed-docs] “Pull structured JSON out of documents that matches my schema”
- [claimed-docs] “Just define your Extraction Configuration (schema and settings), drag and drop any supported document into LlamaParse and extract data from …”
- [claimed-docs] “Extract structured data from SEC filings with citations. Verify values against the source document when extracting from complex financial do…”
- [claimed-docs] “Generate extraction schemas with a prompt”
Extend has dedicated docs for defining an Extraction Schema, returns structured fields with per-field confidence scores and citations pointing to source location, and supports sync/async extraction endpoints; community feedback corroborates strong extraction accuracy. Missing for 10: independent third-party benchmarking of schema-conformance accuracy beyond a single HN endorsement.
- [claimed-docs] “Extraction Schema | Extend Documentation”
- [claimed-docs] “Confidence Scores | Extend Documentation”
- [claimed-docs] “Use them to trust high-confidence values automatically and route the rest to human review.”
- [claimed-docs] “plus per-field confidence scores and citations that point back to the exact spot on the page”
- [claimed-docs] “Every processing endpoint in Extend (extract, classify, split, parse, and edit) supports both a synchronous and asynchronous mode.”
- [community] “We've been using Extend for over a year and have been super happy with the product and accuracy of the data extraction.”
Splitting
data engineerMulti-document packets are classified and split automatically — one upload, per-document results
weight 2 · round drawnLlamaParse's Split API explicitly segments concatenated PDFs into logical document sections based on content categories, and Classify auto-categorizes documents by type, together enabling one-upload, multi-document packet handling with per-document results (llamaparse-docs-5, llamaparse-docs-4, llamaparse-docs-37, llamaparse-docs-38). Missing for 10: independent/hands-on validation of the Split+Classify combined workflow at scale and confirmation of per-document output structure/ordering beyond docs examples.
- [claimed-docs] “The Split API provides a simple way to automatically segment concatenated PDFs into logical document sections based on content categories.”
- [claimed-docs] “Classify lets you automatically categorize documents into types you define (for example: invoice, receipt, contract) using natural-language …”
- [claimed-docs] “Split a PDF containing multiple documents (essays, research papers) into separate segments using AI-powered classification.”
- [claimed-docs] “Classify documents from the CUAD dataset as co-branding contracts or affiliate agreements.”
- [claimed-docs] “Save reusable classification rules as a configuration and reference them by ID across multiple jobs.”
Extend has dedicated Classification and Splitting overview docs plus async processing that applies to every processing endpoint (extract, classify, split, parse), and supports file upload with automatic conversion, indicating a workflow where a single multi-document upload can be classified/split into per-document outputs. Community feedback confirms satisfaction with extraction accuracy over a year of production use. Missing for 10: an explicit end-to-end worked example combining classification+splitting on one upload with per-document result output shown in a single doc, and independent hands-on verification of this exact multi-doc splitting workflow.
- [claimed-docs] “Classification Overview | Extend Documentation”
- [claimed-docs] “Splitting Overview | Extend Documentation”
- [claimed-docs] “Every processing endpoint in Extend (extract, classify, split, parse, and edit) supports both a synchronous and asynchronous mode.”
- [claimed-docs] “This endpoint accepts file contents and registers them as a File in Extend, which can be used for running workflows, creating evaluation set…”
- [claimed-docs] “If an uploaded file is detected as a Word or PowerPoint document, it will be automatically converted to a PDF.”
- [community] “We've been using Extend for over a year and have been super happy with the product and accuracy of the data extraction.”
Table extraction — stories about table extraction in this arenaTable extraction
Stories about table extraction in this arena
Tables
data engineerComplex tables — merged cells, nested headers, multi-page spans — come out as faithful HTML/markdown structure
weight 3 · round to ExtendLlamaParsedisputedcontradicted5/10LlamaParse docs claim to turn 'complex layouts, tables' into clean markdown and to pull tables into structured formats with page provenance, but a hands-on community report says the tool 'can extract numbers in tables, but the output isn't provided in tabular format—access to the numbers is only available through question-answering,' directly contradicting the claim of faithful structured table output. No evidence specifically addresses merged cells, nested headers, or multi-page spans in either direction. missing for 10: explicit documentation/testing of merged-cell, nested-header, and multi-page table handling, and independent corroboration resolving the contradiction.
- [claimed-docs] “Turn even the most complex layouts, tables, charts, handwriting, checkboxes, and images into clean markdown”
- [claimed-docs] “pull every table out into pandas DataFrames with source-page provenance, and turn on Cost Optimizer to keep costs down on long mixed-complex…”
- [claimed-docs] “Turn on the enriched forms pass to get each form page as structured JSON — fields with values, checkbox states, and bounding boxes — and rea…”
- [community] “I found the LlamaParse interface a bit confusing. It can extract numbers in tables, but the output isn't provided in tabular format—access t…”
Extend's pricing page claims it can 'Detect tables, checkboxes, images, and handwriting' and offers Agentic OCR/parsing, suggesting table-aware extraction, but there is no documentation addressing merged cells, nested headers, or multi-page table spans, nor evidence of faithful HTML/markdown structural output. Missing for 10: explicit handling of merged cells, nested headers, multi-page table continuity, and documented HTML/markdown fidelity examples.
- [claimed-docs] “Detect tables, checkboxes, images, and handwriting”
- [claimed-docs] “Agentic OCR”
data engineerI turn extracted tables into typed rows/JSON I can load into a database without manual cleanup
weight 2 · round to ExtendLlamaParsedisputedcontradicted5/10Docs claim strong support for turning tables into structured JSON/DataFrames with schemas (LlamaExtract's schema-based extraction, 'pull every table into pandas DataFrames with source-page provenance', 'table row extraction') that would fit a data-engineer's typed-row/JSON pipeline, but hands-on community feedback contradicts this: a user reports 'it can extract numbers in tables, but the output isn't provided in tabular format—access to the numbers is only available through question-answering,' suggesting real-world table extraction fell short of clean, database-ready structure. missing for 10: recent independent verification that schema-driven table extraction yields clean typed rows without manual cleanup, and resolution of the tabular-output complaint.
- [claimed-docs] “pull every table out into pandas DataFrames with source-page provenance, and turn on Cost Optimizer to keep costs down on long mixed-complex…”
- [claimed-docs] “Pull structured JSON out of documents that matches my schema”
- [claimed-docs] “Extract repeating entities from documents using table row extraction”
- [claimed-docs] “LlamaExtract provides a simple API for extracting structured data from unstructured documents like PDFs, text files, and images.”
- [claimed-docs] “Just define your Extraction Configuration (schema and settings), drag and drop any supported document into LlamaParse and extract data from …”
- [community] “I found the LlamaParse interface a bit confusing. It can extract numbers in tables, but the output isn't provided in tabular format—access t…”
Extend documents schema-based extraction that outputs structured JSON with per-field confidence scores and citations, and explicitly advertises table detection, which supports turning extracted tables into typed data with less manual cleanup; community feedback also praises extraction accuracy. However, there is no explicit documentation of table-specific row/column typing, CSV/DB-ready output formats, or direct database load integrations. Missing for 10: table-to-row schema examples, explicit typed-row/CSV export, and direct DB-loading integration guidance.
- [claimed-docs] “Extraction Schema | Extend Documentation”
- [claimed-docs] “Confidence Scores | Extend Documentation”
- [claimed-docs] “Detect tables, checkboxes, images, and handwriting”
- [claimed-docs] “plus per-field confidence scores and citations that point back to the exact spot on the page”
- [community] “We've been using Extend for over a year and have been super happy with the product and accuracy of the data extraction.”
Not comparable on these axes
ai-native userPlug MCP servers into this product so it can use their tools
weight 3 · not comparableLlamaParsen/aLlamaParse is a document-parsing/extraction service that exposes its own capabilities as an MCP server (mcp.llamaindex.ai/mcp) for other agents to call; it is not itself an agent host that plugs in and orchestrates external MCP servers' tools. The evidence pack shows only the server-provider role (docs-8, docs-16, docs-21, docs-27, probe-4), which is the wrong axis for this story.
Extendnone0/10Extend documents 'Using Extend via MCP' (docs.extend.ai/mcp), which describes exposing Extend's own tools via MCP to other agents — this is the opposite direction of the story (product consuming external MCP servers as a client). No evidence shows Extend can ingest or plug in third-party MCP servers to use their tools.
- [claimed-docs] “Using Extend via MCP | Extend Documentation”
- [probe] “official MCP server documented at https://docs.extend.ai/mcp”
ai-native userRead the product's source under an open license
weight 2 · not comparableLlamaParsenone0/10The evidence shows a GitHub repo (llama_cloud_services) but it is only a client SDK/wrapper for calling the hosted LlamaParse API, not the source of the core parsing engine, and no license terms or open-source claim for the actual product are given. There is no documentation indicating LlamaParse's core service or models are released under an open license.
- [github] “Please migrate to the new packages: - **Python**: `pip install llama-cloud>=1.0`”
Extendn/aExtend is a closed, commercial SaaS document-processing platform with paid tiers, credits, and proprietary API/CLI/SDK access; there is no mention of any open-source license or public source code repository. Open-source licensing is not a fair axis for this kind of hosted proprietary product offering.