Skip to content
ProductArena

How Mistral Document AI’s scores are calculated

The full audit trail, recomputed from the verdict data at build time through the same code that produced the leaderboard: verdict × quality × story weight per cell, cells sum to dimension scores, dimensions blend into the PA Score. Every number on the product page is reproducible from this page alone; for why the formula looks like this, see the methodology.

verdict factors: full ×1.0 · partial ×0.6 · disputed ×0.3 · none ×0.0 · n/a excluded from both sides · cell points = weight × quality × factor · cell max = weight × 10

PA Score20/100

Agent-ready 29.5 × 0.30 = 8.85

API quality 0.0 × 0.20 = 0.00

Openness 18.0 × 0.20 = 3.60

Built-in AI 28.9 × 0.15 = 4.33

Automation 24.0 × 0.15 = 3.60

(8.85 + 0.00 + 3.60 + 4.33 + 3.60) ÷ (0.30 + 0.20 + 0.20 + 0.15 + 0.15) = 20.38 ÷ 1.00 = 20.4

Scores are stored to 1 decimal; the product page’s pills round to whole numbers for display. Each dimension below shows the stories, verdicts, and cited evidence behind its number.

Agent-ready29.5/100×0.30 of the PA blend

Outside-in: can YOUR agent reach and drive this product — API, MCP, CLI, headless runs, agent docs.

Point an agent at llms.txt or agent-oriented docsweight 2

2 (weight) × 6 (quality) × 0.6 (partial) = 7.2 of 20 max

  • [probe] https://docs.mistral.ai/llms.txt“PROBE llms.txt: HTTP 200 at https://docs.mistral.ai/llms.txt # MistralAI ## Docs [Agents & Conversations](https://docs.mistral.ai/docs/agents/agents_and_conversations.md): Agents”
  • [probe] https://docs.mistral.ai/studio/document-processing/overview.md“PROBE docs-md: HTTP 404 at https://docs.mistral.ai/studio/document-processing/overview.md”

Run the product headlessly / in CI for automationweight 2

2 (weight) × 6 (quality) × 0.6 (partial) = 7.2 of 20 max

  • [claimed-docs] https://docs.mistral.ai/resources/sdks“We provide official SDK clients in both Python and Typescript”
  • [claimed-docs] https://docs.mistral.ai/studio/document-processing/basic_ocr“For PDFs, pass a publicly available URL, pass a Base64-encoded PDF, or upload a PDF file.”
  • [community] https://hn.algolia.com/api/v1/items/43282905“This worked indeed. Although I had to cut my document into smaller chunks. 900 pages at once ended with a timeout.”

Plug MCP servers into this product so it can use their toolsweight 3

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Connect an agent via an official MCP serverweight 3

n/a — not applicable to this product: excluded from numerator and denominator

  • [claimed-docs] https://docs.mistral.ai/resources/sdks“We provide official SDK clients in both Python and Typescript”

Use an official CLIweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [claimed-docs] https://docs.mistral.ai/resources/sdks“We provide official SDK clients in both Python and Typescript”

Drive the product through a documented public APIweight 3

3 (weight) × 8 (quality) × 1.0 (full) = 24.0 of 30 max

  • [claimed-docs] https://docs.mistral.ai/studio/document-processing/basic_ocr“Use the Document AI OCR processor to extract text and structured content from PDF documents and images.”
  • [claimed-docs] https://docs.mistral.ai/studio/document-processing/basic_ocr“Block extraction uses the include_blocks parameter. When enabled, each page contains a blocks array with paragraph-level bounding boxes, structural block labels, and extracted content in reading order.”
  • [claimed-docs] https://docs.mistral.ai/studio/document-processing/basic_ocr“Confidence scores are available for extracted content at page, block, or word granularity through the confidence_scores_granularity parameter.”
  • [claimed-docs] https://docs.mistral.ai/studio/document-processing/basic_ocr“For PDFs, pass a publicly available URL, pass a Base64-encoded PDF, or upload a PDF file.”
  • [claimed-docs] https://docs.mistral.ai/studio/document-processing/basic_ocr“Document formats include: `image_url`: PNG, JPEG/JPG, AVIF, and other image formats. `document_url`: PDF, PPTX, DOCX, and other document formats.”
  • [claimed-docs] https://docs.mistral.ai/resources/sdks“We provide official SDK clients in both Python and Typescript”
  • [community] https://hn.algolia.com/api/v1/items/43282905“Dang. Super fast and significantly more accurate than google, Claude and others. Pricing: $1/1000 pages... this looks great at pdf to markdown.”
  • [probe] https://docs.mistral.ai/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.mistral.ai/openapi.json, https://docs.mistral.ai/swagger.json, https://docs.mistral.ai/api/openapi.json, https://docs.mistral.ai/.well-known/openapi.json)”

Issue scoped/least-privilege API credentials for an agentweight 2

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Build against official SDKsweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Subscribe to events via webhooksweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Agent-ready = 38.4 ÷ 130 × 100 = 29.5

API quality0.0/100×0.20 of the PA blend

The programmable surface once an agent is there — machine-readable spec, interactive docs, sandbox, versioning discipline.

Explore an interactive API reference with runnable examplesweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [probe] https://docs.mistral.ai/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.mistral.ai/openapi.json, https://docs.mistral.ai/swagger.json, https://docs.mistral.ai/api/openapi.json, https://docs.mistral.ai/.well-known/openapi.json)”
  • [probe] https://docs.mistral.ai/studio/document-processing/overview.md“PROBE docs-md: HTTP 404 at https://docs.mistral.ai/studio/document-processing/overview.md”
  • [claimed-docs] https://docs.mistral.ai/resources/sdks“We provide official SDK clients in both Python and Typescript”

Download a machine-readable API spec (OpenAPI or equivalent)weight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [probe] https://docs.mistral.ai/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.mistral.ai/openapi.json, https://docs.mistral.ai/swagger.json, https://docs.mistral.ai/api/openapi.json, https://docs.mistral.ai/.well-known/openapi.json)”
  • [claimed-docs] https://docs.mistral.ai/resources/sdks“We provide official SDK clients in both Python and Typescript”

Test against a sandbox environment without touching production dataweight 1

1 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 10 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Rely on versioned APIs with a documented deprecation policyweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [probe] https://docs.mistral.ai/openapi.json“PROBE openapi: all candidate paths 404 (https://docs.mistral.ai/openapi.json, https://docs.mistral.ai/swagger.json, https://docs.mistral.ai/api/openapi.json, https://docs.mistral.ai/.well-known/openapi.json)”
  • [probe] https://docs.mistral.ai/studio/document-processing/overview.md“PROBE docs-md: HTTP 404 at https://docs.mistral.ai/studio/document-processing/overview.md”

API quality = 0.0 ÷ 70 × 100 = 0.0

Openness18.0/100×0.20 of the PA blend

Can you leave, inspect, or self-host — data export, open source, portability.

Do everything through the API that I can do in the UIweight 2

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Export all of my data in open formats and leaveweight 3

3 (weight) × 3 (quality) × 0.6 (partial) = 5.4 of 30 max

  • [claimed-docs] https://docs.mistral.ai/studio/document-processing/basic_ocr“Table formatting supports null, markdown, and html values through the table_format parameter.”
  • [claimed-docs] https://docs.mistral.ai/studio/document-processing/basic_ocr“Table formatting supports `null`, `markdown`, and `html` values through the `table_format` parameter.”
  • [claimed-docs] https://docs.mistral.ai/studio/document-processing/basic_ocr“Block extraction uses the include_blocks parameter. When enabled, each page contains a blocks array with paragraph-level bounding boxes, structural block labels, and extracted content in reading order.”
  • [claimed-docs] https://mistral.ai/solutions/document-ai/“Document AI solution page: positioned for "Compliance-first organizations requiring secure on-premises deployment" with "Secure deployments" — Mistral sells self-hosted/on-prem enterprise deployments of its models alongside La Plateforme, and hosts a public Trust Center at trust.mistral.ai.”

Read the product's source under an open licenseweight 2

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Self-host the core productweight 3

3 (weight) × 3 (quality) × 0.6 (partial) = 5.4 of 30 max

  • [claimed-docs] https://mistral.ai/solutions/document-ai/“Document AI solution page: positioned for "Compliance-first organizations requiring secure on-premises deployment" with "Secure deployments" — Mistral sells self-hosted/on-prem enterprise deployments of its models alongside La Plateforme, and hosts a public Trust Center at trust.mistral.ai.”

Openness = 10.8 ÷ 60 × 100 = 18.0

Built-in AI28.9/100×0.15 of the PA blend

Inside-out: how agentic the product itself is for its users — built-in assistants, autonomous features.

Get AI-generated insights and suggestions from my data inside the productweight 2

2 (weight) × 7 (quality) × 1.0 (full) = 14.0 of 20 max

  • [claimed-docs] https://docs.mistral.ai/studio/document-processing/document_qna“The Document QnA capability combines OCR with large language model capabilities to enable natural language interaction with document content.”
  • [claimed-docs] https://docs.mistral.ai/studio/document-processing/document_qna“This allows you to extract information and insights from documents by asking questions in natural language.”
  • [claimed-docs] https://docs.mistral.ai/studio/document-processing/document_qna“Multi-document queries and comparisons”
  • [claimed-docs] https://docs.mistral.ai/studio/document-processing/annotations“Extraction of key information like vendor details and amounts from invoices for automated accounting.”
  • [claimed-docs] https://docs.mistral.ai/studio/document-processing/annotations“Extraction of key clauses and terms from contracts for easier review and management”
  • [community] https://hn.algolia.com/api/v1/items/48645152“I used Abbyy Finereader for several years. I loved it... Modern VLMs put classic FineReader to shame for processing low-resolution/degraded/non-standard text. If you have an OCR problem, Mistral OCR 4 is probably great.”

Set up automations that run autonomously in the backgroundweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Delegate tasks to a built-in AI assistant inside the productweight 3

3 (weight) × 4 (quality) × 0.6 (partial) = 7.2 of 30 max

  • [claimed-docs] https://docs.mistral.ai/studio/document-processing/document_qna“The Document QnA capability combines OCR with large language model capabilities to enable natural language interaction with document content.”
  • [claimed-docs] https://docs.mistral.ai/studio/document-processing/document_qna“Multi-document queries and comparisons”
  • [claimed-docs] https://docs.mistral.ai/studio/document-processing/document_qna“Building document Q&A applications”
  • [claimed-docs] https://docs.mistral.ai/studio/document-processing/document_qna“This allows you to extract information and insights from documents by asking questions in natural language.”

Operate the product with natural-language commandsweight 2

2 (weight) × 4 (quality) × 0.6 (partial) = 4.8 of 20 max

  • [claimed-docs] https://docs.mistral.ai/studio/document-processing/document_qna“The Document QnA capability combines OCR with large language model capabilities to enable natural language interaction with document content.”
  • [claimed-docs] https://docs.mistral.ai/studio/document-processing/document_qna“Multi-document queries and comparisons”
  • [claimed-docs] https://docs.mistral.ai/studio/document-processing/document_qna“Building document Q&A applications”

Built-in AI = 26.0 ÷ 90 × 100 = 28.9

Automation24.0/100×0.15 of the PA blend

Depth of automation primitives — rules, scheduling, bulk operations, webhooks.

Perform bulk operations across many items at onceweight 2

2 (weight) × 4 (quality) × 0.6 (partial) = 4.8 of 20 max

  • [claimed-docs] https://docs.mistral.ai/studio/document-processing/document_qna“Multi-document queries and comparisons”
  • [community] https://hn.algolia.com/api/v1/items/43282905“This worked indeed. Although I had to cut my document into smaller chunks. 900 pages at once ended with a timeout.”
  • [community] https://hn.algolia.com/api/v1/items/48645152“Yes, we have successfully used Mistral OCR for digitizing handwritten forms. You always have a low percentage that need human review, but overall Mistral has been highly accurate (their price is amazing, too).”

Define rules that trigger actions automatically on eventsweight 3

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Schedule recurring jobs or workflowsweight 2

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Version, review, and roll back my automationsweight 1

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Automation = 4.8 ÷ 20 × 100 = 24.0