Skip to content

How llama.cpp’s scores are calculated

The full audit trail, recomputed from the verdict data at build time through the same code that produced the leaderboard: verdict × quality × story weight per cell, cells sum to dimension scores, dimensions blend into the PA Score. Every number on the product page is reproducible from this page alone; for why the formula looks like this, see the methodology.

verdict factors: full ×1.0 · partial ×0.6 · disputed ×0.3 · none ×0.0 · n/a excluded from both sides · cell points = weight × quality × factor · cell max = weight × 10

PA Score17/100

Agent-ready 17.1 × 0.30 = 5.13

API quality 0.0 × 0.20 = 0.00

Openness 57.2 × 0.20 = 11.44

Built-in AI 0.0 × 0.15 = 0.00

Automation 0.0 × 0.15 = 0.00

(5.13 + 0.00 + 11.44 + 0.00 + 0.00) ÷ (0.30 + 0.20 + 0.20 + 0.15 + 0.15) = 16.57 ÷ 1.00 = 16.6

Scores are stored to 1 decimal; the product page’s pills round to whole numbers for display. Each dimension below shows the stories, verdicts, and cited evidence behind its number.

Agent-ready17.1/100×0.30 of the PA blend

Outside-in: can YOUR agent reach and drive this product — API, MCP, CLI, headless runs, agent docs.

Point an agent at llms.txt or agent-oriented docsweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [probe] https://github.com/llms.txtPROBE llms.txt: HTTP 200 at https://github.com/llms.txt # GitHub > GitHub is a developer platform for building, shipping, and maintaining software. It provides cloud-based Git

Run the product headlessly / in CI for automationweight 2

2 (weight) × 6 (quality) × 0.6 (partial) = 7.2 of 20 max

  • [github] https://github.com/ggml-org/llama.cppllama cli -hf ggml-org/Qwen3.5-0.8B-GGUF
  • [github] https://github.com/ggml-org/llama.cppllama serve -hf ggml-org/Qwen3.5-0.8B-GGUF
  • [github] https://github.com/ggml-org/llama.cppRun with Docker - see our [Docker documentation](docs/docker.md)
  • [github] https://github.com/ggml-org/llama.cppDownload pre-built binaries from the [releases page](https://github.com/ggml-org/llama.cpp/releases)
  • [github] https://github.com/ggml-org/llama.cppPlain C/C++ implementation without any dependencies

Plug MCP servers into this product so it can use their toolsweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Connect an agent via an official MCP serverweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Use an official CLIweight 2

2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max

  • [github] https://github.com/ggml-org/llama.cppllama cli -hf ggml-org/Qwen3.5-0.8B-GGUF
  • [github] https://github.com/ggml-org/llama.cppVLM session with `llama cli`
  • [github] https://github.com/ggml-org/llama.cppDownload pre-built binaries from the [releases page](https://github.com/ggml-org/llama.cpp/releases)
  • [community] https://news.ycombinator.com/item?id=43943047User found the vision feature 'works super well' after compiling from source, using llama-mtmd-cli with quantized multimodal models like Gemma-3, loading images via '/image image.png' in chat.

Drive the product through a documented public APIweight 3

3 (weight) × 4 (quality) × 0.6 (partial) = 7.2 of 30 max

  • [github] https://github.com/ggml-org/llama.cppllama serve -hf ggml-org/Qwen3.5-0.8B-GGUF
  • [github] https://github.com/ggml-org/llama.cppBuilt-in web UI against `llama serve` running Qwen 3.6

Issue scoped/least-privilege API credentials for an agentweight 2

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Build against official SDKsweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [github] https://github.com/ggml-org/llama.cppllama cli -hf ggml-org/Qwen3.5-0.8B-GGUF
  • [github] https://github.com/ggml-org/llama.cppllama serve -hf ggml-org/Qwen3.5-0.8B-GGUF
  • [github] https://github.com/ggml-org/llama.cppPlain C/C++ implementation without any dependencies

Subscribe to events via webhooksweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Connect a coding agent to this product as a working backendweight 3

3 (weight) × 4 (quality) × 0.6 (partial) = 7.2 of 30 max

  • [github] https://github.com/ggml-org/llama.cppllama serve -hf ggml-org/Qwen3.5-0.8B-GGUF
  • [github] https://github.com/ggml-org/llama.cppBuilt-in web UI against `llama serve` running Qwen 3.6

Agent-ready = 37.6 ÷ 220 × 100 = 17.1

API quality0.0/100×0.20 of the PA blend

The programmable surface once an agent is there — machine-readable spec, interactive docs, sandbox, versioning discipline.

Explore an interactive API reference with runnable examplesweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Download a machine-readable API spec (OpenAPI or equivalent)weight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [github] https://github.com/ggml-org/llama.cppllama serve -hf ggml-org/Qwen3.5-0.8B-GGUF
  • [github] https://github.com/ggml-org/llama.cppBuilt-in web UI against `llama serve` running Qwen 3.6

Test against a sandbox environment without touching production dataweight 1

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Rely on versioned APIs with a documented deprecation policyweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [community] https://news.ycombinator.com/item?id=43943047User noted it was 'really sad' when vision support was removed from llama.cpp previously, and expressed thanks that it's been restored.

API quality = 0.0 ÷ 60 × 100 = 0.0

Openness57.2/100×0.20 of the PA blend

Can you leave, inspect, or self-host — data export, open source, portability.

Do everything through the API that I can do in the UIweight 2

2 (weight) × 6 (quality) × 0.6 (partial) = 7.2 of 20 max

  • [github] https://github.com/ggml-org/llama.cppllama serve -hf ggml-org/Qwen3.5-0.8B-GGUF
  • [github] https://github.com/ggml-org/llama.cppBuilt-in web UI against `llama serve` running Qwen 3.6
  • [github] https://github.com/ggml-org/llama.cppVLM session with `llama cli`

Export all of my data in open formats and leaveweight 3

3 (weight) × 5 (quality) × 0.6 (partial) = 9.0 of 30 max

  • [github] https://github.com/ggml-org/llama.cppPlain C/C++ implementation without any dependencies
  • [github] https://github.com/ggml-org/llama.cpp1.5-bit, 2-bit, 3-bit, 4-bit, 5-bit, 6-bit, and 8-bit integer quantization for faster inference and reduced memory use
  • [github] https://github.com/ggml-org/llama.cppRun with Docker - see our [Docker documentation](docs/docker.md)
  • [github] https://github.com/ggml-org/llama.cppDownload pre-built binaries from the [releases page](https://github.com/ggml-org/llama.cpp/releases)
  • [community] https://news.ycombinator.com/item?id=35100086Praise for the minimal, dependency-free implementation: 'awesome being able to experiment with complex models without needing a billion python/c/cpp dependencies.'
  • [community] https://news.ycombinator.com/item?id=36304143"llama.cpp is great. It started off as CPU-only solution and now looks like it wants to support any computation device it can... totally detached from Python ML ecosystem and also popular."

Read the product's source under an open licenseweight 2

2 (weight) × 7 (quality) × 1.0 (full) = 14.0 of 20 max

  • [github] https://github.com/ggml-org/llama.cppContributors can open PRs - Collaborators will be invited based on contributions
  • [github] https://github.com/ggml-org/llama.cppPlain C/C++ implementation without any dependencies

Self-host the core productweight 3

3 (weight) × 9 (quality) × 1.0 (full) = 27.0 of 30 max

  • [github] https://github.com/ggml-org/llama.cppllama serve -hf ggml-org/Qwen3.5-0.8B-GGUF
  • [github] https://github.com/ggml-org/llama.cppPlain C/C++ implementation without any dependencies
  • [github] https://github.com/ggml-org/llama.cppRun with Docker - see our [Docker documentation](docs/docker.md)
  • [github] https://github.com/ggml-org/llama.cppDownload pre-built binaries from the [releases page](https://github.com/ggml-org/llama.cpp/releases)
  • [community] https://news.ycombinator.com/item?id=35100086User got llama.cpp working on M1 iMac trivially easily; performance was very impressive even without using Apple's neural compute hardware, and output lacked political correctness conditioning.
  • [community] https://news.ycombinator.com/item?id=35100086Praise for the minimal, dependency-free implementation: 'awesome being able to experiment with complex models without needing a billion python/c/cpp dependencies.'
  • [community] https://news.ycombinator.com/item?id=43943047User used llama.cpp's vision support with Gemma3 4b to generate keywords/descriptions for trip photos, including basic OCR and context clues to identify photo locations, calling it 'very nice for something self-hosted.'

Openness = 57.2 ÷ 100 × 100 = 57.2

Built-in AI0.0/100×0.15 of the PA blend

Inside-out: how agentic the product itself is for its users — built-in assistants, autonomous features.

Get AI-generated insights and suggestions from my data inside the productweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Set up automations that run autonomously in the backgroundweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Delegate tasks to a built-in AI assistant inside the productweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

  • [github] https://github.com/ggml-org/llama.cppllama cli -hf ggml-org/Qwen3.5-0.8B-GGUF
  • [github] https://github.com/ggml-org/llama.cppllama serve -hf ggml-org/Qwen3.5-0.8B-GGUF
  • [github] https://github.com/ggml-org/llama.cppBuilt-in web UI against `llama serve` running Qwen 3.6
  • [community] https://news.ycombinator.com/item?id=43943047User found the vision feature 'works super well' after compiling from source, using llama-mtmd-cli with quantized multimodal models like Gemma-3, loading images via '/image image.png' in chat.
  • [community] https://news.ycombinator.com/item?id=43943047User used llama.cpp's vision support with Gemma3 4b to generate keywords/descriptions for trip photos, including basic OCR and context clues to identify photo locations, calling it 'very nice for something self-hosted.'

Operate the product with natural-language commandsweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

  • [github] https://github.com/ggml-org/llama.cppllama cli -hf ggml-org/Qwen3.5-0.8B-GGUF
  • [github] https://github.com/ggml-org/llama.cppllama serve -hf ggml-org/Qwen3.5-0.8B-GGUF
  • [github] https://github.com/ggml-org/llama.cppBuilt-in web UI against `llama serve` running Qwen 3.6

Built-in AI = 0.0 ÷ 90 × 100 = 0.0

Automation0.0/100×0.15 of the PA blend

Depth of automation primitives — rules, scheduling, bulk operations, webhooks.

Perform bulk operations across many items at onceweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Define rules that trigger actions automatically on eventsweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Schedule recurring jobs or workflowsweight 2

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Version, review, and roll back my automationsweight 1

n/a — not applicable to this product: excluded from numerator and denominator

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Automation = 0.0 ÷ 50 × 100 = 0.0