How llama.cpp’s scores are calculated
The full audit trail, recomputed from the verdict data at build time through the same code that produced the leaderboard: verdict × quality × story weight per cell, cells sum to dimension scores, dimensions blend into the PA Score. Every number on the product page is reproducible from this page alone; for why the formula looks like this, see the methodology.
verdict factors: full ×1.0 · partial ×0.6 · disputed ×0.3 · none ×0.0 · n/a excluded from both sides · cell points = weight × quality × factor · cell max = weight × 10
PA Score17/100
Agent-ready 17.1 × 0.30 = 5.13
API quality 0.0 × 0.20 = 0.00
Openness 57.2 × 0.20 = 11.44
Built-in AI 0.0 × 0.15 = 0.00
Automation 0.0 × 0.15 = 0.00
(5.13 + 0.00 + 11.44 + 0.00 + 0.00) ÷ (0.30 + 0.20 + 0.20 + 0.15 + 0.15) = 16.57 ÷ 1.00 = 16.6
Scores are stored to 1 decimal; the product page’s pills round to whole numbers for display. Each dimension below shows the stories, verdicts, and cited evidence behind its number.
Agent-ready17.1/100×0.30 of the PA blend
Outside-in: can YOUR agent reach and drive this product — API, MCP, CLI, headless runs, agent docs.
Point an agent at llms.txt or agent-oriented docsweight 2
2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max
- [probe] https://github.com/llms.txt“PROBE llms.txt: HTTP 200 at https://github.com/llms.txt # GitHub > GitHub is a developer platform for building, shipping, and maintaining software. It provides cloud-based Git”
Run the product headlessly / in CI for automationweight 2
2 (weight) × 6 (quality) × 0.6 (partial) = 7.2 of 20 max
- [github] https://github.com/ggml-org/llama.cpp“llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF”
- [github] https://github.com/ggml-org/llama.cpp“llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF”
- [github] https://github.com/ggml-org/llama.cpp“Run with Docker - see our [Docker documentation](docs/docker.md)”
- [github] https://github.com/ggml-org/llama.cpp“Download pre-built binaries from the [releases page](https://github.com/ggml-org/llama.cpp/releases)”
- [github] https://github.com/ggml-org/llama.cpp“Plain C/C++ implementation without any dependencies”
Plug MCP servers into this product so it can use their toolsweight 3
3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
Connect an agent via an official MCP serverweight 3
3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
Use an official CLIweight 2
2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max
- [github] https://github.com/ggml-org/llama.cpp“llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF”
- [github] https://github.com/ggml-org/llama.cpp“VLM session with `llama cli`”
- [github] https://github.com/ggml-org/llama.cpp“Download pre-built binaries from the [releases page](https://github.com/ggml-org/llama.cpp/releases)”
- [community] https://news.ycombinator.com/item?id=43943047“User found the vision feature 'works super well' after compiling from source, using llama-mtmd-cli with quantized multimodal models like Gemma-3, loading images via '/image image.png' in chat.”
Drive the product through a documented public APIweight 3
3 (weight) × 4 (quality) × 0.6 (partial) = 7.2 of 30 max
Issue scoped/least-privilege API credentials for an agentweight 2
n/a — not applicable to this product: excluded from numerator and denominator
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
Build against official SDKsweight 2
2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max
Subscribe to events via webhooksweight 2
2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
Connect a coding agent to this product as a working backendweight 3
3 (weight) × 4 (quality) × 0.6 (partial) = 7.2 of 30 max
Agent-ready = 37.6 ÷ 220 × 100 = 17.1
API quality0.0/100×0.20 of the PA blend
The programmable surface once an agent is there — machine-readable spec, interactive docs, sandbox, versioning discipline.
Explore an interactive API reference with runnable examplesweight 2
2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
Download a machine-readable API spec (OpenAPI or equivalent)weight 2
2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max
Test against a sandbox environment without touching production dataweight 1
n/a — not applicable to this product: excluded from numerator and denominator
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
Rely on versioned APIs with a documented deprecation policyweight 2
2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max
- [community] https://news.ycombinator.com/item?id=43943047“User noted it was 'really sad' when vision support was removed from llama.cpp previously, and expressed thanks that it's been restored.”
API quality = 0.0 ÷ 60 × 100 = 0.0
Openness57.2/100×0.20 of the PA blend
Can you leave, inspect, or self-host — data export, open source, portability.
Do everything through the API that I can do in the UIweight 2
2 (weight) × 6 (quality) × 0.6 (partial) = 7.2 of 20 max
Export all of my data in open formats and leaveweight 3
3 (weight) × 5 (quality) × 0.6 (partial) = 9.0 of 30 max
- [github] https://github.com/ggml-org/llama.cpp“Plain C/C++ implementation without any dependencies”
- [github] https://github.com/ggml-org/llama.cpp“1.5-bit, 2-bit, 3-bit, 4-bit, 5-bit, 6-bit, and 8-bit integer quantization for faster inference and reduced memory use”
- [github] https://github.com/ggml-org/llama.cpp“Run with Docker - see our [Docker documentation](docs/docker.md)”
- [github] https://github.com/ggml-org/llama.cpp“Download pre-built binaries from the [releases page](https://github.com/ggml-org/llama.cpp/releases)”
- [community] https://news.ycombinator.com/item?id=35100086“Praise for the minimal, dependency-free implementation: 'awesome being able to experiment with complex models without needing a billion python/c/cpp dependencies.'”
- [community] https://news.ycombinator.com/item?id=36304143“"llama.cpp is great. It started off as CPU-only solution and now looks like it wants to support any computation device it can... totally detached from Python ML ecosystem and also popular."”
Read the product's source under an open licenseweight 2
2 (weight) × 7 (quality) × 1.0 (full) = 14.0 of 20 max
Self-host the core productweight 3
3 (weight) × 9 (quality) × 1.0 (full) = 27.0 of 30 max
- [github] https://github.com/ggml-org/llama.cpp“llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF”
- [github] https://github.com/ggml-org/llama.cpp“Plain C/C++ implementation without any dependencies”
- [github] https://github.com/ggml-org/llama.cpp“Run with Docker - see our [Docker documentation](docs/docker.md)”
- [github] https://github.com/ggml-org/llama.cpp“Download pre-built binaries from the [releases page](https://github.com/ggml-org/llama.cpp/releases)”
- [community] https://news.ycombinator.com/item?id=35100086“User got llama.cpp working on M1 iMac trivially easily; performance was very impressive even without using Apple's neural compute hardware, and output lacked political correctness conditioning.”
- [community] https://news.ycombinator.com/item?id=35100086“Praise for the minimal, dependency-free implementation: 'awesome being able to experiment with complex models without needing a billion python/c/cpp dependencies.'”
- [community] https://news.ycombinator.com/item?id=43943047“User used llama.cpp's vision support with Gemma3 4b to generate keywords/descriptions for trip photos, including basic OCR and context clues to identify photo locations, calling it 'very nice for something self-hosted.'”
Openness = 57.2 ÷ 100 × 100 = 57.2
Built-in AI0.0/100×0.15 of the PA blend
Inside-out: how agentic the product itself is for its users — built-in assistants, autonomous features.
Get AI-generated insights and suggestions from my data inside the productweight 2
2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
Set up automations that run autonomously in the backgroundweight 2
2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
Delegate tasks to a built-in AI assistant inside the productweight 3
3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max
- [github] https://github.com/ggml-org/llama.cpp“llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF”
- [github] https://github.com/ggml-org/llama.cpp“llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF”
- [github] https://github.com/ggml-org/llama.cpp“Built-in web UI against `llama serve` running Qwen 3.6”
- [community] https://news.ycombinator.com/item?id=43943047“User found the vision feature 'works super well' after compiling from source, using llama-mtmd-cli with quantized multimodal models like Gemma-3, loading images via '/image image.png' in chat.”
- [community] https://news.ycombinator.com/item?id=43943047“User used llama.cpp's vision support with Gemma3 4b to generate keywords/descriptions for trip photos, including basic OCR and context clues to identify photo locations, calling it 'very nice for something self-hosted.'”
Operate the product with natural-language commandsweight 2
2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max
Built-in AI = 0.0 ÷ 90 × 100 = 0.0
Automation0.0/100×0.15 of the PA blend
Depth of automation primitives — rules, scheduling, bulk operations, webhooks.
Perform bulk operations across many items at onceweight 2
2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
Define rules that trigger actions automatically on eventsweight 3
3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
Schedule recurring jobs or workflowsweight 2
n/a — not applicable to this product: excluded from numerator and denominator
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
Version, review, and roll back my automationsweight 1
n/a — not applicable to this product: excluded from numerator and denominator
no evidence cited — the verdict rests on absence of evidence, re-checked on refresh
Automation = 0.0 ÷ 50 × 100 = 0.0