Security Scanners arenaSecurity Scanners
Code-security scanning tools — secret detection, SAST, dependency/SCA, and container scanning — judged on detection accuracy, noise and false-positive management, pre-commit and CI integration, and how well agents can run scans headlessly and act on the results.
55 user stories · 330 judged cells · updated 2026-09-16 · Evidence as of 2026-09-16
Leaderboard — every product ranked by evidenceLeaderboard
| 1 | free-tier vs Semgrep ↗ | 48/100 | 37/100 | 31/100 | 23/100 | 25/100 | 17/33 verified · 5 disputed | 0/100 integrity | |||
| 2 | free-tier vs Gecko Security ↗ | 38/100 | 29/100 | 0/100 | 41/100 | 44/100 | ★ 16.6k▲ 2.5k/yr | 19/40 verified | 38/100 integrity | ||
| 3 | sponsorware vs Gecko Security ↗ | 29/100 | 18/100 | 0/100 | 76/100 | 23/100 | ★ 29.3k▲ 3.4k/yr | 2/18 verified | 0/100 integrity | ||
| 4 | 43/100 | 20/100 | 0/100 | 47/100 | 12/100 | ★ 37.9k▲ 5.1k/yr | 19/23 verified | 70/100 integrity | |||
| 5 | free-tier vs Gecko Security ↗ | 35/100 | 65/100 | 0/100 | 0/100 | 27/100 | ★ 5.7k▲ 521/yrnpm 515.7k/wk | 16/29 verified · 3 disputed | 0/100 integrity | ||
| 6 | enterprise-custom vs Gecko Security ↗ | 31/100 | 15/100 | 0/100 | 38/100 | 24/100 | ★ 27.9k▲ 2.9k/yr | 4/22 verified · 1 disputed | 0/100 integrity |
Best by user type — persona-weighted winnersBest by user type
Per persona, the product with the highest persona-weighted coverage over just that persona's stories — not the same ranking as the overall PA Score leaderboard above.
Best for security-engineer
TruffleHog
49/100
Runner-up:
Semgrep (45/100)
8 security-engineer stories scored
Story matrix — every product × every judged storyStory matrix
Agenticness — how well agents can access and operate the productAgenticness
Agent access
| Story | Persona | ||||||
|---|---|---|---|---|---|---|---|
| Agenticness — how well agents can access and operate the productPoint an agent at llms.txt or agent-oriented docs | ai-native | partialT 4/10 | fullT 8/10 | fullT 8/10 | none 0/10 | none 0/10 | fullT 9/10 |
| Agenticness — how well agents can access and operate the productRun the product headlessly / in CI for automation | ai-native | fullC 7/10 | fullX 9/10 | fullC 8/10 | fullC 8/10 | fullT 9/10 | fullT 7/10 |
| Agenticness — how well agents can access and operate the productPlug MCP servers into this product so it can use their tools | ai-native | n/a | none 0/10 | none 0/10 | n/a | n/a | none 0/10 |
| Agenticness — how well agents can access and operate the productConnect an agent via an official MCP server | ai-native | n/a | fullT 6/10 | fullT 8/10 | n/a | fullT 8/10 | fullT 7/10 |
| Agenticness — how well agents can access and operate the productUse an official CLI | ai-native | fullT 8/10 | fullT 8/10 | fullT 9/10 | fullT 8/10 | fullT 8/10 | none 0/10 |
| Agenticness — how well agents can access and operate the productDrive the product through a documented public API | ai-native | none 0/10 | partialT 6/10 | none 0/10 | none 0/10 | partialT 6/10 | fullT 8/10 |
| Agenticness — how well agents can access and operate the productIssue scoped/least-privilege API credentials for an agent | ai-native | n/a | none 0/10 | none⚿ 0/10 | n/a | n/a | disputedD 4/10 |
| Agenticness — how well agents can access and operate the productBuild against official SDKs | ai-native | none 0/10 | none 0/10 | none 0/10 | n/a | none 0/10 | partialT 5/10 |
| Agenticness — how well agents can access and operate the productSubscribe to events via webhooks | ai-native | partialT 4/10 | none 0/10 | none 0/10 | none 0/10 | none 0/10 | fullC 8/10 |
Agentic features
| Story | Persona | ||||||
|---|---|---|---|---|---|---|---|
| Agenticness — how well agents can access and operate the productGet AI-generated insights and suggestions from my data inside the product | ai-native | none 0/10 | partialX 6/10 | fullT 7/10 | none 0/10 | partialT 5/10 | disputedD 5/10 |
| Agenticness — how well agents can access and operate the productSet up automations that run autonomously in the background | ai-native | partialC 5/10 | partialC 5/10 | partialX 6/10 | partialC 6/10 | partialT 4/10 | fullC 7/10 |
| Agenticness — how well agents can access and operate the productDelegate tasks to a built-in AI assistant inside the product | ai-native | n/a | partialC 5/10 | fullT 7/10 | n/a | none 0/10 | partialT 5/10 |
| Agenticness — how well agents can access and operate the productOperate the product with natural-language commands | ai-native | n/a | partialT 3/10 | fullT 8/10 | n/a | partialT 6/10 | partialT 6/10 |
Api quality
| Story | Persona | ||||||
|---|---|---|---|---|---|---|---|
| Agenticness — how well agents can access and operate the productExplore an interactive API reference with runnable examples | ai-native | none 0/10 | none 0/10 | none 0/10 | none 0/10 | none 0/10 | none 0/10 |
| Agenticness — how well agents can access and operate the productDownload a machine-readable API spec (OpenAPI or equivalent) | ai-native | none 0/10 | none 0/10 | none 0/10 | none 0/10 | none 0/10 | fullT 9/10 |
| Agenticness — how well agents can access and operate the productTest against a sandbox environment without touching production data | ai-native | none 0/10 | n/a | none 0/10 | n/a | none 0/10 | none 0/10 |
| Agenticness — how well agents can access and operate the productRely on versioned APIs with a documented deprecation policy | ai-native | none 0/10 | none 0/10 | none 0/10 | none 0/10 | none 0/10 | partialT 3/10 |
Automation depth — how much of the product can run unattendedAutomation depth
| Story | Persona | ||||||
|---|---|---|---|---|---|---|---|
| Automation depth — how much of the product can run unattendedPerform bulk operations across many items at once | ai-native | partialC 5/10 | partialC 6/10 | fullX 7/10 | partialC 6/10 | partialX 5/10 | partialC 5/10 |
| Automation depth — how much of the product can run unattendedDefine rules that trigger actions automatically on events | ai-native | partialC 6/10 | fullC 8/10 | partialX 4/10 | partialC 5/10 | none 0/10 | partialC 5/10 |
| Automation depth — how much of the product can run unattendedSchedule recurring jobs or workflows | ai-native | none 0/10 | partialC 3/10 | none 0/10 | none 0/10 | n/a | partialC 4/10 |
| Automation depth — how much of the product can run unattendedVersion, review, and roll back my automations | ai-native | n/a | none 0/10 | none 0/10 | n/a | n/a | none 0/10 |
Ci precommit — stories about ci precommit in this arenaCi precommit
Agent scanning
| Story | Persona | ||||||
|---|---|---|---|---|---|---|---|
| Ci precommit — stories about ci precommit in this arenaHave an agent run a scan headlessly via the CLI, parse machine-readable results, and file issues for real findings | ai-native | partialC 5/10 | partialX 7/10 | partialT⚿ 5/10 | partialC 5/10 | partialX 7/10 | partialX 4/10 |
| Ci precommit — stories about ci precommit in this arenaWire the scanner into my coding agent so AI-generated code is scanned for issues before it lands | ai-native | partialC 5/10 | fullT 7/10 | fullT 9/10 | partialC 4/10 | partialT 6/10 | partialT 6/10 |
Ci gates
| Story | Persona | ||||||
|---|---|---|---|---|---|---|---|
| Ci precommit — stories about ci precommit in this arenaRun scans in CI on every pull request and fail the build on new findings via exit codes | devops-lead | partialC 6/10 | fullX 9/10 | partialT⚿ 6/10 | fullC 8/10 | partialX 4/10 | fullC 8/10 |
| Ci precommit — stories about ci precommit in this arenaSee scan findings surfaced as pull-request comments or checks in GitHub or GitLab | developer | none 0/10 | fullC 7/10 | partialC 5/10 | fullC 8/10 | none 0/10 | fullC 8/10 |
Container scanning — stories about container scanning in this arenaContainer scanning
Coverage accuracy — stories about coverage accuracy in this arenaCoverage accuracy
Coverage
| Story | Persona | ||||||
|---|---|---|---|---|---|---|---|
| Coverage accuracy — stories about coverage accuracy in this arenaScan projects across all the major languages and package ecosystems my organization uses | developer | partialC 5/10 | partialX 6/10 | fullX 7/10 | partialC 6/10 | partialX 7/10 | partialX 5/10 |
Output formats
| Story | Persona | ||||||
|---|---|---|---|---|---|---|---|
| Coverage accuracy — stories about coverage accuracy in this arenaEmit scan results in machine-readable formats like JSON or SARIF for downstream tooling | developer | none 0/10 | fullX 9/10 | none 0/10 | partialC 5/10 | none 0/10 | partialT 4/10 |
Dependency sca — stories about dependency sca in this arenaDependency sca
Dependency fixes
| Story | Persona | ||||||
|---|---|---|---|---|---|---|---|
| Dependency sca — stories about dependency sca in this arenaGet concrete upgrade paths or automated fix pull requests that remediate vulnerable dependencies | developer | n/a | none 0/10 | disputedD 5/10 | n/a | none 0/10 | none 0/10 |
Sca scanning
| Story | Persona | ||||||
|---|---|---|---|---|---|---|---|
| Dependency sca — stories about dependency sca in this arenaScan my dependency manifests and lockfiles for packages with known vulnerabilities | developer | n/a | fullX 7/10 | fullT⚿ 8/10 | n/a | fullX 9/10 | none 0/10 |
| Dependency sca — stories about dependency sca in this arenaPrioritize dependency alerts by whether the vulnerable code is actually reachable from my code | security-engineer | n/a | partialC 6/10 | partialC 6/10 | n/a | none 0/10 | none 0/10 |
Openness — open source, data portability, and self-hosting storiesOpenness
| Story | Persona | ||||||
|---|---|---|---|---|---|---|---|
| Openness — open source, data portability, and self-hosting storiesDo everything through the API that I can do in the UI | ai-native | none 0/10 | partialT 4/10 | none 0/10 | n/a | n/a | partialT 6/10 |
| Openness — open source, data portability, and self-hosting storiesExport all of my data in open formats and leave | ai-native | none 0/10 | partialC 5/10 | none 0/10 | n/a | partialC 5/10 | partialT 3/10 |
| Openness — open source, data portability, and self-hosting storiesRead the product's source under an open license | ai-native | fullT 7/10 | partialX 5/10 | none 0/10 | fullT 7/10 | partialT 6/10 | none 0/10 |
| Openness — open source, data portability, and self-hosting storiesSelf-host the core product | ai-native | fullC 8/10 | fullC 7/10 | none⚿ 0/10 | fullC 8/10 | fullT 7/10 | partialC 6/10 |
Privacy posture — data-handling and privacy storiesPrivacy posture
| Story | Persona | ||||||
|---|---|---|---|---|---|---|---|
| Privacy posture — data-handling and privacy storiesChoose where my data is stored (region/residency) | ai-native | partialC 4/10 | none 0/10 | none 0/10 | n/a | n/a | partialC 5/10 |
| Privacy posture — data-handling and privacy storiesPrevent my data from being used to train AI models | ai-native | n/a | partialC 4/10 | none 0/10 | n/a | n/a | none 0/10 |
| Privacy posture — data-handling and privacy storiesControl data retention and deletion | ai-native | none 0/10 | partialC 4/10 | none 0/10 | n/a | none 0/10 | none 0/10 |
| Privacy posture — data-handling and privacy storiesOpt out of telemetry and usage tracking | ai-native | none 0/10 | partialC 5/10 | none 0/10 | none 0/10 | none 0/10 | none 0/10 |
Sast rules — stories about sast rules in this arenaSast rules
Ai remediation
| Story | Persona | ||||||
|---|---|---|---|---|---|---|---|
| Sast rules — stories about sast rules in this arenaHave AI draft remediation pull requests for security findings that I review and merge | ai-native | n/a | partialC 4/10 | disputedD 4/10 | n/a | none 0/10 | fullX 8/10 |
Code analysis
| Story | Persona | ||||||
|---|---|---|---|---|---|---|---|
| Sast rules — stories about sast rules in this arenaDetect taint-style vulnerabilities that flow across files and function boundaries | security-engineer | n/a | fullC 8/10 | fullC 7/10 | n/a | none 0/10 | disputedD 5/10 |
| Sast rules — stories about sast rules in this arenaRun static analysis that finds vulnerability classes like injection and XSS in my source code | developer | n/a | fullX 8/10 | fullC 8/10 | n/a | none 0/10 | disputedD 5/10 |
Custom rules
| Story | Persona | ||||||
|---|---|---|---|---|---|---|---|
| Sast rules — stories about sast rules in this arenaWrite custom static-analysis rules in a simple syntax and share them across repositories | security-engineer | partialC 5/10 | fullX 8/10 | partialC 3/10 | partialC 7/10 | none 0/10 | none 0/10 |
Fix suggestions
| Story | Persona | ||||||
|---|---|---|---|---|---|---|---|
| Sast rules — stories about sast rules in this arenaApply suggested or automatic code fixes for findings instead of only reading reports | developer | none 0/10 | partialC 6/10 | disputedD 6/10 | none 0/10 | none 0/10 | fullC 8/10 |
Secret detection — stories about secret detection in this arenaSecret detection
Custom detection
| Story | Persona | ||||||
|---|---|---|---|---|---|---|---|
| Secret detection — stories about secret detection in this arenaDefine custom detection rules for my organization's internal token and secret formats | security-engineer | fullC 9/10 | partialX 6/10 | none 0/10 | fullC 8/10 | none 0/10 | none 0/10 |
Secret scanning
| Story | Persona | ||||||
|---|---|---|---|---|---|---|---|
| Secret detection — stories about secret detection in this arenaScan a repository and its full git history for hardcoded secrets and credentials | developer | fullC 9/10 | partialC 4/10 | none 0/10 | fullC 9/10 | partialC 5/10 | none 0/10 |
| Secret detection — stories about secret detection in this arenaScan non-git sources like S3 buckets, Docker images, and CI logs for leaked secrets | security-engineer | disputedD 4/10 | none 0/10 | none 0/10 | none 0/10 | partialX 4/10 | n/a |
| Secret detection — stories about secret detection in this arenaHave detected secrets verified against the issuing service so I know which ones are live | security-engineer | fullC 9/10 | fullC 8/10 | none 0/10 | none 0/10 | none 0/10 | n/a |
Supply chain sbom — stories about supply chain sbom in this arenaSupply chain sbom
Licenses
| Story | Persona | ||||||
|---|---|---|---|---|---|---|---|
| Supply chain sbom — stories about supply chain sbom in this arenaDetect the licenses of my dependencies and enforce a license policy in scans | devops-lead | n/a | none 0/10 | partialC 6/10 | n/a | partialC 5/10 | none 0/10 |
Triage noise — stories about triage noise in this arenaTriage noise
Ai triage
| Story | Persona | ||||||
|---|---|---|---|---|---|---|---|
| Triage noise — stories about triage noise in this arenaUse AI to triage findings — explaining them, deduplicating, and flagging likely false positives | ai-native | none 0/10 | partialX 6/10 | partialX 5/10 | none 0/10 | partialT 4/10 | disputedD 4/10 |
Fp management
| Story | Persona | ||||||
|---|---|---|---|---|---|---|---|
| Triage noise — stories about triage noise in this arenaBaseline or ignore existing and false-positive findings so only new issues alert | security-engineer | partialC 5/10 | partialC 4/10 | partialC 5/10 | fullC 8/10 | none 0/10 | partialX 6/10 |
| Triage noise — stories about triage noise in this arenaFilter and gate on findings by severity or confidence so low-value noise does not block builds | developer | partialC 6/10 | none 0/10 | partialC 6/10 | none 0/10 | none 0/10 | partialC 6/10 |
Adjacent arenas — categories often shopped togetherAdjacent arenas
Shopping this category often means shopping these too.