TruffleHog vs Semgrep

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to Semgrep
   TruffleHog~partialprobed4/10

   llms.txt itself returns a 404, but the docs site serves markdown versions of pages (e.g. docs.md, pre-commit-hooks.md, customizing-detection.md) that an agent could fetch directly, showing some agent-oriented doc structure without a formal llms.txt manifest. missing for 10: a working llms.txt file, any explicit agent/LLM-consumption guidance, and confirmation these .md endpoints are documented as an intentional agent-facing interface.

   • [probe] “PROBE llms.txt: HTTP 404 at https://trufflesecurity.com/llms.txt”
   • [probe] “PROBE docs-md: HTTP 200 at https://trufflesecurity.com/docs.md --- title: Choose your adventure slug: docs description: Compare TruffleHog O…”
   • [claimed-docs] “Pre-commit hooks are a useful way to prevent secrets in code from being pushed from a git repository.”
   • [claimed-docs] “The custom regex detector allows you to define your own detector using regular expressions with optional verification using a webhook.”

   proof ↗⚑ flag

   Semgrep✓fullprobed8/10

   Semgrep hosts a working llms.txt at docs.semgrep.dev/llms.txt (HTTP 200, confirmed by probe) plus per-page .md agent-friendly docs (e.g. semgrep-docs-49, semgrep-docs-50), showing an agent can be pointed directly at machine-readable docs. missing for 10: independent/community confirmation that agents actually consume this llms.txt successfully, and no evidence of a broader agent-oriented docs index beyond the single llms.txt file.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.semgrep.dev/llms.txt # Semgrep ## Home - [Semgrep Docs](https://docs.semgrep.dev/index.md): Get s…”
   • [claimed-docs] “scan your first project, which can be any codebase, repository, or folder within a monorepo, for security issues, and view your findings”
   • [claimed-docs] “You can run local Semgrep CLI scans with the Semgrep command-line tool.”
   • [probe] “official CLI documented at https://docs.semgrep.dev/getting-started/cli”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to Semgrep
   TruffleHog✓fullclaimed7/10

   TruffleHog ships as a CLI and Docker image that can be invoked non-interactively (e.g. `trufflehog github --only-verified --repo ...`, `docker run ... trufflesecurity/trufflehog:latest github --org=...`) and supports automation-friendly flags like `--fail` and `--results=verified,unknown` for pipeline gating, plus pre-commit/pre-receive hook integration for git automation. missing for 10: explicit first-party CI/CD platform docs (e.g. GitHub Actions, GitLab CI templates) and independent hands-on confirmation of headless CI usage beyond docs.

   • [claimed-docs] “trufflehog github --only-verified --repo https://github.com/trufflesecurity/test_keys”
   • [claimed-docs] “trufflehog git file://. --since-commit HEAD --results=verified,unknown --fail”
   • [github] “docker run --rm -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --org=trufflesecurity”
   • [github] “docker run --rm -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --repo https://github.com/trufflesecurity/test_keys”
   • [claimed-docs] “Pre-commit hooks are a useful way to prevent secrets in code from being pushed from a git repository.”
   • [claimed-docs] “This approach uses Git's `core.hooksPath` to apply hooks to all repositories without requiring any per-repository setup”
   • [github] “brew install trufflehog”

   proof ↗⚑ flag

   Semgrep✓fullcommunity9/10

   Semgrep has a well-documented CLI (`semgrep ci`, `semgrep scan`) designed for headless/CI use, with dedicated CI provider integrations, pre-commit hooks, SARIF export, no-login/dry-run modes, and an API for programmatic access to findings—clearly supporting automated, non-interactive pipelines. Independent evidence (Zulip, Microsoft SARIF+LLM triage pipeline) confirms real-world CI/automation usage at scale. Missing for 10: no first-party benchmark of CI runtime performance at scale beyond anecdotal community complaints about slowness.

   • [claimed-docs] “navigate to the root of your project, and run your first scan: semgrep ci”
   • [claimed-docs] “This guide walks you through creating a Semgrep job in the following CI providers, which are explicitly supported in Semgrep AppSec Platform”
   • [claimed-docs] “your development cycle benefits from the automated scanning of repositories at various events, such as: Push events; Pull requests or merge …”
   • [claimed-docs] “Semgrep is integrated into CI environments by creating a **job** that is run by the CI provider. After a scan, findings are sent to Semgrep …”
   • [claimed-docs] “The pre-commit framework can run semgrep when you commit changes. This is helpful in preventing secrets and security issues from leaking int…”
   • [claimed-docs] “The `--dry-run` flag ensures that your scans are not uploaded to the Semgrep web app.”
   • [claimed-docs] “You do **not** need to be logged in to run a scan.”
   • [claimed-docs] “To export the results to a SARIF file: semgrep ci --sarif --sarif-output=semgrep.sarif”
   • [claimed-docs] “Semgrep AppSec Platform provides an API that enables you to list deployments, gather findings created by Semgrep”
   • [community] “Built a tool using Semgrep's SARIF output plus prompted LLMs to help prioritize triage; used at Microsoft for the past year to help surface …”
   • [community] “We've used Semgrep for Zulip's Python codebase for months and really appreciate the semantic checks, especially for security lint rules - it…”

   proof ↗⚑ flag

3. ai-native userUse an official CLI

   weight 2 · round drawn
   TruffleHog✓fullprobed8/10

   TruffleHog ships a well-documented official CLI (installable via brew/docker) with a rich set of subcommands and flags (git, github, scan, analyze, --include-detectors, --verify-detectors, --fail, --results) suitable for scripted/agentic invocation, and is explicitly used in automation contexts like pre-commit hooks and CI pipelines. Missing for 10: explicit documentation of machine-readable (JSON) output formats or exit-code semantics tailored for agent/automation consumption, and independent corroboration beyond vendor docs/GitHub.

   • [github] “brew install trufflehog”
   • [github] “docker run --rm -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --org=trufflesecurity”
   • [github] “docker run --rm -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --repo https://github.com/trufflesecurity/test_keys”
   • [claimed-docs] “trufflehog git file://. --since-commit HEAD --results=verified,unknown --fail”
   • [claimed-docs] “./trufflehog scan --config=config.yaml --include-detectors=AWS,GitHub”
   • [claimed-docs] “You can manually specify which detectors to use with the `--include-detectors` flag.”
   • [claimed-docs] “the `--verify-detectors` and `--no-verify-detectors` CLI flags can be used to configure detector-specific verification override settings”
   • [probe] “official CLI documented at https://github.com/trufflesecurity/trufflehog”

   proof ↗⚑ flag

   Semgrep✓fullprobed8/10

   Semgrep ships a well-documented, installable official CLI (pipx/uv install, `semgrep scan`/`semgrep ci`, SARIF export, autofix, dry-run, no-login mode) that is fully scriptable and thus usable by AI agents, and this is corroborated by community usage reports and a dedicated probe confirming the CLI docs page. Missing for 10: explicit documentation of machine-readable (e.g., JSON) output tailored for programmatic/agent consumption and independent evidence of AI agents specifically driving the CLI.

   • [claimed-docs] “pipx install semgrep # preferred: install through uv (https://docs.astral.sh/uv/) uv tool install semgrep”
   • [claimed-docs] “semgrep scan - This is the recommended command for scanning local codebases or scanning a project when you don't have a Semgrep account”
   • [claimed-docs] “navigate to the root of your project, and run your first scan: semgrep ci”
   • [claimed-docs] “The `--dry-run` flag ensures that your scans are not uploaded to the Semgrep web app.”
   • [claimed-docs] “You do **not** need to be logged in to run a scan.”
   • [claimed-docs] “To export the results to a SARIF file: semgrep ci --sarif --sarif-output=semgrep.sarif”
   • [probe] “official CLI documented at https://docs.semgrep.dev/getting-started/cli”
   • [community] “We've used Semgrep for Zulip's Python codebase for months and really appreciate the semantic checks, especially for security lint rules - it…”

   proof ↗⚑ flag

4. ai-native userDrive the product through a documented public API

   weight 3 · round to Semgrep
   TruffleHog—none0/10

   No evidence of a documented public API (REST/GraphQL/SDK) for programmatically driving TruffleHog; probes for llms.txt and openapi/swagger specs both 404, and only CLI flags/config and a webhook for custom-detector verification are documented, not a driving API.

   • [probe] “PROBE llms.txt: HTTP 404 at https://trufflesecurity.com/llms.txt”
   • [probe] “PROBE openapi: all candidate paths 404 (https://trufflesecurity.com/openapi.json, https://trufflesecurity.com/swagger.json, https://truffles…”
   • [claimed-docs] “Verification is done via a webhook POST request to the provided `endpoint`.”

   proof ↗⚑ flag

   Semgrep~partialprobed6/10

   Semgrep AppSec Platform documents a public API for listing deployments, findings, and projects, and the CLI itself is scriptable for automation, but there is no OpenAPI/swagger spec discoverable (probe returned 404s on all candidate paths) and no independent/community corroboration of programmatic API usage. missing for 10: discoverable OpenAPI/schema documentation, broader API surface details (write operations, rate limits), and independent evidence of third-party integrations built against the API.

   • [claimed-docs] “Semgrep AppSec Platform provides an API that enables you to list deployments, gather findings created by Semgrep AppSec Platform, and list p…”
   • [claimed-docs] “Semgrep AppSec Platform provides an API that enables you to list deployments, gather findings created by Semgrep”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.semgrep.dev/openapi.json, https://docs.semgrep.dev/swagger.json, https://docs.semgrep.d…”

   proof ↗⚑ flag

5. ai-native userBuild against official SDKs

   weight 2 · round drawn
   TruffleHog—none0/10

   The evidence pack documents TruffleHog's CLI, Docker image, detectors, and API-like analyzer features, but contains no mention of an official SDK (Python, JS, Go client library, etc.) for developers to build against; probes for OpenAPI specs also returned 404s, indicating no such offering.

   • [probe] “PROBE openapi: all candidate paths 404 (https://trufflesecurity.com/openapi.json, https://trufflesecurity.com/swagger.json, https://truffles…”
   • [github] “brew install trufflehog”
   • [claimed-docs] “trufflehog github --only-verified --repo https://github.com/trufflesecurity/test_keys”

   proof ↗⚑ flag

   Semgrep—none0/10

   The evidence shows Semgrep offers a CLI, a REST API for the AppSec Platform, and an MCP server, but nowhere is an official client SDK (e.g., Python/JS/Go library for programmatic integration) documented or referenced.

   • [claimed-docs] “Semgrep AppSec Platform provides an API that enables you to list deployments, gather findings created by Semgrep AppSec Platform, and list p…”
   • [claimed-docs] “Semgrep AppSec Platform provides an API that enables you to list deployments, gather findings created by Semgrep”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.semgrep.dev/openapi.json, https://docs.semgrep.dev/swagger.json, https://docs.semgrep.d…”
   • [probe] “official CLI documented at https://docs.semgrep.dev/getting-started/cli”

   proof ↗⚑ flag

6. ai-native userSubscribe to events via webhooks

   weight 2 · round to TruffleHog
   TruffleHog~partialprobed4/10

   Docs mention routing findings to webhooks and setting up alerts across chosen platforms, indicating an outbound webhook notification capability exists, but there is no detailed documentation of webhook subscription/configuration, payload schema, or an API/OpenAPI spec (probe shows no OpenAPI found) to confirm depth of support. missing for 10: documented webhook configuration/subscription API, payload schema examples, independent/hands-on confirmation.

   • [claimed-docs] “Route findings to Slack, Jira, email, or webhooks so the right people can act on them.”
   • [claimed-docs] “TruffleHog continuously tracks the status of all key types to identify whether remediation has occurred. Set up alerts across the platform o…”
   • [claimed-docs] “TruffleHog continuously tracks the status of all key types to identify whether remediation has occurred. Set up alerts across the platform o…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://trufflesecurity.com/openapi.json, https://trufflesecurity.com/swagger.json, https://truffles…”

   proof ↗⚑ flag

   Semgrep—none0/10

   Evidence shows Semgrep provides a REST API for querying findings/deployments and CI-integration on push/PR events, but nothing describes a webhook subscription mechanism for events; no docs mention webhook endpoints or event-driven notifications to external systems.

   • [claimed-docs] “Semgrep AppSec Platform provides an API that enables you to list deployments, gather findings created by Semgrep AppSec Platform, and list p…”
   • [claimed-docs] “Semgrep AppSec Platform provides an API that enables you to list deployments, gather findings created by Semgrep”
   • [claimed-docs] “your development cycle benefits from the automated scanning of repositories at various events, such as: Push events; Pull requests or merge …”

   proof ↗⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round to Semgrep
   TruffleHog—none0/10

   TruffleHog is a secrets-scanning/detection tool; the evidence pack describes verification, classification, and analyzer features but no AI-generated insights, summaries, or suggestions surfaced to users—no LLM-based feature is mentioned anywhere.

   ⚑ flag

   Semgrep~partialcommunity6/10

   Semgrep Code's 'AI-Powered Detection' and 'Semgrep Assistant (AI)' features are documented as generating insights (identifying complex business-logic flaws like IDORs, reducing false-positive noise by ~20%, combining AI reasoning with rule-based analysis for triage/remediation), which matches the story of in-product AI-generated insights. However, independent corroboration is thin — the one relevant community datapoint (semgrep-comm-14) describes a third party building their own external LLM triage layer on top of Semgrep's SARIF output rather than validating Semgrep Assistant itself, and another (semgrep-comm-13) notes an LLM hallucinating semgrep syntax when asked to help write rules, which is tangential. missing for 10: independent/hands-on validation of Semgrep Assistant's own AI suggestions, more detail on scope/accuracy of AI-Powered Detection, and confirmation these AI features are broadly available rather than gated/beta.

   • [claimed-docs] “With AI-Powered Detection, you can automatically identify complex business logic flaws, such as IDORs and broken authorization.”
   • [github] “Contextual post-processing of findings with Semgrep Assistant (AI) to further reduce noise by ~20%”
   • [claimed-docs] “Combine AI reasoning with rule-based analysis for detection, triage, and remediation”
   • [claimed-docs] “Semgrep Code is transparent; you can configure the rules it runs and inspect its syntax to understand how the finding was detected.”
   • [community] “Built a tool using Semgrep's SARIF output plus prompted LLMs to help prioritize triage; used at Microsoft for the past year to help surface …”

   proof ↗⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round drawn
   TruffleHog~partialclaimed5/10

   TruffleHog supports background automation in the sense of continuous monitoring of sources (Git, Jira, Slack, etc.), hosted/self-hosted scanning, and automated alert routing (Slack, Jira, webhooks) that can run unattended, which fits the 'runs autonomously in the background' part of the story. However, this is generic security-scanning automation, not an AI-agent-specific or LLM-orchestrated automation framework, and there's no evidence of agent-triggered workflows, scheduling APIs, or AI-native orchestration hooks beyond CI/pre-commit hooks and continuous monitoring. Missing for 10: explicit AI-agent/LLM-triggered automation setup, scheduling/orchestration API for autonomous runs, and independent confirmation that background monitoring runs reliably unattended.

   • [github] “Are you interested in continuously monitoring Git, Jira, Slack, Confluence, Microsoft Teams, Sharepoint (and more) for credentials?”
   • [github] “continuously monitoring Git, Jira, Slack, Confluence, Microsoft Teams, Sharepoint (and more) for credentials”
   • [claimed-docs] “TruffleHog continuously tracks the status of all key types to identify whether remediation has occurred. Set up alerts across the platform o…”
   • [claimed-docs] “TruffleHog continuously tracks the status of all key types to identify whether remediation has occurred. Set up alerts across the platform o…”
   • [claimed-docs] “Deploy hosted or self-hosted scanners, scan your critical systems for secrets, and notify the right teams to remediate and rotate.”
   • [claimed-docs] “Choose hosted (managed by Truffle Security) or self-hosted (running in your environment).”
   • [claimed-docs] “Route findings to Slack, Jira, email, or webhooks so the right people can act on them.”

   proof ↗⚑ flag

   Semgrep~partialclaimed5/10

   Semgrep supports automation via CI triggers on push/PR events, pre-commit hooks, and bulk 'Managed Scans' that run without manual invocation, and it has an AI Assistant/AI-Powered Detection layer for triage — providing real background automation. However, this is rule-based scan automation, not an autonomous agent that independently plans/executes multi-step actions; there's no evidence of scheduled/cron-like agentic orchestration beyond scan-on-event triggers. Missing for 10: evidence of autonomous multi-step agentic decision-making, scheduling/orchestration beyond CI-triggered scans, and independent confirmation that AI-driven remediation runs unattended.

   • [claimed-docs] “your development cycle benefits from the automated scanning of repositories at various events, such as: Push events; Pull requests or merge …”
   • [claimed-docs] “your development cycle benefits from the automated scanning of repositories at various events, such as: Push events, Pull requests or merge …”
   • [claimed-docs] “consider Semgrep Managed Scans, which enables you to bulk onboard and scan your repositories without requiring changes to your CI”
   • [claimed-docs] “The pre-commit framework can run semgrep when you commit changes. This is helpful in preventing secrets and security issues from leaking int…”
   • [claimed-docs] “The pre-commit framework can run `semgrep` when you commit changes. This is helpful in preventing secrets and security issues from leaking i…”
   • [claimed-docs] “With AI-Powered Detection, you can automatically identify complex business logic flaws, such as IDORs and broken authorization.”
   • [github] “Contextual post-processing of findings with Semgrep Assistant (AI) to further reduce noise by ~20%”
   • [claimed-docs] “Combine AI reasoning with rule-based analysis for detection, triage, and remediation”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   TruffleHog—none0/10

   TruffleHog is a CLI/scanning tool with docs and CLI examples, but there is no evidence of an interactive API reference (e.g., Swagger/OpenAPI explorer) with runnable examples; the openapi probe explicitly returned 404s for all candidate API spec paths and no interactive docs are mentioned.

   • [probe] “PROBE openapi: all candidate paths 404 (https://trufflesecurity.com/openapi.json, https://trufflesecurity.com/swagger.json, https://truffles…”
   • [probe] “PROBE llms.txt: HTTP 404 at https://trufflesecurity.com/llms.txt”

   proof ↗⚑ flag

   Semgrep—none0/10

   Semgrep documents an AppSec Platform API (list deployments, findings, projects) but there is no evidence of an interactive API reference with runnable/try-it examples; the openapi.json/swagger probes returned 404 on all candidate paths, indicating no interactive spec-based docs.

   • [claimed-docs] “Semgrep AppSec Platform provides an API that enables you to list deployments, gather findings created by Semgrep AppSec Platform, and list p…”
   • [claimed-docs] “Semgrep AppSec Platform provides an API that enables you to list deployments, gather findings created by Semgrep”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.semgrep.dev/openapi.json, https://docs.semgrep.dev/swagger.json, https://docs.semgrep.d…”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   TruffleHog—none0/10

   A direct probe for OpenAPI/swagger endpoints returned 404 for all candidate paths, and no evidence anywhere in the pack points to a downloadable machine-readable API spec.

   • [probe] “PROBE openapi: all candidate paths 404 (https://trufflesecurity.com/openapi.json, https://trufflesecurity.com/swagger.json, https://truffles…”

   proof ↗⚑ flag

   Semgrep—none0/10

   Semgrep documents a REST API for the AppSec Platform (semgrep-docs-17/59) but a direct probe for machine-readable OpenAPI/Swagger specs at standard paths returned 404 everywhere (semgrep-probe-2), and no evidence of a published API spec exists elsewhere in the pack.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.semgrep.dev/openapi.json, https://docs.semgrep.dev/swagger.json, https://docs.semgrep.d…”
   • [claimed-docs] “Semgrep AppSec Platform provides an API that enables you to list deployments, gather findings created by Semgrep AppSec Platform, and list p…”
   • [claimed-docs] “Semgrep AppSec Platform provides an API that enables you to list deployments, gather findings created by Semgrep”

   proof ↗⚑ flag

3. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   TruffleHog—none0/10

   No evidence of versioned APIs or a documented deprecation policy; probe results show no OpenAPI spec found (404s across all candidate paths), and no docs mention API versioning or deprecation practices.

   • [probe] “PROBE openapi: all candidate paths 404 (https://trufflesecurity.com/openapi.json, https://trufflesecurity.com/swagger.json, https://truffles…”

   proof ↗⚑ flag

   Semgrep—none0/10

   Semgrep documents an AppSec Platform API (semgrep-docs-17/59) but no evidence pack item mentions API versioning scheme or a documented deprecation policy; an OpenAPI spec probe returned 404s (semgrep-probe-2), suggesting no formal API contract is published. missing for 10: versioning scheme documentation, deprecation policy/notice process, changelog or migration guides for API changes.

   • [claimed-docs] “Semgrep AppSec Platform provides an API that enables you to list deployments, gather findings created by Semgrep AppSec Platform, and list p…”
   • [claimed-docs] “Semgrep AppSec Platform provides an API that enables you to list deployments, gather findings created by Semgrep”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.semgrep.dev/openapi.json, https://docs.semgrep.dev/swagger.json, https://docs.semgrep.d…”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Semgrep
   TruffleHog~partialclaimed5/10

   TruffleHog can scan across many sources at once (whole orgs, multiple repos, Docker images, Slack, Jira, Confluence, filesystems, etc.) and supports batch verification/classification of hundreds of secret types, which is a form of bulk operation. However, this is bulk *scanning*, not general-purpose bulk operations across arbitrary 'items' in an AI-native/automation sense (e.g., no evidence of bulk remediation, bulk API-driven item processing, or agent-orchestrated batch workflows beyond scanning). missing for 10: evidence of bulk actions beyond scanning (e.g., bulk remediation/rotation across many secrets programmatically), AI-agent-oriented batch orchestration, and independent confirmation of bulk-scale performance at scale.

   • [claimed-docs] “TruffleHog scans beyond code repositories to identify secrets hidden in comments, Docker images, and more.”
   • [claimed-docs] “TruffleHog can look for secrets in many places including git, chats, wikis, logs, api testing platforms, object stores, filesystems and more”
   • [github] “Are you interested in continuously monitoring Git, Jira, Slack, Confluence, Microsoft Teams, Sharepoint (and more) for credentials?”
   • [github] “continuously monitoring Git, Jira, Slack, Confluence, Microsoft Teams, Sharepoint (and more) for credentials”
   • [claimed-docs] “trufflehog classifies over 800 secret types, mapping them back to the specific identity they belong to”
   • [claimed-docs] “for every secret trufflehog can classify, it can also log in to confirm if that secret is live or not”

   proof ↗⚑ flag

   Semgrep~partialclaimed6/10

   Semgrep supports bulk-style operations such as scanning an entire codebase/monorepo in one command, cross-file/interfile analysis across many files, and 'Managed Scans' explicitly designed to 'bulk onboard and scan your repositories without requiring changes to your CI.' The AppSec Platform API also lets you programmatically list deployments/projects/findings across an organization. However, there is no documented bulk-action tooling (e.g., bulk triage, bulk rule application, batch API calls) beyond onboarding/scanning, so the story is only partially covered. Missing for 10: explicit bulk API endpoints for mass finding triage/remediation, evidence of batch operations beyond scanning/onboarding, and independent confirmation of large-scale bulk workflows in practice.

   • [claimed-docs] “consider Semgrep Managed Scans, which enables you to bulk onboard and scan your repositories without requiring changes to your CI”
   • [claimed-docs] “Learn how to set up Semgrep, scan your first project, which can be any codebase, repository, or folder within a monorepo, for security issue…”
   • [claimed-docs] “scan your first project, which can be any codebase, repository, or folder within a monorepo, for security issues, and view your findings”
   • [claimed-docs] “Semgrep AppSec Platform provides an API that enables you to list deployments, gather findings created by Semgrep AppSec Platform, and list p…”
   • [claimed-docs] “Semgrep AppSec Platform provides an API that enables you to list deployments, gather findings created by Semgrep”
   • [claimed-docs] “To turn on cross-file analysis, which allows you to detect vulnerabilities across files and folders”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to Semgrep
   TruffleHog~partialclaimed6/10

   TruffleHog supports event-driven automation via pre-commit/pre-receive git hooks that block commits/pushes containing secrets, custom detectors with webhook-based verification, and routing of findings to Slack/Jira/email/webhooks with continuous monitoring and alerting when remediation occurs. This is real rule-triggered action but it's scoped narrowly to secret-detection events rather than a general-purpose rule engine where arbitrary conditions map to arbitrary actions. Missing for 10: a documented general rules/policy engine with configurable conditions beyond hooks and webhooks, and independent hands-on confirmation that alerting/webhook routing works end-to-end in practice.

   • [claimed-docs] “Pre-commit hooks are a useful way to prevent secrets in code from being pushed from a git repository.”
   • [claimed-docs] “trufflehog git file://. --since-commit HEAD --results=verified,unknown --fail”
   • [claimed-docs] “This approach uses Git's `core.hooksPath` to apply hooks to all repositories without requiring any per-repository setup”
   • [claimed-docs] “If you run your own git server, consider the pre-receive hook option which can block commits with secrets from being accepted.”
   • [claimed-docs] “Route findings to Slack, Jira, email, or webhooks so the right people can act on them.”
   • [claimed-docs] “TruffleHog continuously tracks the status of all key types to identify whether remediation has occurred. Set up alerts across the platform o…”
   • [claimed-docs] “Verification is done via a webhook POST request to the provided `endpoint`.”
   • [claimed-docs] “The custom regex detector allows you to define your own detector using regular expressions with optional verification using a webhook.”

   proof ↗⚑ flag

   Semgrep✓fullclaimed8/10

   Semgrep rules can be configured to run automatically on events (push, PR/MR, pre-commit) and trigger downstream actions such as PR comments, alerts/notifications, ticket creation, and autofix application — all documented as first-class features (semgrep-docs-25/28/16/46/22/30/38/53/57). This directly matches the story of rules triggering automated actions on events. missing for 10: independent/hands-on evidence confirming these automated trigger-action pipelines (e.g., ticket creation, alerting) work reliably in production beyond vendor docs.

   • [claimed-docs] “your development cycle benefits from the automated scanning of repositories at various events, such as: Push events; Pull requests or merge …”
   • [claimed-docs] “your development cycle benefits from the automated scanning of repositories at various events, such as: Push events, Pull requests or merge …”
   • [claimed-docs] “The pre-commit framework can run semgrep when you commit changes. This is helpful in preventing secrets and security issues from leaking int…”
   • [claimed-docs] “The pre-commit framework can run `semgrep` when you commit changes. This is helpful in preventing secrets and security issues from leaking i…”
   • [claimed-docs] “Triage findings * Send alerts and notifications or create tickets to track findings identified by Semgrep Code”
   • [claimed-docs] “Triage findings Send alerts and notifications or create tickets to track findings identified by Semgrep Code”
   • [claimed-docs] “You can apply the Rule-defined fix directly to the file using the `--autofix` flag.”
   • [claimed-docs] “You can apply the Rule-defined fix directly to the file using the --autofix flag.”
   • [claimed-docs] “You can use Semgrep and its GitHub integration to automate PR comments that you frequently make in code reviews.”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round to Semgrep
   TruffleHog—none0/10

   TruffleHog's docs mention 'continuously monitoring' sources and 'continuously tracks the status of all key types' (trufflehog-docs-17, trufflehog-gh-8/9), but there is no evidence of an actual scheduler, cron integration, or workflow/job orchestration feature that lets a user configure recurring scans or automation pipelines. The CLI and docs focus on one-off scans, CI/pre-commit hook triggers, and hosted dashboards, not recurring job scheduling.

   • [claimed-docs] “TruffleHog continuously tracks the status of all key types to identify whether remediation has occurred. Set up alerts across the platform o…”
   • [github] “Are you interested in continuously monitoring Git, Jira, Slack, Confluence, Microsoft Teams, Sharepoint (and more) for credentials?”
   • [github] “continuously monitoring Git, Jira, Slack, Confluence, Microsoft Teams, Sharepoint (and more) for credentials”
   • [claimed-docs] “Deploy hosted or self-hosted scanners, scan your critical systems for secrets, and notify the right teams to remediate and rotate.”

   proof ↗⚑ flag

   Semgrep~partialclaimed3/10

   Semgrep supports automated scanning triggered by CI events (push, PR/MR) and offers Managed Scans for bulk repo scanning without CI changes, which implies some recurring automation, but there is no explicit documentation of a scheduling/cron mechanism or workflow orchestration for recurring jobs as an AI-native automation feature. missing for 10: explicit scheduling/cron support, documented recurring job configuration, and evidence of AI-native workflow orchestration beyond CI trigger events.

   • [claimed-docs] “consider Semgrep Managed Scans, which enables you to bulk onboard and scan your repositories without requiring changes to your CI”
   • [claimed-docs] “your development cycle benefits from the automated scanning of repositories at various events, such as: Push events; Pull requests or merge …”
   • [claimed-docs] “your development cycle benefits from the automated scanning of repositories at various events, such as: Push events, Pull requests or merge …”
   • [claimed-docs] “This guide walks you through creating a Semgrep job in the following CI providers, which are explicitly supported in Semgrep AppSec Platform”

   proof ↗⚑ flag

Agent scanning

1. ai-native userHave an agent run a scan headlessly via the CLI, parse machine-readable results, and file issues for real findings

   weight 3 · round to Semgrep
   TruffleHog~partialclaimed5/10

   TruffleHog's CLI clearly supports headless, scriptable scanning (e.g. `trufflehog git ... --results=verified,unknown --fail`, `--include-detectors`, `--verify-detectors`) and Docker/CI usage patterns are well documented, and Enterprise routing to Jira/Slack/webhooks is mentioned. However, there is no evidence of a structured/JSON machine-readable output flag, nor any documented mechanism for the CLI itself to file issues (e.g. GitHub issue creation) based on findings. Missing for 10: explicit machine-readable output format documentation, and evidence of automated issue-filing integration from CLI results.

   • [claimed-docs] “trufflehog git file://. --since-commit HEAD --results=verified,unknown --fail”
   • [claimed-docs] “this trufflehog invocation will force verification for AWS and Buildkite secrets, irrespective of whether the configured sources have their …”
   • [claimed-docs] “the `--verify-detectors` and `--no-verify-detectors` CLI flags can be used to configure detector-specific verification override settings”
   • [claimed-docs] “Deploy hosted or self-hosted scanners, scan your critical systems for secrets, and notify the right teams to remediate and rotate.”
   • [claimed-docs] “Route findings to Slack, Jira, email, or webhooks so the right people can act on them.”
   • [github] “docker run --rm -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --org=trufflesecurity”

   proof ↗⚑ flag

   Semgrep~partialcommunity7/10

   Semgrep's CLI clearly supports headless, unauthenticated scanning (`semgrep ci`/`semgrep scan`, no login required, --dry-run) and machine-readable output via SARIF export, which an agent could parse. Community evidence (semgrep-comm-14) shows this pattern used in practice — SARIF output plus LLM triage to surface real findings for human review at Microsoft. Docs also mention 'send alerts/notifications or create tickets to track findings' for Semgrep Code, but this is vague and tied to the platform tier rather than a demonstrated CLI-to-issue-tracker pipeline. Missing for 10: explicit documented JSON/SARIF-to-ticket automation workflow, first-party GitHub/Jira issue-filing integration for CLI findings, and independent hands-on confirmation that filed issues correspond only to 'real' (non-false-positive) findings given community complaints about false positives (semgrep-comm-11).

   • [claimed-docs] “navigate to the root of your project, and run your first scan: semgrep ci”
   • [claimed-docs] “semgrep scan - This is the recommended command for scanning local codebases or scanning a project when you don't have a Semgrep account”
   • [claimed-docs] “The `--dry-run` flag ensures that your scans are not uploaded to the Semgrep web app.”
   • [claimed-docs] “You do **not** need to be logged in to run a scan.”
   • [claimed-docs] “To export the results to a SARIF file:”
   • [claimed-docs] “To export the results to a SARIF file: semgrep ci --sarif --sarif-output=semgrep.sarif”
   • [claimed-docs] “Triage findings * Send alerts and notifications or create tickets to track findings identified by Semgrep Code”
   • [claimed-docs] “Triage findings Send alerts and notifications or create tickets to track findings identified by Semgrep Code”
   • [community] “Built a tool using Semgrep's SARIF output plus prompted LLMs to help prioritize triage; used at Microsoft for the past year to help surface …”

   proof ↗⚑ flag

2. ai-native userWire the scanner into my coding agent so AI-generated code is scanned for issues before it lands

   weight 2 · round to Semgrep
   TruffleHog~partialclaimed5/10

   TruffleHog ships well-documented pre-commit and pre-receive git hooks (trufflehog-docs-5, -13, -19, -20, -26, -30, -31) that scan code for secrets before it is committed, which can technically catch AI-generated code paths through standard git hooks or CI. However there is no evidence of any specific integration point for coding agents (e.g., an MCP server, IDE/agent plugin, or agent-specific pre-generation hook) — it's a generic pre-commit mechanism, not an AI-agent-aware wiring. Missing for 10: explicit coding-agent/AI-tool integration (MCP server, agent plugin, or IDE hook), evidence of scanning AI-generated code specifically, and independent confirmation the pre-commit flow works well with agent-driven commit workflows.

   • [claimed-docs] “Pre-commit hooks are a useful way to prevent secrets in code from being pushed from a git repository.”
   • [claimed-docs] “trufflehog git file://. --since-commit HEAD --results=verified,unknown --fail”
   • [claimed-docs] “This approach uses Git's `core.hooksPath` to apply hooks to all repositories without requiring any per-repository setup”
   • [claimed-docs] “Husky is a popular tool for managing Git hooks in JavaScript/Node.js projects.”
   • [claimed-docs] “In rare cases, you may need to bypass pre-commit hooks: `git commit --no-verify -m "Your commit message"`”
   • [claimed-docs] “If you run your own git server, consider the pre-receive hook option which can block commits with secrets from being accepted.”
   • [claimed-docs] “git commit --no-verify -m "Your commit message"”

   proof ↗⚑ flag

   Semgrep✓fullprobed7/10

   Semgrep ships an official MCP server (docs.semgrep.dev/mcp) letting coding agents call the scanner directly, plus first-party messaging on scanning/fixing AI-generated code the moment it's written and combining AI reasoning with rule-based detection, and a pre-commit hook to catch issues before they land in Git history. Missing for 10: independent/hands-on validation that the MCP integration reliably works inside real agent workflows, and detail on latency or false-positive behavior when used inline with AI code generation.

   • [probe] “official MCP server documented at https://docs.semgrep.dev/mcp”
   • [claimed-docs] “Scan and fix AI-generated code the moment it's written”
   • [claimed-docs] “Combine AI reasoning with rule-based analysis for detection, triage, and remediation”
   • [claimed-docs] “The pre-commit framework can run semgrep when you commit changes. This is helpful in preventing secrets and security issues from leaking int…”
   • [claimed-docs] “The pre-commit framework can run `semgrep` when you commit changes. This is helpful in preventing secrets and security issues from leaking i…”

   proof ↗⚑ flag

Ci gates

1. devops-leadRun scans in CI on every pull request and fail the build on new findings via exit codes

   weight 3 · round to Semgrep
   TruffleHog~partialclaimed6/10

   TruffleHog docs show the `--fail` flag (trufflehog-docs-13) which exits non-zero on findings, and Docker/CLI invocations suitable for CI pipelines (trufflehog-gh-2, trufflehog-gh-6), plus verified-only filtering to reduce noise. However, the evidence is framed around pre-commit/pre-receive hooks rather than an explicit CI-on-PR integration (e.g., GitHub Actions example, diffing against base branch for 'new findings only'), and there's no first-party CI pipeline template or independent confirmation of CI usage. missing for 10: explicit CI/PR pipeline example (e.g. GitHub Actions workflow), documentation of scanning only new commits in a PR diff context, and independent/hands-on confirmation of CI fail-on-new-finding behavior.

   • [claimed-docs] “trufflehog git file://. --since-commit HEAD --results=verified,unknown --fail”
   • [github] “docker run --rm -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --org=trufflesecurity”
   • [github] “docker run --rm -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --repo https://github.com/trufflesecurity/test_keys”
   • [claimed-docs] “Pre-commit hooks are a useful way to prevent secrets in code from being pushed from a git repository.”
   • [claimed-docs] “If you run your own git server, consider the pre-receive hook option which can block commits with secrets from being accepted.”

   proof ↗⚑ flag

   Semgrep✓fullcommunity9/10

   Docs explicitly cover running `semgrep ci` in CI pipelines triggered on push/PR events, with dedicated CI provider guides, SARIF export, and exit-code based build behavior implied by 'ci' command design for gating; community evidence corroborates real-world CI usage. missing for 10: explicit documented mention of exit code semantics for failing builds on new findings (implied but not directly quoted).

   • [claimed-docs] “navigate to the root of your project, and run your first scan: semgrep ci”
   • [claimed-docs] “This guide walks you through creating a Semgrep job in the following CI providers, which are explicitly supported in Semgrep AppSec Platform”
   • [claimed-docs] “your development cycle benefits from the automated scanning of repositories at various events, such as: Push events; Pull requests or merge …”
   • [claimed-docs] “Semgrep is integrated into CI environments by creating a **job** that is run by the CI provider. After a scan, findings are sent to Semgrep …”
   • [claimed-docs] “To export the results to a SARIF file: semgrep ci --sarif --sarif-output=semgrep.sarif”
   • [community] “We've used Semgrep for Zulip's Python codebase for months and really appreciate the semantic checks, especially for security lint rules - it…”

   proof ↗⚑ flag

2. developerSee scan findings surfaced as pull-request comments or checks in GitHub or GitLab

   weight 2 · round to Semgrep
   TruffleHog—none0/10

   Evidence covers pre-commit hooks, pre-receive hooks, CLI scanning, and routing findings to Slack/Jira/email/webhooks, but there is no mention of PR comments or status checks in GitHub or GitLab CI workflows.

   ⚑ flag

   Semgrep✓fullclaimed7/10

   Docs confirm Semgrep integrates into CI at push/PR/MR events, posts PR/MR comments for Secrets findings, and offers a GitHub integration for automating PR review comments, with findings sent to the AppSec Platform after scans. missing for 10: explicit first-party documentation of GitHub/GitLab PR status checks (not just comments) for Semgrep Code/Supply Chain findings, and independent hands-on confirmation of the PR-comment UX beyond docs.

   • [claimed-docs] “informs developers of valid secrets in their PRs and MRs by posting comments directly”
   • [claimed-docs] “your development cycle benefits from the automated scanning of repositories at various events, such as: Push events; Pull requests or merge …”
   • [claimed-docs] “your development cycle benefits from the automated scanning of repositories at various events, such as: Push events, Pull requests or merge …”
   • [claimed-docs] “Semgrep is integrated into CI environments by creating a **job** that is run by the CI provider. After a scan, findings are sent to Semgrep …”
   • [claimed-docs] “You can use Semgrep and its GitHub integration to automate PR comments that you frequently make in code reviews.”
   • [claimed-docs] “This guide walks you through creating a Semgrep job in the following CI providers, which are explicitly supported in Semgrep AppSec Platform”

   proof ↗⚑ flag

Pre commit

1. developerRun the scanner as a pre-commit hook so secrets and issues never enter my git history

   weight 3 · round drawn
   TruffleHog✓fullclaimed9/10

   TruffleHog has dedicated first-party documentation for pre-commit hooks, including a specific example invocation (`trufflehog git file://. --since-commit HEAD --results=verified,unknown --fail`), guidance on repo-wide setup via `core.hooksPath`, Husky integration, a pre-receive hook alternative for server-side enforcement, and even the bypass flag (`--no-verify`) — directly matching the story of preventing secrets from entering git history at commit time. Missing for 10: independent/community hands-on confirmation that the pre-commit workflow works smoothly in practice (community evidence is mostly about unrelated aspects like licensing and other scanners).

   • [claimed-docs] “Pre-commit hooks are a useful way to prevent secrets in code from being pushed from a git repository.”
   • [claimed-docs] “trufflehog git file://. --since-commit HEAD --results=verified,unknown --fail”
   • [claimed-docs] “This approach uses Git's `core.hooksPath` to apply hooks to all repositories without requiring any per-repository setup”
   • [claimed-docs] “Husky is a popular tool for managing Git hooks in JavaScript/Node.js projects.”
   • [claimed-docs] “In rare cases, you may need to bypass pre-commit hooks: `git commit --no-verify -m "Your commit message"`”
   • [claimed-docs] “If you run your own git server, consider the pre-receive hook option which can block commits with secrets from being accepted.”
   • [claimed-docs] “git commit --no-verify -m "Your commit message"”
   • [claimed-docs] “Use pre-commit and pre-receive hooks so that developers can prevent leaked keys in the first place.”

   proof ↗⚑ flag

   Semgrep✓fullclaimed9/10

   Semgrep has explicit first-party documentation for integrating with the pre-commit framework, stating it runs semgrep on commit specifically to prevent secrets and security issues from leaking into git history, matching the story verbatim. Missing for 10: no independent/hands-on community confirmation specifically of the pre-commit hook workflow (community evidence covers CLI/CI use generally, not pre-commit hook experience).

   • [claimed-docs] “The pre-commit framework can run semgrep when you commit changes. This is helpful in preventing secrets and security issues from leaking int…”
   • [claimed-docs] “The pre-commit framework can run `semgrep` when you commit changes. This is helpful in preventing secrets and security issues from leaking i…”
   • [claimed-docs] “The pre-commit framework can run semgr”

   proof ↗⚑ flag

Coverage

1. developerScan projects across all the major languages and package ecosystems my organization uses

   weight 2 · round to Semgrep
   TruffleHog~partialclaimed5/10

   TruffleHog scans raw content (git repos, filesystems, Docker images, chats, wikis, logs, object stores) rather than parsing by programming language, so it inherently ignores language/ecosystem boundaries for secret detection, and docs confirm broad source-type coverage beyond code repos. However there is no explicit documentation enumerating support across major languages or package-ecosystem-specific scanning (e.g., dependency manifests, language-specific SBOM/package files), so coverage-accuracy across ecosystems is implied rather than demonstrated. Missing for 10: explicit language/ecosystem coverage claims, package-manifest-aware scanning, and independent verification that all major ecosystems are handled equally well.

   • [claimed-docs] “TruffleHog scans beyond code repositories to identify secrets hidden in comments, Docker images, and more.”
   • [claimed-docs] “TruffleHog can look for secrets in many places including git, chats, wikis, logs, api testing platforms, object stores, filesystems and more”
   • [github] “Are you interested in continuously monitoring Git, Jira, Slack, Confluence, Microsoft Teams, Sharepoint (and more) for credentials?”
   • [github] “continuously monitoring Git, Jira, Slack, Confluence, Microsoft Teams, Sharepoint (and more) for credentials”

   proof ↗⚑ flag

   Semgrep~partialcommunity6/10

   Docs and community evidence show Semgrep scanning across many languages (Python, JS/TS, Ruby, YAML, HTML, CSS) and separately covering open-source package ecosystems via Semgrep Supply Chain (docs-11,12,13,23,29,44,55), plus a generic-pattern fallback for unsupported languages (docs-39). However there's no explicit doc listing 'all major languages/ecosystems' coverage, and a hands-on report shows Semgrep's parser failing on valid TypeScript syntax (comm-3), plus performance complaints scanning large multi-language codebases (comm-2, comm-7) that temper the breadth claim. missing for 10: an explicit enumerated language/ecosystem coverage matrix, resolution of the TypeScript parser bug, and independent benchmarking across the full claimed language set.

   • [claimed-docs] “If there's a code pattern in the codebase that matches the vulnerability definition, the finding is flagged as reachable.”
   • [claimed-docs] “Generate reports and software bills of materials (SBOM) that provide a complete inventory of your open source components”
   • [claimed-docs] “Supply Chain's Dynamic Dependency Resolution provides a complete inventory of your project's dependencies using a combination of manifest pa…”
   • [claimed-docs] “Detect malicious dependencies associated with Supply Chain attacks”
   • [claimed-docs] “Semgrep Supply Chain is a software composition analysis (SCA) tool that detects security vulnerabilities in your codebase introduced by open…”
   • [claimed-docs] “Semgrep can match generic patterns in languages that it does **not** yet support.”
   • [community] “Semgrep's parser complains that valid TypeScript using a typed catch clause (`catch (error: unknown)`) is invalid and can't be parsed.”
   • [community] “The underlying tree-sitter parser Semgrep uses is amazing - parsing millions of files across JS, TS, Ruby, YAML, HTML, CSS in about half a m…”
   • [community] “Running the CI ruleset on a tiny Django hobby project spun all cores to 100% after 33% progress and made the OS almost unresponsive; had to …”
   • [community] “We've used Semgrep for Zulip's Python codebase for months and really appreciate the semantic checks, especially for security lint rules - it…”

   proof ↗⚑ flag

Output formats

1. developerEmit scan results in machine-readable formats like JSON or SARIF for downstream tooling

   weight 2 · round to Semgrep
   TruffleHog—none0/10

   The evidence pack describes scanning capabilities, verification, and CLI flags extensively, but no citation mentions JSON, SARIF, or any specific machine-readable output format for downstream tooling integration. Absence of evidence for this applicable capability yields 'none'.

   ⚑ flag

   Semgrep✓fullcommunity9/10

   Docs explicitly document exporting SARIF output via `semgrep ci --sarif --sarif-output=semgrep.sarif`, and community evidence shows real-world use of Semgrep's SARIF output for downstream tooling (e.g., piping into LLM triage at Microsoft). JSON output is also a standard supported format implied by the CLI's general reporting capabilities, though not separately documented in this pack. missing for 10: explicit docs snippet for --json flag usage.

   • [claimed-docs] “To export the results to a SARIF file:”
   • [claimed-docs] “To export the results to a SARIF file: semgrep ci --sarif --sarif-output=semgrep.sarif”
   • [community] “Built a tool using Semgrep's SARIF output plus prompted LLMs to help prioritize triage; used at Microsoft for the past year to help surface …”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Semgrep
   TruffleHog—none0/10

   While TruffleHog ships a CLI and an Enterprise UI/platform, the evidence pack shows no documented API (OpenAPI/swagger endpoints all 404) that would let a user replicate UI actions programmatically; only CLI flags and webhook-based verification for custom detectors are documented, not a general platform API.

   • [probe] “PROBE openapi: all candidate paths 404 (https://trufflesecurity.com/openapi.json, https://trufflesecurity.com/swagger.json, https://truffles…”
   • [claimed-docs] “single sign on saml 2 0 or oauth 2 0 ❌ ✅ role based access control ❌ ✅”
   • [claimed-docs] “By the end, you'll have a running scanner, your first source connected, and findings flowing into the web UI.”

   proof ↗⚑ flag

   Semgrep~partialprobed4/10

   Semgrep AppSec Platform ships an API for listing deployments, projects and findings (semgrep-docs-17/59), and CLI/CI features (SARIF export, autofix, rule testing) give some parity with UI actions, but there is no evidence of a comprehensive OpenAPI spec (probe finds 404s at all openapi.json paths) or that triage actions, rule creation/management, policy configuration, or dashboard-only workflows are all exposed via API. missing for 10: documented OpenAPI/API reference covering full UI functionality, API endpoints for rule authoring/publishing and triage actions, independent confirmation that all UI capabilities are API-accessible.

   • [claimed-docs] “Semgrep AppSec Platform provides an API that enables you to list deployments, gather findings created by Semgrep AppSec Platform, and list p…”
   • [claimed-docs] “Semgrep AppSec Platform provides an API that enables you to list deployments, gather findings created by Semgrep”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.semgrep.dev/openapi.json, https://docs.semgrep.dev/swagger.json, https://docs.semgrep.d…”
   • [claimed-docs] “Triage findings * Send alerts and notifications or create tickets to track findings identified by Semgrep Code”
   • [claimed-docs] “Users with Semgrep Code's Team or Enterprise tier can publish rules to the Semgrep Registry as private rules that are not visible to those o…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round to Semgrep
   TruffleHog—none0/10

   The evidence covers scanning, detection, verification, and hosted/self-hosted deployment options, but there is no documentation of any data export feature, open export format, or migration/exit path for users of the hosted platform or CLI findings store.

   ⚑ flag

   Semgrep~partialclaimed5/10

   Semgrep supports open-format export of scan results (SARIF) and a platform API to gather findings/projects, plus local CLI scanning that never uploads code by default (dry-run, no login required), which supports data portability and avoiding lock-in. However there's no documented comprehensive 'export all your data' feature covering full platform history, configs, and org data in one open bundle. Missing for 10: explicit bulk/account-level data export documentation, confirmation of open formats beyond SARIF (e.g., JSON schema docs), and independent verification of a full data portability workflow.

   • [claimed-docs] “To export the results to a SARIF file:”
   • [claimed-docs] “To export the results to a SARIF file: semgrep ci --sarif --sarif-output=semgrep.sarif”
   • [claimed-docs] “The `--dry-run` flag ensures that your scans are not uploaded to the Semgrep web app.”
   • [claimed-docs] “You do **not** need to be logged in to run a scan.”
   • [claimed-docs] “Semgrep AppSec Platform provides an API that enables you to list deployments, gather findings created by Semgrep”
   • [github] “Semgrep analyzes code locally on your computer or in your build environment: by default, code is never upload”

   proof ↗⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round to TruffleHog
   TruffleHog✓fullprobed7/10

   TruffleHog's source is hosted publicly on GitHub (trufflehog-gh-1..8) and community discussion confirms it is licensed under AGPL (v3, shifted from GPL) — an OSI-recognized open license, satisfying the ability to read the source under an open license. Missing for 10: no direct citation of a LICENSE file or official docs statement confirming license terms, and no independent corroboration beyond the HN comment.

   • [community] “Note that v3 shifts to AGPL, from GPL in prior versions. Doesn't bother me personally, but I know a lot of companies won't touch AGPL'd proj…”
   • [github] “brew install trufflehog”
   • [probe] “official CLI documented at https://github.com/trufflesecurity/trufflehog”

   proof ↗⚑ flag

   Semgrep~partialcommunity5/10

   The core Semgrep engine is hosted on a public GitHub repo (semgrep-gh-1/2/3), showing source code is publicly readable, but no evidence pack item states an explicit open-source license (e.g., LGPL) or points to a LICENSE file. Community evidence also notes that a commercial component, Semgrep Supply Chain, is explicitly 'not open source' (semgrep-comm-12), meaning the broader product is not uniformly open-licensed. missing for 10: explicit license text/citation, confirmation covering all product components, independent verification of license terms.

   • [github] “Semgrep would match x = 1; y = x + 1 when searching for 2”
   • [github] “Semgrep analyzes code locally on your computer or in your build environment: by default, code is never upload”
   • [github] “Contextual post-processing of findings with Semgrep Assistant (AI) to further reduce noise by ~20%”
   • [community] “Regarding Semgrep Supply Chain: "This is not open source, though? It does make a big difference for some whether you're able to run the chec…”

   proof ↗⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round to TruffleHog
   TruffleHog✓fullclaimed8/10

   TruffleHog is open source (CLI installable via brew/Docker/GitHub) and docs explicitly offer a self-hosted deployment option distinct from the hosted/managed offering, with getting-started guidance for running your own scanner. Missing for 10: detailed self-hosting infrastructure/deployment docs (e.g., Kubernetes/Helm charts, scaling guidance) and independent hands-on confirmation of self-hosting the full platform (vs just the OSS CLI).

   • [claimed-docs] “Deploy hosted or self-hosted scanners, scan your critical systems for secrets, and notify the right teams to remediate and rotate.”
   • [claimed-docs] “Choose hosted (managed by Truffle Security) or self-hosted (running in your environment).”
   • [claimed-docs] “By the end, you'll have a running scanner, your first source connected, and findings flowing into the web UI.”
   • [github] “brew install trufflehog”
   • [github] “docker run --rm -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --org=trufflesecurity”

   proof ↗⚑ flag

   Semgrep✓fullclaimed7/10

   Semgrep's core CLI/engine is open source and runs entirely on local infrastructure by default, never uploading code, and works without an account or login, which is effectively self-hosting the core scanning product (semgrep-gh-2, semgrep-docs-33/34/50/51/27). Missing for 10: explicit documentation of self-hosting the full AppSec Platform/server components, and independent hands-on confirmation of a self-hosted enterprise deployment.

   • [github] “Semgrep analyzes code locally on your computer or in your build environment: by default, code is never upload”
   • [claimed-docs] “The `--dry-run` flag ensures that your scans are not uploaded to the Semgrep web app.”
   • [claimed-docs] “You do **not** need to be logged in to run a scan.”
   • [claimed-docs] “You can run local Semgrep CLI scans with the Semgrep command-line tool.”
   • [claimed-docs] “The --dry-run flag ensures that your scans are not uploaded to the Semgrep web app.”
   • [claimed-docs] “semgrep scan - This is the recommended command for scanning local codebases or scanning a project when you don't have a Semgrep account”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round to TruffleHog
   TruffleHog~partialclaimed4/10

   TruffleHog offers a self-hosted deployment option ("Choose hosted (managed by Truffle Security) or self-hosted (running in your environment)"), which lets an org keep all scan data within its own infrastructure/region, giving indirect control over data residency. However, there is no explicit region-selection feature or documented multi-region hosting for the managed/enterprise offering. Missing for 10: explicit region/residency selection controls for the hosted service, and any documentation addressing data-locality guarantees.

   • [claimed-docs] “Choose hosted (managed by Truffle Security) or self-hosted (running in your environment).”
   • [claimed-docs] “Deploy hosted or self-hosted scanners, scan your critical systems for secrets, and notify the right teams to remediate and rotate.”

   proof ↗⚑ flag

   Semgrep—none0/10

   The evidence discusses local scanning (data never uploaded by default) and CLI dry-run options, but there is no mention of data residency, regional hosting options, or ability to choose storage location for the Semgrep AppSec Platform's cloud data. missing for 10: explicit region/residency selection controls, documentation of where cloud-stored findings/data reside, any enterprise data-residency options.

   • [github] “Semgrep analyzes code locally on your computer or in your build environment: by default, code is never upload”
   • [claimed-docs] “The `--dry-run` flag ensures that your scans are not uploaded to the Semgrep web app.”

   proof ↗⚑ flag

2. ai-native userControl data retention and deletion

   weight 2 · round to Semgrep
   TruffleHog—none0/10

   TruffleHog is a secrets-scanning tool (self-hosted or hosted scanner); the evidence pack shows scanning, detection, verification, and alerting capabilities but contains no mention of data retention policies, deletion controls, or user-facing settings for managing stored scan data/findings lifecycle. Since TruffleHog does offer a hosted enterprise option (docs-28), retention/deletion controls are a fair question but no evidence confirms them.

   ⚑ flag

   Semgrep~partialclaimed4/10

   Docs show privacy-preserving defaults (local-only scanning, --dry-run to avoid upload, no login required), giving users some control over whether data leaves their machine, but there is no evidence of explicit data retention periods, deletion mechanisms, or account/data-erasure controls for data that is uploaded to the Semgrep AppSec Platform. missing for 10: documented retention policy, explicit deletion/erasure controls for platform-stored findings, GDPR/CCPA data handling commitments.

   • [github] “Semgrep analyzes code locally on your computer or in your build environment: by default, code is never upload”
   • [claimed-docs] “The `--dry-run` flag ensures that your scans are not uploaded to the Semgrep web app.”
   • [claimed-docs] “The --dry-run flag ensures that your scans are not uploaded to the Semgrep web app.”
   • [claimed-docs] “You do **not** need to be logged in to run a scan.”

   proof ↗⚑ flag

3. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round to Semgrep
   TruffleHog—none0/10

   No evidence pack item mentions telemetry, usage tracking, or any opt-out mechanism/flag for TruffleHog; the docs cover secret scanning features but are silent on data collection practices.

   ⚑ flag

   Semgrep~partialclaimed5/10

   Semgrep's docs and GitHub README indicate privacy-conscious defaults: code is analyzed locally and never uploaded by default, no login is required to run a scan, and the --dry-run flag explicitly prevents scan results from being uploaded to the Semgrep web app. However, there's no explicit documentation of a dedicated 'telemetry/usage tracking' opt-out setting (e.g., an env var or flag specifically for metrics/telemetry) beyond the scan-upload behavior. missing for 10: explicit telemetry/usage-metrics opt-out flag or setting, independent confirmation of what usage data (if any) is collected outside of code content.

   • [github] “Semgrep analyzes code locally on your computer or in your build environment: by default, code is never upload”
   • [claimed-docs] “The `--dry-run` flag ensures that your scans are not uploaded to the Semgrep web app.”
   • [claimed-docs] “You do **not** need to be logged in to run a scan.”
   • [claimed-docs] “The --dry-run flag ensures that your scans are not uploaded to the Semgrep web app.”

   proof ↗⚑ flag

Custom rules

1. security-engineerWrite custom static-analysis rules in a simple syntax and share them across repositories

   weight 2 · round to Semgrep
   TruffleHog~partialclaimed5/10

   TruffleHog supports 'custom regex detectors' defined in a config file with optional webhook verification, which is a form of custom rule authoring in a simple (regex/YAML) syntax, and these config files can in principle be reused across scans/repos via `--config=config.yaml`. However, this is scoped narrowly to secret-detection regex patterns rather than general static-analysis rules, and there is no documented feature for centrally sharing/distributing these custom detector configs across repositories (e.g. a rule registry or team-wide rule sync). Missing for 10: broader SAST rule language beyond regex, explicit cross-repo rule-sharing/distribution mechanism, and independent evidence of teams using this for rule sharing.

   • [claimed-docs] “The custom regex detector allows you to define your own detector using regular expressions with optional verification using a webhook.”
   • [claimed-docs] “Verification is done via a webhook POST request to the provided `endpoint`.”
   • [claimed-docs] “./trufflehog scan --config=config.yaml --include-detectors=AWS,GitHub”
   • [claimed-docs] “You can manually specify which detectors to use with the `--include-detectors` flag.”

   proof ↗⚑ flag

   Semgrep✓fullcommunity8/10

   Semgrep's docs extensively cover writing custom rules with a simple YAML-based pattern syntax, testing them, using taint tracking, and sharing rules across repos via the Registry and private rules for teams; community evidence (HN) confirms security engineers actually write custom rules to catch codebase-specific antipatterns and find the syntax readable/flexible across languages. Missing for 10: independent evidence specifically validating cross-repo sharing workflows at scale, and community reports note real limitations (bypassable rules, parser errors, performance issues) that slightly temper confidence in robustness.

   • [claimed-docs] “you can write custom rules to determine what Semgrep detects in your repositories”
   • [claimed-docs] “you can write and test using the [Semgrep Editor]”
   • [claimed-docs] “you can write rules that: Automate code review comments. Identify secure coding violations. Scan configuration files.”
   • [claimed-docs] “Users with Semgrep Code's Team or Enterprise tier can publish rules to the Semgrep Registry as private rules that are not visible to those o…”
   • [claimed-docs] “Semgrep provides a testing mechanism for your rules. You can write code and provide annotations to let Semgrep know where you are or aren't …”
   • [claimed-docs] “Taint tracking (known also as taint analysis) enables you to write simple rules that catch complex injection bugs, such as those that can re…”
   • [claimed-docs] “Semgrep supports taint analysis, also known as taint tracking, through taint rules.”
   • [community] “As a security consultant, I write custom Semgrep rules to match antipatterns unique to a codebase (e.g., controllers missing an authorizatio…”
   • [community] “A simple Semgrep 'no-prints' rule can be trivially bypassed with a workaround like `import builtins; builtins.print("whee")`, showing the to…”

   proof ↗⚑ flag

Fix suggestions

1. developerApply suggested or automatic code fixes for findings instead of only reading reports

   weight 2 · round to Semgrep
   TruffleHog—none0/10

   TruffleHog is a secrets-detection/scanning tool that finds, verifies, and reports on exposed secrets, and notifies teams to rotate/remediate them manually; there is no evidence of any auto-fix, patch-suggestion, or code-remediation capability—only alerting, tracking remediation status, and pre-commit/pre-receive blocking.

   • [claimed-docs] “TruffleHog continuously tracks the status of all key types to identify whether remediation has occurred. Set up alerts across the platform o…”
   • [claimed-docs] “TruffleHog continuously tracks the status of all key types to identify whether remediation has occurred. Set up alerts across the platform o…”
   • [claimed-docs] “Pre-commit hooks are a useful way to prevent secrets in code from being pushed from a git repository.”
   • [claimed-docs] “If you run your own git server, consider the pre-receive hook option which can block commits with secrets from being accepted.”

   proof ↗⚑ flag

   Semgrep~partialclaimed6/10

   Semgrep supports rule-defined autofix via the --autofix flag, letting developers apply suggested fixes directly to files (semgrep-docs-38, semgrep-docs-53), and AppSec Platform mentions 'remediation' and 'fix AI-generated code' (semgrep-docs-45, semgrep-docs-47). However, autofix depends on individual rules defining a fix and is not universal across all findings, and there's no independent/community evidence confirming real-world reliability of autofix. missing for 10: community/hands-on validation of autofix accuracy, evidence of coverage breadth (what % of rules ship with fixes), and any first-party numbers on fix acceptance rates.

   • [claimed-docs] “You can apply the Rule-defined fix directly to the file using the `--autofix` flag.”
   • [claimed-docs] “You can apply the Rule-defined fix directly to the file using the --autofix flag.”
   • [claimed-docs] “Semgrep is integrated into CI environments by creating a **job** that is run by the CI provider. After a scan, findings are sent to Semgrep …”
   • [claimed-docs] “Scan and fix AI-generated code the moment it's written”
   • [claimed-docs] “Combine AI reasoning with rule-based analysis for detection, triage, and remediation”

   proof ↗⚑ flag

Custom detection

1. security-engineerDefine custom detection rules for my organization's internal token and secret formats

   weight 2 · round to TruffleHog
   TruffleHog✓fullclaimed9/10

   TruffleHog explicitly supports custom regex detectors for defining organization-specific secret formats, with optional webhook-based verification (trufflehog-docs-6, trufflehog-docs-27), plus CLI flags to include/exclude detectors and control verification (trufflehog-docs-7, trufflehog-docs-14, trufflehog-docs-21). This is documented first-party capability directly matching the story. Missing for 10: independent/community corroboration specifically of the custom-detector feature working in practice (community evidence in the pack discusses licensing/comparisons, not custom rules).

   • [claimed-docs] “The custom regex detector allows you to define your own detector using regular expressions with optional verification using a webhook.”
   • [claimed-docs] “Verification is done via a webhook POST request to the provided `endpoint`.”
   • [claimed-docs] “You can manually specify which detectors to use with the `--include-detectors` flag.”
   • [claimed-docs] “./trufflehog scan --config=config.yaml --include-detectors=AWS,GitHub”
   • [claimed-docs] “the `--verify-detectors` and `--no-verify-detectors` CLI flags can be used to configure detector-specific verification override settings”

   proof ↗⚑ flag

   Semgrep~partialcommunity6/10

   Semgrep's generic rule-writing engine is well documented (custom rules, pattern matching, Editor for testing) and community evidence confirms security engineers write custom rules for org-specific patterns, which extends naturally to matching internal token/secret formats via pattern or regex-based rules. However, there is no explicit documentation showing how to author custom rules specifically within the Semgrep Secrets product (e.g., custom secret validators or format definitions) — the evidence only shows built-in secret detection (docs-8/9/10) and generic custom rule capability (docs-4/5/21), not a documented workflow for adding custom secret-format rules to Semgrep Secrets. Missing for 10: explicit docs/examples for authoring custom Semgrep Secrets rule types (validators, custom regex-based secret detectors), and independent confirmation of doing so for secrets specifically.

   • [claimed-docs] “you can write custom rules to determine what Semgrep detects in your repositories”
   • [claimed-docs] “you can write and test using the [Semgrep Editor]”
   • [claimed-docs] “you can write rules that: Automate code review comments. Identify secure coding violations. Scan configuration files.”
   • [claimed-docs] “Semgrep Secrets scans code to detect exposed API keys, passwords, and other credentials.”
   • [claimed-docs] “valid secrets are those that have been tested against a web service and confirmed to successfully grant resources or authentication. They ar…”
   • [community] “As a security consultant, I write custom Semgrep rules to match antipatterns unique to a codebase (e.g., controllers missing an authorizatio…”

   proof ↗⚑ flag

Secret scanning

1. developerScan a repository and its full git history for hardcoded secrets and credentials

   weight 3 · round to TruffleHog
   TruffleHog✓fullclaimed9/10

   TruffleHog's core documented functionality is scanning git repos (and full history via commands like `trufflehog github --repo ...`) for hardcoded secrets, with 800+ detectors and live verification, corroborated by both official docs and GitHub README. Community commentary criticizes maturity/hype but does not contradict the core git-history scanning capability. Missing for 10: independent hands-on benchmark of full-history scan accuracy/performance beyond vendor docs.

   • [claimed-docs] “trufflehog github --only-verified --repo https://github.com/trufflesecurity/test_keys”
   • [github] “docker run --rm -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --repo https://github.com/trufflesecurity/test_keys”
   • [claimed-docs] “TruffleHog™ is a secrets scanning tool that digs deep into your code repositories to find secrets, passwords, and sensitive keys.”
   • [claimed-docs] “TruffleHog classifies over 800 secret types, mapping them back to the specific identity they belong to.”
   • [github] “For every secret TruffleHog can classify, it can also log in to confirm if that secret is live or not.”
   • [claimed-docs] “trufflehog git file://. --since-commit HEAD --results=verified,unknown --fail”

   proof ↗⚑ flag

   Semgrep~partialclaimed4/10

   Semgrep Secrets is documented to scan code for exposed API keys, passwords, and credentials and validate them (semgrep-docs-8, semgrep-docs-9, semgrep-docs-43), and pre-commit hooks aim to prevent secrets from entering git history (semgrep-docs-16/46), but this only prevents future leaks — none of the evidence shows Semgrep actually scanning the full historical git commit log/diffs for previously committed secrets (a capability tools like gitleaks/trufflehog specialize in). missing for 10: explicit documentation or feature for scanning past git commits/history (not just current working tree), evidence of retroactive history-wide secret detection, independent confirmation this works in practice.

   • [claimed-docs] “Semgrep Secrets scans code to detect exposed API keys, passwords, and other credentials.”
   • [claimed-docs] “valid secrets are those that have been tested against a web service and confirmed to successfully grant resources or authentication. They ar…”
   • [claimed-docs] “Semgrep saves security engineers time and effort by prioritizing valid leaked secrets and informs developer”
   • [claimed-docs] “The pre-commit framework can run semgrep when you commit changes. This is helpful in preventing secrets and security issues from leaking int…”
   • [claimed-docs] “The pre-commit framework can run `semgrep` when you commit changes. This is helpful in preventing secrets and security issues from leaking i…”

   proof ↗⚑ flag

2. security-engineerScan non-git sources like S3 buckets, Docker images, and CI logs for leaked secrets

   weight 2 · round to TruffleHog
   TruffleHog!disputedcontradicted4/10

   TruffleHog's docs claim broad non-git scanning coverage including Docker images, object stores, filesystems, and logs (trufflehog-docs-12, trufflehog-docs-18), but there is no first-party evidence naming S3 or CI logs specifically, and independent community evidence directly contradicts the Docker-image claim: a user reported they 'couldn't find anything comparable to Trufflehog for Docker images' and built their own tool instead (trufflehog-comm-6). Missing for 10: explicit named support/examples for S3 buckets and CI log scanning, and independent corroboration that Docker image scanning actually works as documented.

   • [claimed-docs] “TruffleHog scans beyond code repositories to identify secrets hidden in comments, Docker images, and more.”
   • [claimed-docs] “TruffleHog can look for secrets in many places including git, chats, wikis, logs, api testing platforms, object stores, filesystems and more”
   • [community] “I couldn't find anything comparable to Trufflehog for Docker images, even though I have constantly read articles about 'secrets discovered i…”

   proof ↗⚑ flag

   Semgrep—none0/10

   Semgrep Secrets is documented to scan code/repositories for exposed credentials, but nothing in the evidence pack shows support for scanning non-git sources like S3 buckets, Docker images, or CI logs — all scanning references are to codebases, folders, monorepos, or CI pipelines running against git-based projects.

   • [claimed-docs] “Semgrep Secrets scans code to detect exposed API keys, passwords, and other credentials.”
   • [claimed-docs] “valid secrets are those that have been tested against a web service and confirmed to successfully grant resources or authentication. They ar…”
   • [claimed-docs] “Semgrep saves security engineers time and effort by prioritizing valid leaked secrets and informs developer”
   • [claimed-docs] “Semgrep Secrets scans code to detect exposed API ke”

   proof ↗⚑ flag

3. security-engineerHave detected secrets verified against the issuing service so I know which ones are live

   weight 2 · round to TruffleHog
   TruffleHog✓fullclaimed9/10

   TruffleHog's core credential verification feature logs into the issuing service to confirm if a secret is live, with CLI flags (--only-verified, --verify-detectors) and docs/README consistently describing this capability across hundreds of detectors. missing for 10: independent hands-on benchmark confirming verification accuracy/false-positive rates at scale.

   • [claimed-docs] “trufflehog github --only-verified --repo https://github.com/trufflesecurity/test_keys”
   • [claimed-docs] “for every secret trufflehog can classify, it can also log in to confirm if that secret is live or not”
   • [github] “For every secret TruffleHog can classify, it can also log in to confirm if that secret is live or not.”
   • [claimed-docs] “the `--verify-detectors` and `--no-verify-detectors` CLI flags can be used to configure detector-specific verification override settings”
   • [claimed-docs] “this trufflehog invocation will force verification for AWS and Buildkite secrets, irrespective of whether the configured sources have their …”
   • [claimed-docs] “TruffleHog classifies over 800 secret types, mapping them back to the specific identity they belong to.”

   proof ↗⚑ flag

   Semgrep✓fullclaimed8/10

   Semgrep Secrets explicitly performs validity checks by testing detected secrets against the issuing web service, distinguishing 'valid' (live, actively in use) secrets from invalid ones, and surfaces this in PR/MR comments to prioritize live secrets. Missing for 10: independent/hands-on corroboration of validation accuracy and broader coverage details across secret types/providers.

   • [claimed-docs] “valid secrets are those that have been tested against a web service and confirmed to successfully grant resources or authentication. They ar…”
   • [claimed-docs] “Semgrep Secrets scans code to detect exposed API keys, passwords, and other credentials.”
   • [claimed-docs] “informs developers of valid secrets in their PRs and MRs by posting comments directly”
   • [claimed-docs] “Semgrep saves security engineers time and effort by prioritizing valid leaked secrets and informs developer”

   proof ↗⚑ flag

Ai triage

1. ai-native userUse AI to triage findings — explaining them, deduplicating, and flagging likely false positives

   weight 3 · round to Semgrep
   TruffleHog—none0/10

   TruffleHog's evidence shows credential verification, classification, and permission analysis (analyzer) to reduce false positives, but none of this involves AI/LLM-based explanation, deduplication, or triage of findings — it's rule-based verification, not AI reasoning. No mention of AI, LLM, or natural-language explanation features appears anywhere in the pack.

   • [claimed-docs] “for every secret trufflehog can classify, it can also log in to confirm if that secret is live or not”
   • [claimed-docs] “TruffleHog Analyze evaluates access patterns to identify a secret’s metadata, permissions, and resources.”
   • [github] “instead of sending one request to check if the secret can log in, TruffleHog can send many requests to learn everything there is to know abo…”
   • [claimed-docs] “TruffleHog classifies over 800 secret types, mapping them back to the specific identity they belong to.”

   proof ↗⚑ flag

   Semgrep~partialcommunity6/10

   Semgrep ships a native AI feature (Semgrep Assistant) explicitly described as 'contextual post-processing of findings...to further reduce noise by ~20%', and marketing docs claim combining 'AI reasoning with rule-based analysis for detection, triage, and remediation,' plus AI-powered detection for complex logic flaws — directly supporting AI-assisted false-positive reduction and triage. However, deduplication specifically is never mentioned, and the strongest independent evidence of AI triage in practice comes from a third-party tool built on Semgrep's SARIF output plus external LLMs (used at Microsoft) rather than the native Assistant, suggesting the first-party AI-triage feature is thin on independent corroboration. Missing for 10: explicit deduplication capability, deeper documentation of Assistant's explanation/false-positive-flagging behavior, independent hands-on evidence validating the native AI triage feature itself.

   • [github] “Contextual post-processing of findings with Semgrep Assistant (AI) to further reduce noise by ~20%”
   • [claimed-docs] “With AI-Powered Detection, you can automatically identify complex business logic flaws, such as IDORs and broken authorization.”
   • [claimed-docs] “Combine AI reasoning with rule-based analysis for detection, triage, and remediation”
   • [community] “Built a tool using Semgrep's SARIF output plus prompted LLMs to help prioritize triage; used at Microsoft for the past year to help surface …”

   proof ↗⚑ flag

Fp management

1. security-engineerBaseline or ignore existing and false-positive findings so only new issues alert

   weight 3 · round to TruffleHog
   TruffleHog~partialclaimed5/10

   TruffleHog supports scanning only new commits via `--since-commit` (used in pre-commit hooks) and reduces false positives via `--only-verified`/verification flags, which can approximate 'ignore existing/false positives, alert only new'. However there is no documented allowlist, ignore-file, or finding-suppression mechanism for marking specific findings as accepted/false-positive across full scans. missing for 10: explicit baseline/ignore-list feature, per-finding suppression or allowlisting, independent confirmation that verified-only filtering meaningfully reduces false positives in practice.

   • [claimed-docs] “trufflehog git file://. --since-commit HEAD --results=verified,unknown --fail”
   • [claimed-docs] “this trufflehog invocation will force verification for AWS and Buildkite secrets, irrespective of whether the configured sources have their …”
   • [claimed-docs] “the `--verify-detectors` and `--no-verify-detectors` CLI flags can be used to configure detector-specific verification override settings”
   • [claimed-docs] “trufflehog github --only-verified --repo https://github.com/trufflesecurity/test_keys”

   proof ↗⚑ flag

   Semgrep~partialclaimed4/10

   Docs mention a 'Triage findings' capability (alerts/tickets for tracking findings) and Semgrep Assistant's contextual post-processing to reduce noise by ~20%, implying some findings-management workflow, but there is no explicit evidence of baseline scanning (e.g., baseline-commit diffing), ignore-comments (nosemgrep), or a documented false-positive suppression/triage-state workflow. missing for 10: explicit baseline-commit/diff scanning documentation, .semgrepignore or inline nosemgrep suppression evidence, and a documented 'mark as false positive / ignored' triage state in the platform.

   • [claimed-docs] “Triage findings * Send alerts and notifications or create tickets to track findings identified by Semgrep Code”
   • [claimed-docs] “Triage findings Send alerts and notifications or create tickets to track findings identified by Semgrep Code”
   • [github] “Contextual post-processing of findings with Semgrep Assistant (AI) to further reduce noise by ~20%”

   proof ↗⚑ flag

2. developerFilter and gate on findings by severity or confidence so low-value noise does not block builds

   weight 2 · round to TruffleHog
   TruffleHog~partialclaimed6/10

   TruffleHog supports filtering/gating via verification status rather than explicit severity/confidence scores — flags like --only-verified, --results=verified,unknown --fail, --include-detectors, and --verify-detectors/--no-verify-detectors let developers restrict findings and control build-blocking behavior (e.g., in pre-commit hooks). This addresses the triage-noise goal but doesn't map to a formal severity/confidence scale as the story implies. Missing for 10: explicit severity levels or confidence scoring, independent/hands-on confirmation of noise reduction in CI gating.

   • [claimed-docs] “trufflehog github --only-verified --repo https://github.com/trufflesecurity/test_keys”
   • [claimed-docs] “trufflehog git file://. --since-commit HEAD --results=verified,unknown --fail”
   • [claimed-docs] “the `--verify-detectors` and `--no-verify-detectors` CLI flags can be used to configure detector-specific verification override settings”
   • [claimed-docs] “You can manually specify which detectors to use with the `--include-detectors` flag.”
   • [claimed-docs] “this trufflehog invocation will force verification for AWS and Buildkite secrets, irrespective of whether the configured sources have their …”

   proof ↗⚑ flag

   Semgrep—none0/10

   The evidence pack describes general triage/noise features (AI post-processing to reduce noise, ticketing/alerts, reachability analysis) but never mentions severity levels, confidence scores, or a mechanism to filter/gate CI builds based on them. Missing for 10: explicit severity/confidence filtering flags (e.g., --severity), CI gating configuration based on severity/confidence thresholds, and documentation tying these to build pass/fail behavior.

   ⚑ flag