Snyk vs Gecko Security

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to Gecko Security
   Snyk✓fullprobed8/10

   Snyk publishes an llms.txt file confirmed live via direct probe (HTTP 200) with structured docs navigation, and also maintains extensive agent-oriented documentation (MCP server setup, Snyk Studio, remediation agent) that an AI agent could be pointed at. missing for 10: no independent/community corroboration of agents actually consuming llms.txt in practice, and no evidence of additional agent-friendly formats (e.g., structured API schemas, since openapi.json probes 404).

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.snyk.io/llms.txt # Snyk User Docs ## Discover Snyk - [Homepage](https://docs.snyk.io/readme.md): …”
   • [claimed-docs] “Snyk Studio supports a default hooks-based approach for some agentic development environments (ADEs) to ensure code written by AI coding ass…”
   • [claimed-docs] “The Snyk MCP Server is designed as a local MCP server, running on your system using the Snyk CLI to ensure local file access.”
   • [probe] “official MCP server documented at https://docs.snyk.io/agent-security/agentic-security-with-snyk-studio/getting-started-with-snyk-studio”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.snyk.io/openapi.json, https://docs.snyk.io/swagger.json, https://docs.snyk.io/api/opena…”

   proof ↗⚑ flag

   Gecko Security✓fullprobed9/10

   Direct probes confirm Gecko serves both llms.txt (HTTP 200) and a docs.md aggregate at the docs root, plus an OpenAPI spec and documented MCP/AI-tool integration for agents to consume — exactly the agent-oriented docs surface the story asks for. Missing for 10: no independent/community confirmation that an agent actually consumed llms.txt successfully in practice.

   • [probe] “PROBE llms.txt: HTTP 200 at https://gecko.security/llms.txt # Gecko Security > Gecko Security is an AI security engineer that finds exploit…”
   • [probe] “PROBE docs-md: HTTP 200 at https://gecko.security/docs.md --- title: "Gecko Security - Gecko Security" description: "An AI security engineer…”
   • [probe] “PROBE openapi: HTTP 200 at https://gecko.security/openapi.json — contains "openapi" key”
   • [claimed-docs] “Connect it to an AI tool and the tool can list your repositories and scans, pull full vulnerability evidence (proof of concept, call chain, …”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to Snyk
   Snyk✓fullclaimed8/10

   Snyk CLI explicitly supports running in CI/CD pipelines (snyk-gh-3, snyk-gh-7) with test/code test commands producing terminal reports (snyk-gh-4, snyk-gh-5, snyk-gh-6), and Snyk Code/Open Source docs confirm scanning integrates across CI/CD pipelines (snyk-docs-18, snyk-docs-26, snyk-docs-33). missing for 10: no independent/community corroboration of CI usage specifically (only vendor docs/GitHub), and no explicit example of a CI YAML config or automation-friendly non-interactive flags/output formats (e.g., JSON output for pipelines) in the evidence.

   • [github] “You can run the CLI locally from the command line or in an IDE. You can also run the CLI in your CI/CD pipeline.”
   • [github] “The Snyk CLI brings the functionality of Snyk into your development workflow. You can run the CLI locally from the command line or in an IDE…”
   • [github] “Look at the `test` command report in your terminal. The report shows the vulnerabilities Snyk found in the package.”
   • [github] “To scan your source code, run `snyk code test`.”
   • [github] “The report shows the vulnerabilities Snyk found in the package. For each issue found, Snyk reports the severity of the issue, provides a lin…”
   • [claimed-docs] “Snyk Code is a developer-first static application security testing (SAST) solution. By scanning code in real-time and providing actionable i…”
   • [claimed-docs] “By scanning code in real-time and providing actionable insights directly in the developer workflow across IDEs, repositories, and CI/CD pipe…”
   • [claimed-docs] “Snyk Code is a developer-first static application security testing (SAST) solution. By scanning code in real-time and providing actionable i…”

   proof ↗⚑ flag

   Gecko Security✓fullprobed7/10

   Gecko exposes a v1 API to trigger scans and read/triage results, dedicated CI/CD-labeled API keys, PR-check integration with configurable fail-on-severity thresholds that post pass/fail status back to GitHub/GitLab, and webhooks for automation pipelines — all enabling headless operation in CI. Missing for 10: independent/hands-on verification that CI pipelines actually work end-to-end, and no explicit example CI configs (e.g., GitHub Actions YAML) are shown.

   • [claimed-docs] “Gecko's v1 API lets you trigger scans, read scan, repository, and vulnerability data, triage findings, manage scan schedules, rotate your Gi…”
   • [claimed-docs] “Give the key a descriptive name, such as `CI/CD Pipeline` or `Production`.”
   • [claimed-docs] “Gecko posts a pass/fail check back to GitHub or GitLab so reviewers see the security status inline.”
   • [claimed-docs] “**Fail-on severity**: the severity threshold that fails the check (for example, fail on High and above).”
   • [claimed-docs] “Gecko can push events to your systems as scans run and findings change: page a channel when a scan fails, open a workflow when a critical la…”
   • [probe] “PROBE openapi: HTTP 200 at https://gecko.security/openapi.json — contains "openapi" key”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round drawn
   Snyk—none0/10

   All MCP-related evidence describes Snyk shipping its own local MCP server (snyk-docs-28, snyk-probe-3, snyk-probe-6) so other agents/IDEs can call Snyk's tools — this is Snyk acting as an MCP tool provider, not as a client that plugs in external MCP servers to use their tools. There is no evidence that Snyk's Remediation Agent, Snyk Studio, or CLI can consume/connect to third-party MCP servers.

   • [claimed-docs] “The Snyk MCP Server is designed as a local MCP server, running on your system using the Snyk CLI to ensure local file access.”
   • [probe] “official MCP server documented at https://docs.snyk.io/agent-security/agentic-security-with-snyk-studio/getting-started-with-snyk-studio”
   • [probe] “PROBE mcp-handshake (local run): `snyk mcp -t stdio` answered a JSON-RPC initialize with serverInfo {"name":"Snyk MCP Server","version":"1.1…”
   • [claimed-docs] “The Remediation Agent is an LLM-driven tool that automates vulnerability remediation in your projects. It scans your Project with Snyk, gene…”
   • [claimed-docs] “prompt your coding agent with natural language, for example, "scan my directory for security issues".”

   proof ↗⚑ flag

   Gecko Security—none0/10

   All evidence describes Gecko exposing its own data via an MCP server so external AI tools/agents can call into Gecko (docs-9, probe-4) — the reverse relationship. There is no evidence that Gecko itself can act as an MCP client and plug in external MCP servers to use their tools.

   • [claimed-docs] “Connect it to an AI tool and the tool can list your repositories and scans, pull full vulnerability evidence (proof of concept, call chain, …”
   • [probe] “official MCP server documented at https://gecko.security/docs/ai-tools/overview”

   proof ↗⚑ flag

4. ai-native userConnect an agent via an official MCP server

   weight 3 · round to Snyk
   Snyk✓fullprobed8/10

   Snyk documents an official local MCP server (Snyk Studio/MCP server running via the CLI) that lets coding agents connect and be prompted with natural language to scan for security issues, and a hands-on probe confirms the CLI actually responds to a JSON-RPC MCP handshake exposing tools/prompts/resources. missing for 10: independent third-party corroboration of MCP client interoperability across multiple agents, and clearer documentation on authentication requirements for the MCP server itself.

   • [claimed-docs] “Snyk Studio supports a default hooks-based approach for some agentic development environments (ADEs) to ensure code written by AI coding ass…”
   • [claimed-docs] “To validate the MCP server configuration, prompt your coding agent with natural language, for example, "scan my directory for security issue…”
   • [claimed-docs] “The Snyk MCP Server is designed as a local MCP server, running on your system using the Snyk CLI to ensure local file access.”
   • [probe] “official MCP server documented at https://docs.snyk.io/agent-security/agentic-security-with-snyk-studio/getting-started-with-snyk-studio”
   • [probe] “PROBE mcp-handshake (local run): `snyk mcp -t stdio` answered a JSON-RPC initialize with serverInfo {"name":"Snyk MCP Server","version":"1.1…”

   proof ↗⚑ flag

   Gecko Security✓fullprobed7/10

   Gecko documents an official MCP/AI-tools integration (docs-9, probe-4) letting a connected AI agent list repositories/scans, pull full vulnerability evidence, triage findings, and file tickets, and the docs.md/llms.txt endpoints confirm agent-facing documentation exists. Missing for 10: independent or hands-on confirmation that the MCP server works as described, and details on setup/auth flow for connecting an external agent.

   • [claimed-docs] “Connect it to an AI tool and the tool can list your repositories and scans, pull full vulnerability evidence (proof of concept, call chain, …”
   • [probe] “official MCP server documented at https://gecko.security/docs/ai-tools/overview”
   • [probe] “PROBE docs-md: HTTP 200 at https://gecko.security/docs.md --- title: "Gecko Security - Gecko Security" description: "An AI security engineer…”

   proof ↗⚑ flag

5. ai-native userUse an official CLI

   weight 2 · round to Snyk
   Snyk✓fullprobed9/10

   Snyk ships a well-documented official CLI (snyk-gh-1/3/7, snyk-probe-4/7) supporting local, IDE, and CI/CD workflows, plus explicit AI-native/agentic capabilities: `snyk fix --agentic` for human-in-the-loop remediation, `/snyk-fix` and `/snyk-batch-fix` slash commands for coding assistants, and a built-in MCP server (`snyk mcp -t stdio`) verified live via handshake probe. This directly satisfies the ai-native CLI story with both docs and hands-on confirmation. missing for 10: independent/community corroboration specifically praising the CLI's agentic features (community evidence here is generic and mixed, not focused on this), and probe-5 shows every scan requires prior authentication (no keyless/unauthenticated use).

   • [github] “The Snyk CLI brings the functionality of Snyk into your development workflow. You can run the CLI locally from the command line or in an IDE…”
   • [github] “You can run the CLI locally from the command line or in an IDE. You can also run the CLI in your CI/CD pipeline.”
   • [github] “The Snyk CLI brings the functionality of Snyk into your development workflow. You can run the CLI locally from the command line or in an IDE…”
   • [probe] “official CLI documented at https://docs.snyk.io/developer-tools/snyk-cli/snyk-cli”
   • [probe] “PROBE mcp-handshake (local run): `snyk mcp -t stdio` answered a JSON-RPC initialize with serverInfo {"name":"Snyk MCP Server","version":"1.1…”
   • [probe] “PROBE cli-version (local run): `snyk --version` printed 1.1307.0 and exited 0 after `brew tap snyk/tap && brew install snyk`.”
   • [claimed-docs] “Snyk CLI: Run `snyk fix --agentic` from your terminal to trigger a human-in-the-loop interactive remediation experience without an IDE.”
   • [claimed-docs] “Run `/snyk-fix` in your coding assistant to scan the project and apply a fix for the top vulnerability. Use `/snyk-batch-fix` to address mul…”
   • [claimed-docs] “The Snyk MCP Server is designed as a local MCP server, running on your system using the Snyk CLI to ensure local file access.”
   • [claimed-docs] “Run /snyk-fix in your coding assistant to scan the project and apply a fix for the top vulnerability.”

   proof ↗⚑ flag

   Gecko Security—none0/10

   No evidence of an official CLI; Gecko offers a GitHub App, REST API, webhooks, and an MCP server for AI tools, but nothing in the docs, changelog, or probes mentions a CLI tool for AI-native workflows.

   ⚑ flag

6. ai-native userDrive the product through a documented public API

   weight 3 · round to Gecko Security
   Snyk—none0/10

   The evidence pack shows CLI and MCP-server access but no documented public REST/GraphQL API; a direct probe for OpenAPI/swagger specs on docs.snyk.io returned 404 for all candidate paths, indicating no discoverable public API documentation.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.snyk.io/openapi.json, https://docs.snyk.io/swagger.json, https://docs.snyk.io/api/opena…”

   proof ↗⚑ flag

   Gecko Security✓fullprobed8/10

   Gecko documents a public v1 API (trigger scans, read scan/repo/vulnerability data, triage findings, manage schedules, rotate tokens, webhooks) with a live OpenAPI spec confirmed by probe, plus API key management and per-request logging. missing for 10: independent third-party developer confirmation of API usage/reliability beyond docs and probes, and more detail on API rate limits/versioning stability.

   • [claimed-docs] “Gecko's v1 API lets you trigger scans, read scan, repository, and vulnerability data, triage findings, manage scan schedules, rotate your Gi…”
   • [claimed-docs] “API keys are now visible to the whole team with owner attribution and per-request usage logging”
   • [claimed-docs] “Gecko can push events to your systems as scans run and findings change: page a channel when a scan fails, open a workflow when a critical la…”
   • [claimed-docs] “Give the key a descriptive name, such as `CI/CD Pipeline` or `Production`.”
   • [probe] “PROBE openapi: HTTP 200 at https://gecko.security/openapi.json — contains "openapi" key”

   proof ↗⚑ flag

7. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round to Gecko Security
   Snyk—none0/10

   No evidence describes scoped or least-privilege API credentials/tokens for AI agents; the only authentication evidence shows the CLI/MCP server requiring a standard Snyk account/API token with no mention of granular scopes or agent-specific permission levels. missing for 10: any documentation of scoped API tokens, role/permission-limited credentials, or agent-specific least-privilege auth mechanisms.

   • [probe] “PROBE cli-scan (local run, macOS, snyk 1.1307.0): `snyk test <dir>` on a fixture with known-vulnerable lodash 4.17.15 exited 2 with "ERROR A…”
   • [probe] “PROBE mcp-handshake (local run): `snyk mcp -t stdio` answered a JSON-RPC initialize with serverInfo {"name":"Snyk MCP Server","version":"1.1…”

   proof ↗⚑ flag

   Gecko Security!disputedcontradicted4/10

   Gecko documents API keys with named scopes, GitHub App 'scoped, auto-rotating access' and GitLab token rotation (docs-11, docs-25, docs-26, docs-12), which nominally support least-privilege credential issuance for connected tools/agents. However, community hands-on reports concretely contradict the least-privilege claim: the GitHub OAuth flow requests broad 'act on my behalf' and profile/email write permissions rather than fine-grained scopes, prompting explicit user pushback (comm-1, comm-7). Missing for 10: documented per-agent scope configuration, evidence that API keys/tokens can be restricted to specific actions/resources, and resolution of the OAuth over-permissioning complaint.

   • [claimed-docs] “The App grants Gecko scoped, auto-rotating access, with no personal access token to manage.”
   • [claimed-docs] “Give the key a descriptive name, such as `CI/CD Pipeline` or `Production`.”
   • [claimed-docs] “API keys are now visible to the whole team with owner attribution and per-request usage logging”
   • [claimed-docs] “PATCH /integrations/gitlab swaps in a fresh token, built for secret managers and rotation scripts.”
   • [community] “OAuth flow asks for permission to write GitHub email/profile settings and 'act on my behalf' - concerning for a vuln scan tool, not just fin…”
   • [community] “User objects to GitHub OAuth requesting 'act on your behalf' permission, equating it to full admin access across all repos and orgs; asks Ge…”

   proof ↗⚑ flag

8. ai-native userBuild against official SDKs

   weight 2 · round to Gecko Security
   Snyk—none0/10

   The evidence pack shows Snyk's CLI, MCP server, and various product scanners, but contains no mention of an official SDK (e.g., Python, Node, Java client libraries) for programmatic integration, and a probe for an OpenAPI/swagger spec returned 404s, indicating no discoverable official API SDK.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.snyk.io/openapi.json, https://docs.snyk.io/swagger.json, https://docs.snyk.io/api/opena…”
   • [github] “The Snyk CLI brings the functionality of Snyk into your development workflow. You can run the CLI locally from the command line or in an IDE…”
   • [claimed-docs] “The Snyk MCP Server is designed as a local MCP server, running on your system using the Snyk CLI to ensure local file access.”

   proof ↗⚑ flag

   Gecko Security~partialprobed5/10

   Gecko documents a v1 REST API with OpenAPI spec, API keys, and webhooks that developers can build against, but there is no evidence of official client SDK libraries (e.g., Python/JS packages) beyond the raw API/OpenAPI reference. missing for 10: dedicated SDK libraries/language clients, independent developer corroboration of building against them.

   • [claimed-docs] “Gecko's v1 API lets you trigger scans, read scan, repository, and vulnerability data, triage findings, manage scan schedules, rotate your Gi…”
   • [claimed-docs] “API keys are now visible to the whole team with owner attribution and per-request usage logging”
   • [probe] “PROBE openapi: HTTP 200 at https://gecko.security/openapi.json — contains "openapi" key”

   proof ↗⚑ flag

9. ai-native userSubscribe to events via webhooks

   weight 2 · round to Gecko Security
   Snyk—none0/10

   No evidence pack items mention webhooks or event subscription mechanisms; Snyk's integrations directory references Jira, Slack, GitHub/GitLab brokers, but nothing about outbound webhook events for AI-native consumption.

   ⚑ flag

   Gecko Security✓fullclaimed8/10

   Docs explicitly describe webhook subscriptions for scan/finding lifecycle events (scan fail, critical finding) and the API reference lists webhook events as a first-class capability alongside the API. This directly matches the story's ask for event subscription via webhooks. Missing for 10: independent/hands-on confirmation that webhook delivery works reliably and no detail on payload schema/retry semantics.

   • [claimed-docs] “Gecko can push events to your systems as scans run and findings change: page a channel when a scan fails, open a workflow when a critical la…”
   • [claimed-docs] “Gecko's v1 API lets you trigger scans, read scan, repository, and vulnerability data, triage findings, manage scan schedules, rotate your Gi…”

   proof ↗⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round to Snyk
   Snyk✓fullprobed7/10

   Snyk provides AI-generated insights directly in-product: DeepCode AI-powered risk scoring/prioritization, actionable fix advice, an LLM-driven Remediation Agent that scans, generates fix plans enriched with Snyk security intelligence, applies fixes, and verifies results, plus natural-language querying via MCP/Snyk Studio. missing for 10: independent/hands-on validation of insight quality and accuracy (community evidence is largely about unrelated PR/dependency issues, not AI-insight quality), and no data on how these insights surface for non-code data sources beyond code/deps/IaC.

   • [claimed-docs] “The Remediation Agent is an LLM-driven tool that automates vulnerability remediation in your projects. It scans your Project with Snyk, gene…”
   • [claimed-docs] “DeepCode AI powers 85%-accurate security autofixes and comprehensive app coverage, letting developers build fast while staying secure.”
   • [claimed-docs] “It scans your Project with Snyk, generates a fix plan enriched with Snyk security intelligence, applies the fixes, and verifies the result, …”
   • [claimed-docs] “DeepCode AI supports Snyk’s context-aware risk scoring, assessing the popularity of the affected package and the reachability of vulnerable …”
   • [claimed-docs] “Snyk CLI: Run `snyk fix --agentic` from your terminal to trigger a human-in-the-loop interactive remediation experience without an IDE.”
   • [claimed-docs] “Run `/snyk-fix` in your coding assistant to scan the project and apply a fix for the top vulnerability. Use `/snyk-batch-fix` to address mul…”
   • [claimed-docs] “Snyk Studio supports a default hooks-based approach for some agentic development environments (ADEs) to ensure code written by AI coding ass…”
   • [claimed-docs] “To validate the MCP server configuration, prompt your coding agent with natural language, for example, "scan my directory for security issue…”
   • [probe] “PROBE mcp-handshake (local run): `snyk mcp -t stdio` answered a JSON-RPC initialize with serverInfo {"name":"Snyk MCP Server","version":"1.1…”

   proof ↗⚑ flag

   Gecko Security!disputedcontradicted5/10

   Gecko generates AI insights (severity, confidence score, call chain, PoC, suggested patch, AI-written repo wiki, endpoint map, PR review summaries) directly from scanned code data, per docs-2/17/18/19. However, hands-on community reports show these AI-generated findings can be inaccurate — 100%-confidence 'vulnerabilities' that weren't real, missed real issues a competing tool caught, and a trivial/irrelevant finding on another scan — directly contradicting the reliability of the insights. Missing for 10: independent validation of insight accuracy at scale, resolution of the false-positive/false-negative reports.

   • [claimed-docs] “Each finding includes a severity (CVSS 4.0), a confidence score, the full source-to-sink call chain, a proof of concept, and a suggested pat…”
   • [claimed-docs] “Repository wiki: An AI-written map of your app: architecture, routing, and security model.”
   • [claimed-docs] “Endpoint map: The HTTP attack surface Gecko discovered in your code.”
   • [claimed-docs] “PR reviews & fixes: On pull requests, a security review summary and one-click fixes.”
   • [community] “Tool gave 100% confidence for two vulnerabilities (severity 9.4 and 6.5) that turned out not to be real.”
   • [community] “Ran Gecko on a C project after Semgrep AI flagged a minor real issue; Gecko found nothing. After removing the fix and recreating the project…”
   • [community] “Uploaded a project (sydbox) to scan; took 10 minutes and found only one vulnerability, an 'RCE: IRC Message Command Execution Bypass' in an …”

   proof ↗⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Gecko Security
   Snyk~partialcommunity6/10

   Snyk supports autonomous background operation via CI/CD pipeline scanning (snyk-gh-3) and an LLM-driven Remediation Agent that scans, generates a fix plan, applies fixes and verifies results without manual step-by-step intervention (snyk-docs-12, snyk-docs-29). However, the CLI's flagship 'agentic' fix mode is explicitly described as a 'human-in-the-loop interactive' experience rather than fully autonomous (snyk-docs-13), and community reports describe Snyk's older automated PR/remediation feature as noisy and low-value, leading users to disable it (snyk-comm-6, snyk-comm-11) — a concrete real-world caveat on unattended automation quality. Missing for 10: independent verification that the newer Remediation Agent runs unattended in production pipelines at scale, and confirmation that automated PR/fix quality issues raised by users have been resolved.

   • [claimed-docs] “The Remediation Agent is an LLM-driven tool that automates vulnerability remediation in your projects. It scans your Project with Snyk, gene…”
   • [claimed-docs] “Snyk CLI: Run `snyk fix --agentic` from your terminal to trigger a human-in-the-loop interactive remediation experience without an IDE.”
   • [claimed-docs] “Run `/snyk-fix` in your coding assistant to scan the project and apply a fix for the top vulnerability. Use `/snyk-batch-fix` to address mul…”
   • [claimed-docs] “It scans your Project with Snyk, generates a fix plan enriched with Snyk security intelligence, applies the fixes, and verifies the result, …”
   • [claimed-docs] “Run /snyk-fix in your coding assistant to scan the project and apply a fix for the top vulnerability.”
   • [github] “You can run the CLI locally from the command line or in an IDE. You can also run the CLI in your CI/CD pipeline.”
   • [community] “I'm not convinced their tools are better than npm audit + a license checker package... I've been very disappointed with their PR tools, and …”
   • [community] “we have this thrust on us ... we get pull requests for point release updates but it misses entire versions, it decides that packages that ha…”

   proof ↗⚑ flag

   Gecko Security✓fullclaimed7/10

   Gecko documents autonomous background workflows: scheduled scans via the API ('manage scan schedules'), automatic PR-triggered scans with pass/fail checks, auto-fix PR generation and re-verification, ticket creation, and webhook events pushed to external systems as scans run — all without manual intervention once configured. This directly matches the 'automations that run autonomously in the background' story for a security-scanning product. missing for 10: independent/hands-on confirmation that scheduled scans and auto-fix loops run reliably unattended over time, and more detail on configuring recurring schedules beyond a brief API mention.

   • [claimed-docs] “Gecko's v1 API lets you trigger scans, read scan, repository, and vulnerability data, triage findings, manage scan schedules, rotate your Gi…”
   • [claimed-docs] “Gecko can push events to your systems as scans run and findings change: page a channel when a scan fails, open a workflow when a critical la…”
   • [claimed-docs] “Gecko posts a pass/fail check back to GitHub or GitLab so reviewers see the security status inline.”
   • [claimed-docs] “Click Request fix to open a gecko/* pull request with the patch applied, or apply the patch yourself.”
   • [claimed-docs] “Gecko rechecks the finding and marks it Fix verified once the vulnerability is gone.”
   • [claimed-docs] “create a ticket in Jira, Linear, ClickUp, or Shortcut”
   • [claimed-docs] “Commit to an existing PR: For findings from a PR scan, Gecko can commit the fix directly to the PR's head branch so the fix rides along with…”

   proof ↗⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round to Snyk
   Snyk✓fullprobed7/10

   Snyk ships a documented built-in AI 'Remediation Agent' that scans a project, generates and applies a fix plan, and verifies results directly inside the coding assistant or CLI (snyk-docs-12/13/23/29/30), backed by DeepCode AI-driven autofixes (snyk-docs-14/15/36) and a local MCP server confirmed to actually start and respond via handshake (snyk-probe-6). This goes beyond simple scanning to actual task delegation to an in-product AI agent. Missing for 10: independent/hands-on evidence validating the remediation agent's real-world fix quality/success rate (only vendor-claimed 85% accuracy), and no third-party review of the delegation UX itself.

   • [claimed-docs] “The Remediation Agent is an LLM-driven tool that automates vulnerability remediation in your projects. It scans your Project with Snyk, gene…”
   • [claimed-docs] “Snyk CLI: Run `snyk fix --agentic` from your terminal to trigger a human-in-the-loop interactive remediation experience without an IDE.”
   • [claimed-docs] “Run `/snyk-fix` in your coding assistant to scan the project and apply a fix for the top vulnerability. Use `/snyk-batch-fix` to address mul…”
   • [claimed-docs] “It scans your Project with Snyk, generates a fix plan enriched with Snyk security intelligence, applies the fixes, and verifies the result, …”
   • [claimed-docs] “Run /snyk-fix in your coding assistant to scan the project and apply a fix for the top vulnerability.”
   • [claimed-docs] “The Snyk MCP Server is designed as a local MCP server, running on your system using the Snyk CLI to ensure local file access.”
   • [probe] “PROBE mcp-handshake (local run): `snyk mcp -t stdio` answered a JSON-RPC initialize with serverInfo {"name":"Snyk MCP Server","version":"1.1…”
   • [claimed-docs] “DeepCode AI powers 85%-accurate security autofixes and comprehensive app coverage, letting developers build fast while staying secure.”
   • [claimed-docs] “DeepCode AI supports Snyk’s context-aware risk scoring, assessing the popularity of the affected package and the reachability of vulnerable …”

   proof ↗⚑ flag

   Gecko Security~partialprobed5/10

   Gecko is framed as an 'AI security engineer' that autonomously scans, triages, patches, and files tickets, so users can delegate security tasks (request fix, ticket creation, PR fixes) to its built-in AI (gecko-security-docs-3, gecko-security-docs-6, gecko-security-docs-15, gecko-security-probe-1). However, this is a fixed pipeline of security actions rather than a general-purpose in-product assistant you can converse with or direct on arbitrary tasks; community reports also raise doubts about the AI's accuracy in these delegated tasks (gecko-security-comm-3, gecko-security-comm-5). Missing for 10: an explicit conversational/task-delegation assistant UI, evidence of open-ended task delegation beyond the scan/fix/ticket workflow, and independent confirmation the delegated AI reliably completes tasks.

   • [claimed-docs] “Click Request fix to open a gecko/* pull request with the patch applied, or apply the patch yourself.”
   • [claimed-docs] “create a ticket in Jira, Linear, ClickUp, or Shortcut”
   • [claimed-docs] “Click **Request fix** to open a `gecko/*` pull request with the patch applied, or apply the patch yourself.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://gecko.security/llms.txt # Gecko Security > Gecko Security is an AI security engineer that finds exploit…”
   • [community] “Tool gave 100% confidence for two vulnerabilities (severity 9.4 and 6.5) that turned out not to be real.”
   • [community] “Uploaded a project (sydbox) to scan; took 10 minutes and found only one vulnerability, an 'RCE: IRC Message Command Execution Bypass' in an …”

   proof ↗⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round to Snyk
   Snyk✓fullprobed8/10

   Snyk documents an MCP server and 'Snyk Studio' agentic workflow where users prompt coding agents in natural language (e.g., 'scan my directory for security issues'), plus slash-style commands (/snyk-fix, /snyk-batch-fix) and an LLM-driven Remediation Agent that plans and applies fixes autonomously. A hands-on probe confirms the MCP server actually starts and responds to JSON-RPC initialize, corroborating the documented natural-language/agentic workflow beyond marketing claims. Missing for 10: broader independent (non-Snyk) user reports specifically validating natural-language command accuracy/reliability in real projects.

   • [claimed-docs] “To validate the MCP server configuration, prompt your coding agent with natural language, for example, "scan my directory for security issue…”
   • [claimed-docs] “prompt your coding agent with natural language, for example, "scan my directory for security issues".”
   • [claimed-docs] “Run `/snyk-fix` in your coding assistant to scan the project and apply a fix for the top vulnerability. Use `/snyk-batch-fix` to address mul…”
   • [claimed-docs] “Run /snyk-fix in your coding assistant to scan the project and apply a fix for the top vulnerability.”
   • [claimed-docs] “The Snyk MCP Server is designed as a local MCP server, running on your system using the Snyk CLI to ensure local file access.”
   • [claimed-docs] “The Remediation Agent is an LLM-driven tool that automates vulnerability remediation in your projects. It scans your Project with Snyk, gene…”
   • [claimed-docs] “Snyk CLI: Run `snyk fix --agentic` from your terminal to trigger a human-in-the-loop interactive remediation experience without an IDE.”
   • [probe] “official MCP server documented at https://docs.snyk.io/agent-security/agentic-security-with-snyk-studio/getting-started-with-snyk-studio”
   • [probe] “PROBE mcp-handshake (local run): `snyk mcp -t stdio` answered a JSON-RPC initialize with serverInfo {"name":"Snyk MCP Server","version":"1.1…”

   proof ↗⚑ flag

   Gecko Security~partialprobed6/10

   Gecko documents an official MCP/AI-tools integration letting a connected AI assistant list repos/scans, pull vulnerability evidence, triage findings, and file tickets via natural language, plus a full v1 API and OpenAPI spec (docs-9, docs-10, probe-3, probe-4). However, there's no hands-on or independent evidence confirming the natural-language workflow actually works well end-to-end — community reports focus on scan accuracy issues rather than the NL/agent interface itself. Missing for 10: independent/hands-on verification of the MCP natural-language workflow, and detail on command coverage/limitations.

   • [claimed-docs] “Connect it to an AI tool and the tool can list your repositories and scans, pull full vulnerability evidence (proof of concept, call chain, …”
   • [claimed-docs] “Gecko's v1 API lets you trigger scans, read scan, repository, and vulnerability data, triage findings, manage scan schedules, rotate your Gi…”
   • [probe] “PROBE openapi: HTTP 200 at https://gecko.security/openapi.json — contains "openapi" key”
   • [probe] “official MCP server documented at https://gecko.security/docs/ai-tools/overview”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   Snyk—none0/10

   No evidence of an interactive API reference with runnable examples; the evidence pack includes an explicit probe showing all candidate OpenAPI/Swagger spec URLs return 404, and no docs reference an API explorer or 'try it' console.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.snyk.io/openapi.json, https://docs.snyk.io/swagger.json, https://docs.snyk.io/api/opena…”

   proof ↗⚑ flag

   Gecko Security—none0/10

   While Gecko exposes an OpenAPI spec (gecko-security-probe-3) and documents API capabilities (gecko-security-docs-10), there is no evidence of an interactive API reference UI with runnable/try-it examples for developers or AI agents to explore.

   • [probe] “PROBE openapi: HTTP 200 at https://gecko.security/openapi.json — contains "openapi" key”
   • [claimed-docs] “Gecko's v1 API lets you trigger scans, read scan, repository, and vulnerability data, triage findings, manage scan schedules, rotate your Gi…”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round to Gecko Security
   Snyk—none0/10

   A probe explicitly checked for an OpenAPI/swagger spec at Snyk's docs domain and found all candidate paths (openapi.json, swagger.json, etc.) returning 404 — no machine-readable API spec was found. No other evidence in the pack shows a downloadable OpenAPI/Swagger file for Snyk's API.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.snyk.io/openapi.json, https://docs.snyk.io/swagger.json, https://docs.snyk.io/api/opena…”

   proof ↗⚑ flag

   Gecko Security✓fullprobed9/10

   A probe confirms a live OpenAPI spec at gecko.security/openapi.json (HTTP 200, contains 'openapi' key), and docs describe a full v1 API for scans, vulnerabilities, webhooks, and integrations. Missing for 10: no independent/community confirmation of the spec's completeness or usage beyond the vendor's own probe.

   • [probe] “PROBE openapi: HTTP 200 at https://gecko.security/openapi.json — contains "openapi" key”
   • [claimed-docs] “Gecko's v1 API lets you trigger scans, read scan, repository, and vulnerability data, triage findings, manage scan schedules, rotate your Gi…”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round drawn
   Snyk—none0/10

   No evidence describes a dedicated sandbox/test environment isolated from production data; Snyk's CLI, remediation agent, and MCP server all operate directly on the user's local project or repository, and no docs mention an explicit non-production sandbox mode for AI-native agent testing.

   ⚑ flag

   Gecko Security—none0/10

   The evidence describes scanning source repos/ZIP uploads and hybrid AWS deployment (scanner runs in customer's own account), but nothing explicitly describes a dedicated sandbox environment for safely testing exploits or PoCs without touching production data. missing for 10: explicit sandbox/isolated test environment description, confirmation that PoC execution or exploit verification never touches live production systems or data.

   • [claimed-docs] “Paste a public repository URL or upload a ZIP instead. No provider connection required.”
   • [claimed-docs] “the scanner, S3 storage, and all AI inference run inside your AWS account. Gecko hosts only the dashboard, database, and scan orchestration.”
   • [claimed-docs] “the scanner, S3 storage, and all AI inference run inside your AWS account.”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round to Gecko Security
   Snyk—none0/10

   The evidence pack contains no documentation of a versioned API or a deprecation policy; in fact the probe for OpenAPI/swagger specs at docs.snyk.io returned 404 on all candidate paths, and no other artifact (CLI, MCP server, docs) references API versioning or deprecation guarantees.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.snyk.io/openapi.json, https://docs.snyk.io/swagger.json, https://docs.snyk.io/api/opena…”

   proof ↗⚑ flag

   Gecko Security~partialprobed3/10

   Gecko documents a versioned v1 API with an OpenAPI spec and a release/channel/version resolution endpoint (digest-pinned image refs), showing some notion of API versioning, but there is no documented deprecation policy, version sunset timeline, or migration guidance for API consumers. missing for 10: explicit deprecation policy, version support lifecycle/EOL dates, migration guides between API versions.

   • [claimed-docs] “Gecko's v1 API lets you trigger scans, read scan, repository, and vulnerability data, triage findings, manage scan schedules, rotate your Gi…”
   • [claimed-docs] “resolves a version to its release... the response includes the digest-pinned image_ref to pull”
   • [claimed-docs] “GET /releases/{channel}/{version} resolves a version to its release.”
   • [probe] “PROBE openapi: HTTP 200 at https://gecko.security/openapi.json — contains "openapi" key”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Snyk
   Snyk✓fullcommunity7/10

   Snyk explicitly documents a `/snyk-batch-fix` slash command for coding assistants to fix multiple vulnerabilities in one operation, and the Remediation Agent scans, generates a fix plan, and applies fixes across a whole project rather than one issue at a time. Independent community evidence corroborates bulk behavior in practice (Snyk generating 'dozens or hundreds of PRs' for dependency updates), showing the batch capability actually operates at scale, even if some users found the volume unwieldy. Missing for 10: a documented bulk/batch REST API for programmatic multi-item operations, and clearer scale limits or first-party benchmarks for large-scale automation.

   • [claimed-docs] “Run `/snyk-fix` in your coding assistant to scan the project and apply a fix for the top vulnerability. Use `/snyk-batch-fix` to address mul…”
   • [claimed-docs] “The Remediation Agent is an LLM-driven tool that automates vulnerability remediation in your projects. It scans your Project with Snyk, gene…”
   • [claimed-docs] “It scans your Project with Snyk, generates a fix plan enriched with Snyk security intelligence, applies the fixes, and verifies the result, …”
   • [community] “I'm not convinced their tools are better than npm audit + a license checker package... I've been very disappointed with their PR tools, and …”

   proof ↗⚑ flag

   Gecko Security~partialclaimed5/10

   The v1 API and AI-tool integration let a user programmatically list repositories/scans, pull vulnerability data, triage findings, and trigger scans across many items, which supports scripted bulk-style automation, but there is no documented bulk-fix, bulk-triage, or multi-select UI action for operating on many findings/repos simultaneously. missing for 10: a dedicated bulk-action endpoint or UI (e.g., bulk fix/triage/dismiss across many findings at once), independent evidence of bulk workflows in practice.

   • [claimed-docs] “Connect it to an AI tool and the tool can list your repositories and scans, pull full vulnerability evidence (proof of concept, call chain, …”
   • [claimed-docs] “Gecko's v1 API lets you trigger scans, read scan, repository, and vulnerability data, triage findings, manage scan schedules, rotate your Gi…”
   • [claimed-docs] “Gecko can push events to your systems as scans run and findings change: page a channel when a scan fails, open a workflow when a critical la…”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to Gecko Security
   Snyk~partialcommunity4/10

   Snyk supports some automated, event-driven actions (auto-generated fix PRs on new vulnerabilities, CI/CD pipeline gating, hooks-based Studio integration for agentic environments, and policy management mentioned in pricing docs) but there is no documented general-purpose rule engine letting an AI-native user define custom triggers/conditions to fire arbitrary actions. Community feedback (snyk-comm-6, snyk-comm-11) even notes the automated PR behavior is inflexible and noisy rather than configurable. missing for 10: explicit custom rule/condition builder, documentation of webhook or event-subscription API for arbitrary triggers, evidence of user-configurable automation beyond default PRs/hooks.

   • [claimed-docs] “Snyk Studio supports a default hooks-based approach for some agentic development environments (ADEs) to ensure code written by AI coding ass…”
   • [claimed-docs] “Snyk provides actionable fix advice for vulnerabilities and supports workflows to fix vulnerabilities using pull requests.”
   • [claimed-docs] “Policy management”
   • [community] “I'm not convinced their tools are better than npm audit + a license checker package... I've been very disappointed with their PR tools, and …”
   • [community] “we have this thrust on us ... we get pull requests for point release updates but it misses entire versions, it decides that packages that ha…”

   proof ↗⚑ flag

   Gecko Security~partialclaimed5/10

   Gecko supports some event-driven automation—webhooks that fire on scan completion/finding changes, configurable fail-on-severity thresholds for PR checks, and automatic ticket/ PR creation on findings—but these are fixed, product-defined triggers rather than a user-configurable rules engine where arbitrary conditions map to arbitrary actions. missing for 10: a general-purpose rule/condition builder, support for chaining multiple custom actions per event, and evidence of user-defined logic beyond severity thresholds.

   • [claimed-docs] “Gecko can push events to your systems as scans run and findings change: page a channel when a scan fails, open a workflow when a critical la…”
   • [claimed-docs] “**Fail-on severity**: the severity threshold that fails the check (for example, fail on High and above).”
   • [claimed-docs] “create a ticket in Jira, Linear, ClickUp, or Shortcut”
   • [claimed-docs] “Click Request fix to open a gecko/* pull request with the patch applied, or apply the patch yourself.”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round to Gecko Security
   Snyk—none0/10

   Snyk supports scanning via CLI, CI/CD pipelines, and IDE integrations, but there is no evidence of a native scheduler or built-in mechanism for recurring/cron-style jobs or workflows; recurring scans would need to be built externally via CI/CD cron triggers, which is not documented as a Snyk feature.

   ⚑ flag

   Gecko Security~partialclaimed4/10

   The v1 API reference states it lets you 'manage scan schedules,' implying recurring scan jobs can be configured programmatically, but there is no dedicated doc page, UI walkthrough, or example showing how to set cadence, or evidence this extends to broader 'workflows' beyond scanning. missing for 10: dedicated scheduling documentation/UI, example of setting recurring cadence, evidence of scheduling beyond scans (e.g., recurring triage/ticket workflows), independent confirmation it works as claimed.

   • [claimed-docs] “Gecko's v1 API lets you trigger scans, read scan, repository, and vulnerability data, triage findings, manage scan schedules, rotate your Gi…”

   proof ↗⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round drawn
   Snyk—none0/10

   Snyk is a security scanning/remediation tool; there is no evidence of versioning, reviewing, or rolling back 'automations' (e.g., saved workflows, agent configurations, or fix scripts) — the closest features are PR-based fixes and a remediation agent, but no version history, review/audit trail, or rollback mechanism for automations is documented.

   ⚑ flag

   Gecko Security—none0/10

   Gecko Security is a vulnerability-scanning/security tool, not an automation-building or workflow tool; the evidence shows PR creation, fix verification, and ticket filing but nothing about versioning, reviewing, or rolling back 'automations' (i.e., the scanning/triage workflows themselves). No changelog history, version control, or rollback mechanism for the automations it runs is documented.

   • [claimed-docs] “Click Request fix to open a gecko/* pull request with the patch applied, or apply the patch yourself.”
   • [claimed-docs] “Gecko rechecks the finding and marks it Fix verified once the vulnerability is gone.”
   • [claimed-docs] “create a ticket in Jira, Linear, ClickUp, or Shortcut”

   proof ↗⚑ flag

Agent scanning

1. ai-native userHave an agent run a scan headlessly via the CLI, parse machine-readable results, and file issues for real findings

   weight 3 · round to Snyk
   Snyk~partialprobed5/10

   The Snyk CLI is confirmed to run headlessly in CI/CD pipelines (snyk-gh-3, snyk-gh-7) and produces a terminal report of vulnerabilities with severity, path, and fix guidance (snyk-gh-4, snyk-gh-6), and Jira/Slack integrations exist for tracking issues (snyk-docs-6, snyk-intdir-1). However, the pack never documents a machine-readable (JSON/SARIF) output flag for CLI scans, nor an automated pipeline that parses CLI results and files issues for real findings specifically — Jira export appears to be a platform-level feature rather than a CLI-driven headless workflow. missing for 10: documented CLI JSON/SARIF output flag, evidence of an agent parsing that output and auto-filing issues end-to-end.

   • [github] “You can run the CLI locally from the command line or in an IDE. You can also run the CLI in your CI/CD pipeline.”
   • [github] “The Snyk CLI brings the functionality of Snyk into your development workflow. You can run the CLI locally from the command line or in an IDE…”
   • [github] “Look at the `test` command report in your terminal. The report shows the vulnerabilities Snyk found in the package.”
   • [github] “The report shows the vulnerabilities Snyk found in the package. For each issue found, Snyk reports the severity of the issue, provides a lin…”
   • [claimed-docs] “Track and export Snyk issues to your Jira project.”
   • [probe] “PROBE cli-scan (local run, macOS, snyk 1.1307.0): `snyk test <dir>` on a fixture with known-vulnerable lodash 4.17.15 exited 2 with "ERROR A…”

   proof ↗⚑ flag

   Gecko Security~partialcommunity4/10

   Gecko exposes a v1 API and webhooks that can trigger scans, pull machine-readable vulnerability data, triage findings, and file tickets in Jira/Linear/ClickUp/Shortcut, and an MCP/AI-tools connector for agents to do the same — satisfying the 'headless, parse, file issues' intent via API rather than a literal CLI. However, no dedicated CLI binary/command is documented anywhere in the evidence, and community reports raise concrete concerns about false positives (100% confidence findings that weren't real, missed real issues) undermining 'file issues for real findings'. Missing for 10: an actual CLI tool/subcommand documented, and independent confirmation that AI-driven ticket filing acts only on validated/real findings rather than the false positives community testers observed.

   • [claimed-docs] “Connect it to an AI tool and the tool can list your repositories and scans, pull full vulnerability evidence (proof of concept, call chain, …”
   • [claimed-docs] “Gecko's v1 API lets you trigger scans, read scan, repository, and vulnerability data, triage findings, manage scan schedules, rotate your Gi…”
   • [claimed-docs] “create a ticket in Jira, Linear, ClickUp, or Shortcut”
   • [claimed-docs] “Gecko can push events to your systems as scans run and findings change: page a channel when a scan fails, open a workflow when a critical la…”
   • [community] “Tool gave 100% confidence for two vulnerabilities (severity 9.4 and 6.5) that turned out not to be real.”
   • [community] “Uploaded a project (sydbox) to scan; took 10 minutes and found only one vulnerability, an 'RCE: IRC Message Command Execution Bypass' in an …”

   proof ↗⚑ flag

2. ai-native userWire the scanner into my coding agent so AI-generated code is scanned for issues before it lands

   weight 2 · round to Snyk
   Snyk✓fullprobed9/10

   Snyk documents and a hands-on probe confirm a local MCP server (snyk mcp -t stdio) that plugs into coding agents/ADEs, plus hooks-based secure-at-inception workflows, natural-language scan prompts, and slash-commands (/snyk-fix, /snyk-batch-fix) for scanning and remediating AI-generated code before it lands. This is corroborated independently by a local probe showing the MCP server actually starts and responds to JSON-RPC initialize. Missing for 10: independent/community verification specifically of the agentic MCP workflow's real-world reliability (community evidence found is about the CLI/PR features generally, not the MCP-agent integration).

   • [claimed-docs] “Snyk Studio supports a default hooks-based approach for some agentic development environments (ADEs) to ensure code written by AI coding ass…”
   • [claimed-docs] “To validate the MCP server configuration, prompt your coding agent with natural language, for example, "scan my directory for security issue…”
   • [claimed-docs] “The Snyk MCP Server is designed as a local MCP server, running on your system using the Snyk CLI to ensure local file access.”
   • [claimed-docs] “Run `/snyk-fix` in your coding assistant to scan the project and apply a fix for the top vulnerability. Use `/snyk-batch-fix` to address mul…”
   • [claimed-docs] “Run /snyk-fix in your coding assistant to scan the project and apply a fix for the top vulnerability.”
   • [probe] “official MCP server documented at https://docs.snyk.io/agent-security/agentic-security-with-snyk-studio/getting-started-with-snyk-studio”
   • [probe] “PROBE mcp-handshake (local run): `snyk mcp -t stdio` answered a JSON-RPC initialize with serverInfo {"name":"Snyk MCP Server","version":"1.1…”

   proof ↗⚑ flag

   Gecko Security~partialprobed6/10

   Gecko documents an MCP/AI-tool integration that lets an agent list repos/scans, pull vulnerability evidence, triage findings, and file tickets, plus PR/CI checks that fail builds above a severity threshold — together enabling a coding-agent workflow to scan AI-generated code before merge. However, independent hands-on reports raise real accuracy concerns (false positives at 100% confidence, missed real bugs, UI/scan bugs), which undercuts confidence that the scanning is reliable enough for a pre-merge gate. Missing for 10: independent corroboration that the MCP/agent wiring works smoothly in practice, and evidence addressing the accuracy complaints specifically in an agent-driven pre-commit/pre-merge context.

   • [claimed-docs] “Connect it to an AI tool and the tool can list your repositories and scans, pull full vulnerability evidence (proof of concept, call chain, …”
   • [probe] “official MCP server documented at https://gecko.security/docs/ai-tools/overview”
   • [claimed-docs] “Gecko posts a pass/fail check back to GitHub or GitLab so reviewers see the security status inline.”
   • [claimed-docs] “**Fail-on severity**: the severity threshold that fails the check (for example, fail on High and above).”
   • [community] “Tool gave 100% confidence for two vulnerabilities (severity 9.4 and 6.5) that turned out not to be real.”
   • [community] “Ran Gecko on a C project after Semgrep AI flagged a minor real issue; Gecko found nothing. After removing the fix and recreating the project…”
   • [community] “Uploaded a project (sydbox) to scan; took 10 minutes and found only one vulnerability, an 'RCE: IRC Message Command Execution Bypass' in an …”

   proof ↗⚑ flag

Ci gates

1. devops-leadRun scans in CI on every pull request and fail the build on new findings via exit codes

   weight 3 · round to Gecko Security
   Snyk~partialprobed6/10

   Evidence confirms Snyk CLI is explicitly designed to run in CI/CD pipelines (snyk-gh-3, snyk-gh-7) and that `snyk test`/`snyk code test` reports vulnerabilities found (snyk-gh-4, snyk-gh-6), which is the standard mechanism CI systems use to gate builds. However, the pack has no explicit documentation of PR-trigger configuration or exit-code semantics for failing builds, and the probe shows the CLI requires authentication for every scan (snyk-probe-5), adding an operational caveat for CI setup. Missing for 10: explicit docs on PR-triggered CI workflows, documented exit-code behavior (e.g., non-zero on new findings), and any GitHub Actions/PR-check integration guide.

   • [github] “You can run the CLI locally from the command line or in an IDE. You can also run the CLI in your CI/CD pipeline.”
   • [github] “The Snyk CLI brings the functionality of Snyk into your development workflow. You can run the CLI locally from the command line or in an IDE…”
   • [github] “Look at the `test` command report in your terminal. The report shows the vulnerabilities Snyk found in the package.”
   • [github] “The report shows the vulnerabilities Snyk found in the package. For each issue found, Snyk reports the severity of the issue, provides a lin…”
   • [probe] “PROBE cli-scan (local run, macOS, snyk 1.1307.0): `snyk test <dir>` on a fixture with known-vulnerable lodash 4.17.15 exited 2 with "ERROR A…”

   proof ↗⚑ flag

   Gecko Security✓fullclaimed8/10

   Docs describe PR-triggered scans that post a pass/fail check to GitHub/GitLab, with a configurable fail-on-severity threshold and separation of new vs pre-existing findings, which together enable failing a PR build on new findings. This matches the CI/PR-gating story closely, though it's implemented as a GitHub/GitLab check status rather than an explicit CLI exit code and lacks independent hands-on confirmation of build-failing behavior. Missing for 10: independent verification that the pass/fail check actually blocks merges/builds in practice, and explicit mention of CLI exit-code semantics for non-GitHub CI runners.

   • [claimed-docs] “Gecko posts a pass/fail check back to GitHub or GitLab so reviewers see the security status inline.”
   • [claimed-docs] “Findings introduced by the PR are separated from pre-existing ones, so reviewers focus on what the change adds.”
   • [claimed-docs] “**Fail-on severity**: the severity threshold that fails the check (for example, fail on High and above).”
   • [claimed-docs] “install the Gecko GitHub App and pick repositories”

   proof ↗⚑ flag

2. developerSee scan findings surfaced as pull-request comments or checks in GitHub or GitLab

   weight 2 · round to Gecko Security
   Snyk~partialclaimed5/10

   Docs confirm Snyk integrates scanning results into 'repositories' and 'source code managers' and supports GitHub/GitLab via Snyk Broker (snyk-intdir-3, snyk-intdir-4), plus CI/CD pipeline integration (snyk-docs-18, snyk-gh-3) and PR-based fix workflows (snyk-docs-21). However, no evidence explicitly describes PR comments or GitHub/GitLab check-run surfaces for findings—only generic 'integrations' and 'fix PR' language. Missing for 10: explicit documentation or screenshot of PR comment/check UI, independent confirmation of check-status behavior, and GitLab-specific PR comment support.

   • [claimed-docs] “Snyk Code is a developer-first static application security testing (SAST) solution. By scanning code in real-time and providing actionable i…”
   • [claimed-docs] “Snyk provides actionable fix advice for vulnerabilities and supports workflows to fix vulnerabilities using pull requests.”
   • [claimed-docs] “By scanning code in real-time and providing actionable insights directly in the developer workflow across IDEs, repositories, and CI/CD pipe…”
   • [claimed-docs] “Real-time code scanning Integrations with IDE, CLI, and source code managers”
   • [github] “You can run the CLI locally from the command line or in an IDE. You can also run the CLI in your CI/CD pipeline.”
   • [claimed-docs] “GitHub - install and configure Snyk Broker”
   • [claimed-docs] “GitLab - install and configure Snyk Broker:”

   proof ↗⚑ flag

   Gecko Security✓fullclaimed8/10

   Docs describe a GitHub/GitLab App integration that posts pass/fail checks and PR security review summaries, separates new vs pre-existing findings, supports fail-on-severity thresholds, and can auto-commit fixes to the PR — a fairly complete PR-check/comment workflow (gecko-security-docs-5, -19, -28, -29, -30). Missing for 10: independent/hands-on confirmation that PR comments/checks actually render as described (community evidence only discusses finding accuracy, not the PR-check UX itself).

   • [claimed-docs] “Gecko posts a pass/fail check back to GitHub or GitLab so reviewers see the security status inline.”
   • [claimed-docs] “PR reviews & fixes: On pull requests, a security review summary and one-click fixes.”
   • [claimed-docs] “Findings introduced by the PR are separated from pre-existing ones, so reviewers focus on what the change adds.”
   • [claimed-docs] “**Fail-on severity**: the severity threshold that fails the check (for example, fail on High and above).”
   • [claimed-docs] “Commit to an existing PR: For findings from a PR scan, Gecko can commit the fix directly to the PR's head branch so the fix rides along with…”

   proof ↗⚑ flag

Pre commit

1. developerRun the scanner as a pre-commit hook so secrets and issues never enter my git history

   weight 3 · round drawn
   Snyk—none0/10

   Evidence shows Snyk CLI can run locally, in IDEs, and in CI/CD pipelines, but there is no mention of a pre-commit hook integration, and Snyk's product line (SAST, SCA, container, IaC) does not include secrets scanning at all. Missing for 10: any documentation of a pre-commit hook workflow, and any secrets-detection capability.

   • [github] “You can run the CLI locally from the command line or in an IDE. You can also run the CLI in your CI/CD pipeline.”
   • [github] “The Snyk CLI brings the functionality of Snyk into your development workflow. You can run the CLI locally from the command line or in an IDE…”
   • [claimed-docs] “Snyk Code is a developer-first static application security testing (SAST) solution. By scanning code in real-time and providing actionable i…”

   proof ↗⚑ flag

   Gecko Security—none0/10

   Gecko's documented workflow is GitHub App/PR-based scanning with pass/fail PR checks, not a local pre-commit git hook that blocks commits before they enter history — no evidence of a pre-commit hook, CLI hook script, or local git-hook integration exists in the evidence pack.

   ⚑ flag

Containers

1. devops-leadScan container images for OS-package and application-layer vulnerabilities before deploying them

   weight 3 · round to Snyk
   Snyk✓fullcommunity7/10

   Snyk explicitly ships 'Snyk Container' for scanning container images and is described platform-wide as scanning 'container images' alongside code and dependencies, and one community comment even notes hearing good things about the container security tooling specifically. missing for 10: explicit doc language distinguishing OS-package vs application-layer vulnerability detection within images, and any hands-on/independent verification of a container scan (the probes only exercise `snyk test`/MCP, not `snyk container test`).

   • [claimed-docs] “Snyk Container provides tools and integrations to quickly find and fix vulnerabilities. This allows you to create images that have security …”
   • [claimed-docs] “Scan, prioritize, and fix vulnerabilities in your code, open-source dependencies, container images, and cloud configurations.”
   • [community] “Dependabot doesn't work well for a Fortune 50 company... Snyk like most security tools lacks perspective that takes business and how product…”

   proof ↗⚑ flag

   Gecko Security—none0/10

   Gecko's documented capability is AI-driven source-code (SAST) analysis across languages like TypeScript, Python, Go, Java, etc., producing call-chain/PoC findings and PR fixes — no evidence describes scanning container images for OS-package (e.g., CVE feeds on apt/rpm packages) or layered image vulnerabilities. The only container-adjacent mention is a changelog entry resolving a version to a digest-pinned image_ref, which is about release artifact resolution, not vulnerability scanning of container layers.

   • [claimed-docs] “Compiler-accurate (precise cross-file analysis) | TypeScript · JavaScript · Python · Go · Java · Scala · C# · Rust”
   • [claimed-docs] “resolves a version to its release... the response includes the digest-pinned image_ref to pull”
   • [claimed-docs] “GET /releases/{channel}/{version} resolves a version to its release.”

   proof ↗⚑ flag

Iac misconfig

1. devops-leadScan Terraform, Kubernetes, and other IaC files for security misconfigurations

   weight 2 · round to Snyk
   Snyk✓fullclaimed8/10

   Snyk IaC explicitly documents scanning Terraform, CloudFormation, Kubernetes, and Azure ARM configs for misconfigurations across IDE, SCM, CLI, and CI workflows, plus drift detection for manually-created cloud resources. This is corroborated by CLI docs and the general Snyk platform overview listing cloud configuration scanning as a core capability. Missing for 10: independent hands-on validation of IaC scan accuracy/false-positive rates and no direct probe confirming `snyk iac test` output.

   • [claimed-docs] “With Snyk IaC, you can: Write secure configurations for HashiCorp Terraform, AWS CloudFormation, Kubernetes, and Azure Resource Manager (ARM…”
   • [claimed-docs] “Detect manually created resources in your cloud.”
   • [claimed-docs] “Write secure configurations for HashiCorp Terraform, AWS CloudFormation, Kubernetes, and Azure Resource Manager (ARM) - for IDE, SCM, CLI, a…”
   • [claimed-docs] “Write secure configurations for HashiCorp Terraform, AWS CloudFormation, Kubernetes, and Azure Resource Manager (ARM)... View issues and rec…”
   • [claimed-docs] “Scan, prioritize, and fix vulnerabilities in your code, open-source dependencies, container images, and cloud configurations.”

   proof ↗⚑ flag

   Gecko Security—none0/10

   Gecko's documented scanning scope is source-code vulnerability analysis (TypeScript, JavaScript, Python, Go, Java, Scala, C#, Rust) with call-chain/PoC findings; no evidence anywhere mentions Terraform, Kubernetes manifests, or IaC misconfiguration scanning. The axis is plausible for a security-scanning product, but no capability is evidenced, so this is 'none' rather than 'na'.

   • [claimed-docs] “Compiler-accurate (precise cross-file analysis) | TypeScript · JavaScript · Python · Go · Java · Scala · C# · Rust”
   • [claimed-docs] “Repository wiki: An AI-written map of your app: architecture, routing, and security model.”
   • [claimed-docs] “Endpoint map: The HTTP attack surface Gecko discovered in your code.”

   proof ↗⚑ flag

Coverage

1. developerScan projects across all the major languages and package ecosystems my organization uses

   weight 2 · round to Snyk
   Snyk✓fullcommunity7/10

   Snyk documents dedicated scanners spanning open-source dependencies (with indirect/transitive dependency detection), SAST (Snyk Code) across IDEs/repos/CI, container images, and IaC configs (Terraform, CloudFormation, Kubernetes, ARM), plus a CLI usable locally, in IDEs, and in CI/CD — collectively covering the major ecosystems a typical org would need. Community feedback raises accuracy/noise concerns (e.g., mismatched results vs npm audit, PR spam) but does not concretely dispute the breadth of language/ecosystem coverage itself. Missing for 10: an explicit enumerated list of supported languages/package managers and independent verification of coverage completeness across less-common ecosystems.

   • [claimed-docs] “Snyk Open Source allows you to find and fix vulnerabilities in the open-source libraries used by your applications.”
   • [claimed-docs] “Open-source packages often reference other packages, and many vulnerabilities are found in these indirect dependencies.”
   • [claimed-docs] “Snyk Code is a developer-first static application security testing (SAST) solution. By scanning code in real-time and providing actionable i…”
   • [claimed-docs] “Snyk Code is a developer-first static application security testing (SAST) solution. By scanning code in real-time and providing actionable i…”
   • [claimed-docs] “Snyk Container provides tools and integrations to quickly find and fix vulnerabilities. This allows you to create images that have security …”
   • [claimed-docs] “With Snyk IaC, you can: Write secure configurations for HashiCorp Terraform, AWS CloudFormation, Kubernetes, and Azure Resource Manager (ARM…”
   • [claimed-docs] “Write secure configurations for HashiCorp Terraform, AWS CloudFormation, Kubernetes, and Azure Resource Manager (ARM) - for IDE, SCM, CLI, a…”
   • [github] “The Snyk CLI brings the functionality of Snyk into your development workflow. You can run the CLI locally from the command line or in an IDE…”
   • [github] “You can run the CLI locally from the command line or in an IDE. You can also run the CLI in your CI/CD pipeline.”
   • [community] “They use a different vulnerability database. Snyk's contains vulnerabilities that NPM's doesn't have, and vice versa. We're using them both …”
   • [community] “These confuse me because they don't align with yarn/npm audit. Which one is right? It is pretty unrealistic to dig into it myself.”

   proof ↗⚑ flag

   Gecko Security~partialcommunity5/10

   Docs list a defined set of 'compiler-accurate' languages (TypeScript, JavaScript, Python, Go, Java, Scala, C#, Rust) supporting cross-file analysis, but there is no mention of package/dependency-ecosystem scanning (npm, pip, Maven, Cargo, etc.) or of broader language coverage (C/C++, Ruby, PHP, Kotlin, Swift). Community testing on a C project (unsupported language) found nothing, reinforcing that coverage is limited to the stated language list rather than 'all major languages and ecosystems.' Missing for 10: explicit package/dependency-ecosystem scanning support, broader language list, and independent verification of coverage breadth.

   • [claimed-docs] “Compiler-accurate (precise cross-file analysis) | TypeScript · JavaScript · Python · Go · Java · Scala · C# · Rust”
   • [community] “Ran Gecko on a C project after Semgrep AI flagged a minor real issue; Gecko found nothing. After removing the fix and recreating the project…”

   proof ↗⚑ flag

Output formats

1. developerEmit scan results in machine-readable formats like JSON or SARIF for downstream tooling

   weight 2 · round to Gecko Security
   Snyk—none0/10

   The evidence pack documents Snyk CLI's terminal reports, CI/CD integration, and various scan types, but never mentions JSON or SARIF output formats or any machine-readable export capability for downstream tooling. This is a fair and applicable axis for a security scanner, but no evidence confirms it.

   • [github] “Look at the `test` command report in your terminal. The report shows the vulnerabilities Snyk found in the package.”
   • [github] “The report shows the vulnerabilities Snyk found in the package. For each issue found, Snyk reports the severity of the issue, provides a lin…”
   • [github] “You can run the CLI locally from the command line or in an IDE. You can also run the CLI in your CI/CD pipeline.”

   proof ↗⚑ flag

   Gecko Security~partialprobed4/10

   Gecko's v1 API lets developers read scan, repository, and vulnerability data programmatically (implying JSON responses) and an OpenAPI spec is published, but there is no explicit documentation of a SARIF export format or dedicated 'JSON report' output for CI pipelines. Missing for 10: explicit SARIF format support, documented JSON report export command, and any CI-integration example showing machine-readable output being consumed downstream.

   • [claimed-docs] “Gecko's v1 API lets you trigger scans, read scan, repository, and vulnerability data, triage findings, manage scan schedules, rotate your Gi…”
   • [probe] “PROBE openapi: HTTP 200 at https://gecko.security/openapi.json — contains "openapi" key”
   • [claimed-docs] “Gecko can push events to your systems as scans run and findings change: page a channel when a scan fails, open a workflow when a critical la…”

   proof ↗⚑ flag

Dependency fixes

1. developerGet concrete upgrade paths or automated fix pull requests that remediate vulnerable dependencies

   weight 2 · round to Snyk
   Snyk!disputedcontradicted5/10

   Snyk docs claim actionable fix advice and automated PR-based remediation workflows (snyk-docs-21), reinforced by CLI fix guidance (snyk-gh-6) and newer agentic 'snyk fix --agentic'/Remediation Agent features (snyk-docs-12, snyk-docs-13, snyk-docs-29). However, hands-on community reports concretely contradict the automated-PR experience: users describe disappointment with the PR tool generating dozens/hundreds of noisy PRs instead of proper rewrites, missing whole version bumps, and disabling the feature entirely in favor of Dependabot (snyk-comm-6, snyk-comm-11). missing for 10: independent verification that the newer agentic/Remediation Agent fix-PR flow resolves these historically reported PR-quality issues, and quantitative data on fix-PR success/merge rates.

   • [claimed-docs] “Snyk provides actionable fix advice for vulnerabilities and supports workflows to fix vulnerabilities using pull requests.”
   • [claimed-docs] “The Remediation Agent is an LLM-driven tool that automates vulnerability remediation in your projects. It scans your Project with Snyk, gene…”
   • [claimed-docs] “Snyk CLI: Run `snyk fix --agentic` from your terminal to trigger a human-in-the-loop interactive remediation experience without an IDE.”
   • [github] “The report shows the vulnerabilities Snyk found in the package. For each issue found, Snyk reports the severity of the issue, provides a lin…”
   • [community] “I'm not convinced their tools are better than npm audit + a license checker package... I've been very disappointed with their PR tools, and …”
   • [community] “we have this thrust on us ... we get pull requests for point release updates but it misses entire versions, it decides that packages that ha…”

   proof ↗⚑ flag

   Gecko Security—none0/10

   Gecko's documented capability is source-to-sink code vulnerability detection (SAST-style) with a generic 'Request fix' PR mechanism for findings it detects in application code; there is no evidence of dependency/package manifest scanning, CVE-to-package matching, or specific upgrade-path recommendations for vulnerable dependencies (SCA). Missing for 10: any mention of software composition analysis, dependency manifest parsing, package version resolution, or upgrade-path guidance tied to third-party library CVEs.

   • [claimed-docs] “Each finding includes a severity (CVSS 4.0), a confidence score, the full source-to-sink call chain, a proof of concept, and a suggested pat…”
   • [claimed-docs] “Click Request fix to open a gecko/* pull request with the patch applied, or apply the patch yourself.”
   • [claimed-docs] “Click **Request fix** to open a `gecko/*` pull request with the patch applied, or apply the patch yourself.”
   • [claimed-docs] “Compiler-accurate (precise cross-file analysis) | TypeScript · JavaScript · Python · Go · Java · Scala · C# · Rust”

   proof ↗⚑ flag

Sca scanning

1. developerScan my dependency manifests and lockfiles for packages with known vulnerabilities

   weight 3 · round to Snyk
   Snyk✓fullprobed8/10

   Snyk Open Source is documented extensively as a dependency/manifest/lockfile scanner for known vulnerabilities, including indirect dependencies, with CLI (`snyk test`) and CI/IDE integration, actionable fix advice, and PR-based remediation (snyk-docs-1,19,20,21,27; snyk-gh-1,2,4,6). Hands-on probes confirm the CLI scan functionality actually works end-to-end (reporting severity, path, fix guidance) and community reports corroborate real-world usage, though some users note noisy/low-value alerts and mismatches with npm audit. missing for 10: independent benchmark data on false-positive rate, and confirmation of coverage across all lockfile formats (e.g., poetry.lock, Gemfile.lock) beyond npm examples.

   • [claimed-docs] “Snyk Open Source allows you to find and fix vulnerabilities in the open-source libraries used by your applications.”
   • [claimed-docs] “Open-source packages often reference other packages, and many vulnerabilities are found in these indirect dependencies.”
   • [claimed-docs] “Snyk Open Source can also scan your Projects for license compliance, checking against licenses known to Snyk.”
   • [claimed-docs] “Snyk provides actionable fix advice for vulnerabilities and supports workflows to fix vulnerabilities using pull requests.”
   • [claimed-docs] “By using Snyk Open Source, you can reduce the risks introduced by open-source components.”
   • [github] “The Snyk CLI brings the functionality of Snyk into your development workflow. You can run the CLI locally from the command line or in an IDE…”
   • [github] “Alternatively, you can perform a quick test on a public npm package, for example `snyk test ionic`.”
   • [github] “Look at the `test` command report in your terminal. The report shows the vulnerabilities Snyk found in the package.”
   • [github] “The report shows the vulnerabilities Snyk found in the package. For each issue found, Snyk reports the severity of the issue, provides a lin…”
   • [community] “It's a good tool, and from what I can tell, the free version is probably enough for most small teams or sole developers. The main benefit of…”
   • [community] “They use a different vulnerability database. Snyk's contains vulnerabilities that NPM's doesn't have, and vice versa. We're using them both …”
   • [community] “These confuse me because they don't align with yarn/npm audit. Which one is right? It is pretty unrealistic to dig into it myself.”
   • [probe] “PROBE cli-scan (local run, macOS, snyk 1.1307.0): `snyk test <dir>` on a fixture with known-vulnerable lodash 4.17.15 exited 2 with "ERROR A…”

   proof ↗⚑ flag

   Gecko Security—none0/10

   Gecko's documented capability is AI-driven source-code (SAST) analysis — call-chain tracing, endpoint mapping, PR-level code review — with no mention of scanning package manifests, lockfiles, or third-party dependency CVEs. Since Gecko is a security scanner, dependency/SCA scanning is a fair axis to ask about, but no evidence shows it exists.

   ⚑ flag

2. security-engineerPrioritize dependency alerts by whether the vulnerable code is actually reachable from my code

   weight 2 · round to Snyk
   Snyk~partialclaimed6/10

   Snyk's DeepCode AI documentation explicitly states it supports 'context-aware risk scoring, assessing the popularity of the affected package and the reachability of vulnerable code' (snyk-docs-36), directly addressing reachability-based prioritization, and Snyk Code offers risk scoring incorporating prevalence and ease of fix (snyk-docs-4). However, this reachability capability is described only briefly and tied to DeepCode AI/Code rather than clearly extended to Snyk Open Source (dependency/SCA) alerts specifically, and there is no independent or hands-on evidence confirming reachability analysis works reliably in practice for dependency vulnerabilities. missing for 10: explicit documentation that reachability analysis applies to Snyk Open Source (SCA) alerts specifically (not just SAST/DeepCode), independent/hands-on verification of reachability accuracy, and detail on how reachability signals surface in the alert triage UI/workflow.

   • [claimed-docs] “DeepCode AI supports Snyk’s context-aware risk scoring, assessing the popularity of the affected package and the reachability of vulnerable …”
   • [claimed-docs] “Sort by and prioritize the more important issues by incorporating factors such as issue prevalence, ease of fix, and risk factor into a sing…”
   • [claimed-docs] “Open-source packages often reference other packages, and many vulnerabilities are found in these indirect dependencies.”

   proof ↗⚑ flag

   Gecko Security—none0/10

   Gecko's docs describe source-to-sink call-chain analysis and reachability for vulnerabilities found in first-party code, but there is no evidence it ingests dependency/SCA alerts (e.g., known-CVE package advisories) or prioritizes them by reachability — it's positioned as a SAST-style code vulnerability scanner, not a dependency/SCA tool.

   • [claimed-docs] “Each finding includes a severity (CVSS 4.0), a confidence score, the full source-to-sink call chain, a proof of concept, and a suggested pat…”
   • [claimed-docs] “Compiler-accurate (precise cross-file analysis) | TypeScript · JavaScript · Python · Go · Java · Scala · C# · Rust”
   • [claimed-docs] “Repository wiki: An AI-written map of your app: architecture, routing, and security model.”
   • [claimed-docs] “Endpoint map: The HTTP attack surface Gecko discovered in your code.”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Gecko Security
   Snyk—none0/10

   The evidence pack contains no documentation of a public Snyk API offering feature parity with the UI/CLI; the only probe for an OpenAPI spec returned 404s at all candidate locations, and all other evidence focuses on CLI, IDE, MCP, and UI-based workflows rather than a comprehensive API surface for AI-native/programmatic use.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.snyk.io/openapi.json, https://docs.snyk.io/swagger.json, https://docs.snyk.io/api/opena…”
   • [github] “The Snyk CLI brings the functionality of Snyk into your development workflow. You can run the CLI locally from the command line or in an IDE…”
   • [github] “You can run the CLI locally from the command line or in an IDE. You can also run the CLI in your CI/CD pipeline.”

   proof ↗⚑ flag

   Gecko Security~partialprobed6/10

   Gecko exposes a documented v1 API (and OpenAPI spec) covering scans, repository/vulnerability data, triage, scheduling, token rotation and webhooks, plus an AI-tools/MCP-style integration that can list repos, pull evidence, triage, and file tickets — a substantial subset of UI functionality. However, there's no explicit claim or evidence that every UI feature (e.g., repository wiki generation, endpoint map, PR review summaries, initial GitHub App connection setup) is API-accessible, so full UI/API parity isn't demonstrated. Missing for 10: explicit parity statement or evidence that wiki/endpoint-map/PR-review generation and account/connection setup are API-triggerable, independent confirmation of API completeness.

   • [claimed-docs] “Gecko's v1 API lets you trigger scans, read scan, repository, and vulnerability data, triage findings, manage scan schedules, rotate your Gi…”
   • [claimed-docs] “Connect it to an AI tool and the tool can list your repositories and scans, pull full vulnerability evidence (proof of concept, call chain, …”
   • [probe] “PROBE openapi: HTTP 200 at https://gecko.security/openapi.json — contains "openapi" key”
   • [claimed-docs] “Repository wiki: An AI-written map of your app: architecture, routing, and security model.”
   • [claimed-docs] “Endpoint map: The HTTP attack surface Gecko discovered in your code.”
   • [claimed-docs] “PR reviews & fixes: On pull requests, a security review summary and one-click fixes.”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round to Gecko Security
   Snyk—none0/10

   Evidence shows Snyk can export individual issues to Jira and CLI scan reports print to terminal, but there is no documentation of a comprehensive data export feature (projects, scan history, findings) in open/portable formats, nor any account-deletion/data-portability guidance for users wanting to leave the platform.

   • [claimed-docs] “Track and export Snyk issues to your Jira project.”
   • [github] “Look at the `test` command report in your terminal. The report shows the vulnerabilities Snyk found in the package.”
   • [github] “The report shows the vulnerabilities Snyk found in the package. For each issue found, Snyk reports the severity of the issue, provides a lin…”

   proof ↗⚑ flag

   Gecko Security~partialprobed3/10

   The v1 API lets users read scan, repository, and vulnerability data programmatically (JSON via OpenAPI), which provides a technical path to export findings data, but there is no documented bulk 'export all data' feature, no mention of open-format guarantees for full account data, and no offboarding/exit workflow. missing for 10: explicit full-data export tooling, documented open-format (e.g. CSV/JSON) export guarantee, and account-closure/data-portability policy.

   • [claimed-docs] “Gecko's v1 API lets you trigger scans, read scan, repository, and vulnerability data, triage findings, manage scan schedules, rotate your Gi…”
   • [probe] “PROBE openapi: HTTP 200 at https://gecko.security/openapi.json — contains "openapi" key”

   proof ↗⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round drawn
   Snyk—none0/10

   Snyk is a proprietary SaaS security platform; only the CLI wrapper is on GitHub (snyk-gh-1..7) with no license text or claim of an open license provided in the evidence, and the core scanning engine, Snyk Code, DeepCode AI, and platform backend remain closed-source. No evidence pack item states an open-source license for any Snyk component, so the story is unmet. missing for 10: explicit open-source license (e.g., MIT/Apache) for the CLI repo, open-sourcing of core scanning/backend code, any statement of source availability for Snyk Code/DeepCode AI.

   • [github] “The Snyk CLI brings the functionality of Snyk into your development workflow. You can run the CLI locally from the command line or in an IDE…”
   • [github] “You can run the CLI locally from the command line or in an IDE. You can also run the CLI in your CI/CD pipeline.”
   • [github] “The Snyk CLI brings the functionality of Snyk into your development workflow. You can run the CLI locally from the command line or in an IDE…”

   proof ↗⚑ flag

   Gecko Security—none0/10

   No evidence that Gecko Security's source code is published or licensed openly; it is a proprietary SaaS/security scanning product with documented APIs and hybrid deployment, but nothing indicates its codebase is open source.

   ⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round to Gecko Security
   Snyk—none0/10

   No evidence anywhere in the pack of a self-hosted/on-prem deployment option for Snyk's core scanning platform; instead, hands-on probe evidence shows the opposite — `snyk test` fails with a 401 Authentication error unless connected to Snyk's cloud, and the CLI/MCP server function as clients to a hosted backend, with only a 'Broker' for connecting on-prem SCM to Snyk's cloud (not a self-hosted core).

   • [probe] “PROBE cli-scan (local run, macOS, snyk 1.1307.0): `snyk test <dir>` on a fixture with known-vulnerable lodash 4.17.15 exited 2 with "ERROR A…”
   • [probe] “PROBE mcp-handshake (local run): `snyk mcp -t stdio` answered a JSON-RPC initialize with serverInfo {"name":"Snyk MCP Server","version":"1.1…”
   • [claimed-docs] “GitHub - install and configure Snyk Broker”
   • [claimed-docs] “GitLab - install and configure Snyk Broker:”

   proof ↗⚑ flag

   Gecko Security~partialclaimed6/10

   Docs confirm a 'hybrid' deployment where the scanner, S3 storage, and AI inference all run inside the customer's AWS account, and the pricing page lists an 'On-prem / self-hosted / private cloud' tier — both indicating genuine self-hosting options exist. However, the same hybrid docs state Gecko still hosts the dashboard, database, and scan orchestration, so the core control plane is not actually self-hostable in the standard offering, and no architecture/setup docs for the full on-prem tier are provided. Missing for 10: detailed on-prem deployment guide/architecture for the fully self-hosted tier, and confirmation that dashboard/orchestration can also run on customer infrastructure.

   • [claimed-docs] “the scanner, S3 storage, and all AI inference run inside your AWS account. Gecko hosts only the dashboard, database, and scan orchestration.”
   • [claimed-docs] “the scanner, S3 storage, and all AI inference run inside your AWS account.”
   • [claimed-docs] “Unlimited scanning * On-prem / self-hosted / private cloud”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round to Gecko Security
   Snyk—none0/10

   No evidence in the pack addresses data residency, regional storage options, or data location controls for Snyk's platform or AI features.

   ⚑ flag

   Gecko Security~partialclaimed5/10

   Gecko offers a hybrid deployment where the scanner, S3 storage, and AI inference run inside the customer's own AWS account, and an on-prem/self-hosted/private cloud pricing tier, both of which let customers effectively control where their data resides. However, there's no explicit UI/setting for choosing a region for the Gecko-hosted dashboard/database components, and no documentation of specific supported regions or compliance certifications tied to residency. Missing for 10: explicit region-selection controls for Gecko-hosted components, list of supported regions, and residency/compliance documentation (e.g., GDPR/data-locality guarantees).

   • [claimed-docs] “the scanner, S3 storage, and all AI inference run inside your AWS account. Gecko hosts only the dashboard, database, and scan orchestration.”
   • [claimed-docs] “the scanner, S3 storage, and all AI inference run inside your AWS account.”
   • [claimed-docs] “Unlimited scanning * On-prem / self-hosted / private cloud”

   proof ↗⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   Snyk—none0/10

   The evidence pack contains no documentation of an opt-out or data-usage policy specifically preventing customer code/data from being used to train Snyk's AI models (DeepCode AI, Remediation Agent, MCP-based agents). The only related evidence is a general licensing clause granting Snyk broad rights to uploaded content (snyk-comm-13), which if anything raises concern rather than confirming a training opt-out. Since Snyk processes source code via AI features, this axis clearly applies, but no capability or setting addressing AI-training data use is evidenced.

   • [community] “From their policies page: 'by uploading or posting content to the Platform and providing access to your system's source code repository, you…”

   proof ↗⚑ flag

   Gecko Security—none0/10

   No evidence in the pack addresses data-training opt-out, AI training data policies, or any privacy controls governing whether user code/data is used to train AI models; Gecko's docs focus on scanning, deployment, and integrations, not this privacy axis.

   ⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round drawn
   Snyk—none0/10

   No evidence pack item documents Snyk offering configurable data-retention windows or a user-initiated deletion mechanism for scanned code/data; the only tangential evidence is a community citation noting Snyk's terms grant it a broad, perpetual license to uploaded source code, which runs counter to user control over retention rather than supporting it. missing for 10: documented retention policy/settings, data deletion request workflow, DPA/GDPR deletion tooling, any first-party privacy-controls page.

   • [community] “From their policies page: 'by uploading or posting content to the Platform and providing access to your system's source code repository, you…”

   proof ↗⚑ flag

   Gecko Security—none0/10

   The evidence pack covers scanning, PR fixes, integrations, hybrid deployment, and API/webhooks, but there is no mention of data retention policies, data deletion controls, or user ability to purge stored findings/scan data. Hybrid deployment keeps scanner/AI inference in the customer's AWS account, but this does not address retention/deletion of dashboard-hosted data. missing for 10: explicit data retention policy documentation, a deletion/purge mechanism or API, and any account/data-export or GDPR-style deletion workflow.

   • [claimed-docs] “the scanner, S3 storage, and all AI inference run inside your AWS account. Gecko hosts only the dashboard, database, and scan orchestration.”
   • [claimed-docs] “the scanner, S3 storage, and all AI inference run inside your AWS account.”

   proof ↗⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Snyk—none0/10

   No evidence pack item discusses a telemetry/usage-tracking opt-out mechanism for Snyk CLI, IDE plugins, or MCP server; scans require authentication with a Snyk account (per snyk-probe-5), implying data is sent to Snyk servers, but no opt-out or privacy controls are documented.

   ⚑ flag

   Gecko Security—none0/10

   No evidence anywhere in the pack of a telemetry/usage-tracking opt-out setting, privacy policy control, or data-collection toggle; the docs focus on scanning, PR fixes, API keys, and deployment but never mention telemetry opt-out.

   ⚑ flag

Ai remediation

1. ai-native userHave AI draft remediation pull requests for security findings that I review and merge

   weight 2 · round to Gecko Security
   Snyk!disputedcontradicted4/10

   Snyk documents both a rule-based fix-PR workflow for Open Source vulnerabilities (snyk-docs-21) and a newer LLM-driven 'Remediation Agent' that scans, generates a fix plan, and applies fixes via CLI/coding assistant (snyk-docs-12, snyk-docs-29, snyk-docs-30), which aligns with 'AI drafts a remediation PR for review.' However, hands-on community feedback specifically about Snyk's automated PR feature reports it as poor quality — generating dozens/hundreds of noisy PRs rather than meaningful rewrites, to the point the user disabled it in favor of Dependabot (snyk-comm-6) — directly contradicting the smooth 'draft-then-review-and-merge' experience implied by the story. Missing for 10: first-party or independent evidence the Remediation Agent itself opens a git PR (docs show it applying fixes locally/in-IDE, not confirmed PR creation), and corroboration that the AI-drafted PR quality is trustworthy enough for simple review-and-merge.

   • [claimed-docs] “Snyk provides actionable fix advice for vulnerabilities and supports workflows to fix vulnerabilities using pull requests.”
   • [claimed-docs] “The Remediation Agent is an LLM-driven tool that automates vulnerability remediation in your projects. It scans your Project with Snyk, gene…”
   • [claimed-docs] “It scans your Project with Snyk, generates a fix plan enriched with Snyk security intelligence, applies the fixes, and verifies the result, …”
   • [claimed-docs] “Run /snyk-fix in your coding assistant to scan the project and apply a fix for the top vulnerability.”
   • [community] “I'm not convinced their tools are better than npm audit + a license checker package... I've been very disappointed with their PR tools, and …”

   proof ↗⚑ flag

   Gecko Security✓fullcommunity8/10

   Docs describe a complete AI-drafted remediation PR workflow: 'Request fix' opens a gecko/* PR with the patch applied, fixes can commit directly to the PR's head branch, and Gecko rechecks and marks 'Fix verified' once merged and the vulnerability is resolved — squarely matching the ai-native review-and-merge story. Community reports raise concerns about finding accuracy/false positives, but do not contradict that the PR-drafting/remediation mechanism itself works as documented. Missing for 10: independent hands-on confirmation that the auto-generated patches are high quality and mergeable without heavy rework.

   • [claimed-docs] “Click Request fix to open a gecko/* pull request with the patch applied, or apply the patch yourself.”
   • [claimed-docs] “Click **Request fix** to open a `gecko/*` pull request with the patch applied, or apply the patch yourself.”
   • [claimed-docs] “When the fix merges, Gecko rechecks the finding and marks it **Fix verified** once the vulnerability is gone.”
   • [claimed-docs] “Commit to an existing PR: For findings from a PR scan, Gecko can commit the fix directly to the PR's head branch so the fix rides along with…”
   • [claimed-docs] “PR reviews & fixes: On pull requests, a security review summary and one-click fixes.”
   • [community] “Tool gave 100% confidence for two vulnerabilities (severity 9.4 and 6.5) that turned out not to be real.”

   proof ↗⚑ flag

Code analysis

1. security-engineerDetect taint-style vulnerabilities that flow across files and function boundaries

   weight 2 · round to Snyk
   Snyk✓fullclaimed7/10

   Snyk Code's docs explicitly describe SAST scanning with 'source to sink' step-by-step flow visualization, which is textbook taint-style vulnerability detection tracing data flow through code paths (snyk-docs-2, snyk-docs-3, snyk-docs-18/26/33). This is corroborated by the CLI's snyk code test command for source code scanning (snyk-gh-5). missing for 10: explicit documentation stating flows are tracked across separate files/modules and function boundaries specifically (vs. within a single file), and independent hands-on validation of cross-file taint accuracy

   • [claimed-docs] “Snyk Code is a developer-first static application security testing (SAST) solution. By scanning code in real-time and providing actionable i…”
   • [claimed-docs] “Visualize the path of the issue from source to sink with a step-by-step flow.”
   • [claimed-docs] “Snyk Code is a developer-first static application security testing (SAST) solution. By scanning code in real-time and providing actionable i…”
   • [claimed-docs] “Snyk Code is a developer-first static application security testing (SAST) solution. By scanning code in real-time and providing actionable i…”
   • [github] “To scan your source code, run `snyk code test`.”

   proof ↗⚑ flag

   Gecko Security!disputedcontradicted5/10

   Docs explicitly claim 'compiler-accurate' cross-file analysis and 'full source-to-sink call chain' reporting, which directly describes taint-style detection across file/function boundaries (gecko-security-docs-2, gecko-security-docs-20). However, independent hands-on reports show concrete failures of this claim: 100%-confidence findings that were not real vulnerabilities, missed a real known issue flagged by another tool, and an inaccurate/irrelevant finding on a scanned project (gecko-security-comm-3, gecko-security-comm-4, gecko-security-comm-5). Missing for 10: independent verification of true-positive cross-file taint chains, resolution of the reported false positive/negative cases, and reproducible benchmark data.

   • [claimed-docs] “Each finding includes a severity (CVSS 4.0), a confidence score, the full source-to-sink call chain, a proof of concept, and a suggested pat…”
   • [claimed-docs] “Compiler-accurate (precise cross-file analysis) | TypeScript · JavaScript · Python · Go · Java · Scala · C# · Rust”
   • [community] “Tool gave 100% confidence for two vulnerabilities (severity 9.4 and 6.5) that turned out not to be real.”
   • [community] “Ran Gecko on a C project after Semgrep AI flagged a minor real issue; Gecko found nothing. After removing the fix and recreating the project…”
   • [community] “Uploaded a project (sydbox) to scan; took 10 minutes and found only one vulnerability, an 'RCE: IRC Message Command Execution Bypass' in an …”

   proof ↗⚑ flag

2. developerRun static analysis that finds vulnerability classes like injection and XSS in my source code

   weight 3 · round to Snyk
   Snyk✓fullclaimed8/10

   Snyk Code is explicitly documented as a developer-first SAST solution that scans source code in real-time across IDEs, repos, and CI/CD, with source-to-sink flow visualization and CLI support (`snyk code test`) confirming actual static analysis capability. While injection/XSS are common SAST vulnerability classes, the evidence doesn't explicitly name these specific classes as detected, and community feedback focuses mostly on Open Source/dependency scanning rather than Snyk Code's SAST accuracy for these specific vuln types. Missing for 10: explicit mention of injection/XSS as detected vulnerability classes, independent hands-on verification of SAST rule coverage/accuracy for these specific classes.

   • [claimed-docs] “Snyk Code is a developer-first static application security testing (SAST) solution. By scanning code in real-time and providing actionable i…”
   • [claimed-docs] “Visualize the path of the issue from source to sink with a step-by-step flow.”
   • [claimed-docs] “Snyk Code is a developer-first static application security testing (SAST) solution. By scanning code in real-time and providing actionable i…”
   • [claimed-docs] “By scanning code in real-time and providing actionable insights directly in the developer workflow across IDEs, repositories, and CI/CD pipe…”
   • [claimed-docs] “Snyk Code is a developer-first static application security testing (SAST) solution. By scanning code in real-time and providing actionable i…”
   • [github] “To scan your source code, run `snyk code test`.”

   proof ↗⚑ flag

   Gecko Security!disputedcontradicted5/10

   Gecko's docs describe compiler-accurate, cross-file static analysis with source-to-sink call chains, CVSS severity, and confidence scores that would encompass classes like SQLi/XSS (docs-2, docs-20), and it supports multiple languages. However, independent hands-on testing on Hacker News reports both missed real vulnerabilities and high-confidence false positives (comm-3, comm-4, comm-5), directly contradicting the accuracy implied by the docs. Missing for 10: independent benchmark data on injection/XSS detection rates, and resolution of the false-positive/false-negative reports.

   • [claimed-docs] “Each finding includes a severity (CVSS 4.0), a confidence score, the full source-to-sink call chain, a proof of concept, and a suggested pat…”
   • [claimed-docs] “Compiler-accurate (precise cross-file analysis) | TypeScript · JavaScript · Python · Go · Java · Scala · C# · Rust”
   • [community] “Tool gave 100% confidence for two vulnerabilities (severity 9.4 and 6.5) that turned out not to be real.”
   • [community] “Ran Gecko on a C project after Semgrep AI flagged a minor real issue; Gecko found nothing. After removing the fix and recreating the project…”
   • [community] “Uploaded a project (sydbox) to scan; took 10 minutes and found only one vulnerability, an 'RCE: IRC Message Command Execution Bypass' in an …”

   proof ↗⚑ flag

Custom rules

1. security-engineerWrite custom static-analysis rules in a simple syntax and share them across repositories

   weight 2 · round to Snyk
   Snyk~partialclaimed3/10

   Only one thin mention (writing custom queries with DeepCode AI logic and autocomplete) suggests any custom rule authoring, but there is no detail on the rule syntax, no evidence of how rules are structured or shared across repositories, and no mention of a rule-sharing/policy mechanism analogous to Semgrep-style custom SAST rules. Missing for 10: documented custom rule syntax/DSL, examples of custom rules, and a clear cross-repo sharing/config mechanism.

   • [claimed-docs] “Write your own queries using DeepCode AI logic (with autocomplete) and leverage the hybrid AI to instantly find matches in your code.”
   • [claimed-docs] “DeepCode AI supports Snyk’s context-aware risk scoring, assessing the popularity of the affected package and the reachability of vulnerable …”

   proof ↗⚑ flag

   Gecko Security—none0/10

   Gecko Security is an AI-driven vulnerability scanner using compiler-accurate cross-file analysis, not a rule-based SAST engine; there is no evidence of a custom rule-writing syntax (e.g., YAML/DSL rules like Semgrep) or any mechanism to define and share such rules across repositories.

   • [claimed-docs] “Compiler-accurate (precise cross-file analysis) | TypeScript · JavaScript · Python · Go · Java · Scala · C# · Rust”
   • [claimed-docs] “Repository wiki: An AI-written map of your app: architecture, routing, and security model.”

   proof ↗⚑ flag

Fix suggestions

1. developerApply suggested or automatic code fixes for findings instead of only reading reports

   weight 2 · round to Gecko Security
   Snyk!disputedcontradicted6/10

   Snyk's docs document multiple concrete fix-application paths beyond reporting: PR-based fix workflows for open-source (snyk-docs-21), suggested-fix ignore/config (snyk-docs-5), IaC 'fix advice... make changes directly to code' (snyk-docs-34), and newer agentic tooling — Remediation Agent that 'applies the fixes and verifies the result', `snyk fix --agentic`, and `/snyk-fix`/`/snyk-batch-fix` slash commands (snyk-docs-12,13,23,29,30) plus DeepCode AI's '85%-accurate security autofixes' (snyk-docs-14). However, independent hands-on community feedback specifically on the PR-based auto-fix workflow contradicts vendor claims of quality: a developer reports being 'very disappointed with their PR tools' to the point of disabling automated PRs, saying Dependabot is '100x better to work with' (snyk-comm-6), and another calls the appsec/PR tooling 'not worth it' after a demo (snyk-comm-12). missing for 10: independent verification that the newer Remediation Agent/agentic autofix and SAST/IaC suggested-fix mechanisms (as opposed to the older PR-fix flow) actually work reliably in practice, and resolution of the direct community contradiction on fix-PR quality.

   • [claimed-docs] “Snyk provides actionable fix advice for vulnerabilities and supports workflows to fix vulnerabilities using pull requests.”
   • [claimed-docs] “Configure Snyk to ignore suggested fixes for an issue to suppress specific warnings.”
   • [claimed-docs] “Write secure configurations for HashiCorp Terraform, AWS CloudFormation, Kubernetes, and Azure Resource Manager (ARM)... View issues and rec…”
   • [claimed-docs] “The Remediation Agent is an LLM-driven tool that automates vulnerability remediation in your projects. It scans your Project with Snyk, gene…”
   • [claimed-docs] “Snyk CLI: Run `snyk fix --agentic` from your terminal to trigger a human-in-the-loop interactive remediation experience without an IDE.”
   • [claimed-docs] “Run `/snyk-fix` in your coding assistant to scan the project and apply a fix for the top vulnerability. Use `/snyk-batch-fix` to address mul…”
   • [claimed-docs] “It scans your Project with Snyk, generates a fix plan enriched with Snyk security intelligence, applies the fixes, and verifies the result, …”
   • [claimed-docs] “Run /snyk-fix in your coding assistant to scan the project and apply a fix for the top vulnerability.”
   • [claimed-docs] “DeepCode AI powers 85%-accurate security autofixes and comprehensive app coverage, letting developers build fast while staying secure.”
   • [community] “I'm not convinced their tools are better than npm audit + a license checker package... I've been very disappointed with their PR tools, and …”
   • [community] “Dependabot doesn't work well for a Fortune 50 company... Snyk like most security tools lacks perspective that takes business and how product…”

   proof ↗⚑ flag

   Gecko Security✓fullclaimed8/10

   Docs describe a full remediation loop: suggested patches with 'Request fix' opening a gecko/* PR, ability to commit fixes directly to an existing PR's head branch, and automatic re-scan marking 'Fix verified' once resolved — this goes beyond read-only reporting into applied/automatic fixes. missing for 10: independent/hands-on confirmation that generated fixes actually work correctly (community evidence focuses on detection false positives, not fix application quality).

   • [claimed-docs] “Click Request fix to open a gecko/* pull request with the patch applied, or apply the patch yourself.”
   • [claimed-docs] “Gecko rechecks the finding and marks it Fix verified once the vulnerability is gone.”
   • [claimed-docs] “Commit to an existing PR: For findings from a PR scan, Gecko can commit the fix directly to the PR's head branch so the fix rides along with…”
   • [claimed-docs] “PR reviews & fixes: On pull requests, a security review summary and one-click fixes.”
   • [claimed-docs] “Click **Request fix** to open a `gecko/*` pull request with the patch applied, or apply the patch yourself.”
   • [claimed-docs] “When the fix merges, Gecko rechecks the finding and marks it **Fix verified** once the vulnerability is gone.”

   proof ↗⚑ flag

Custom detection

1. security-engineerDefine custom detection rules for my organization's internal token and secret formats

   weight 2 · round drawn
   Snyk—none0/10

   The evidence pack covers Snyk's SAST, SCA, container, IaC, and agentic remediation features but contains no mention of secret detection or custom secret/token pattern rules; the story requires a capability not evidenced anywhere in the pack.

   ⚑ flag

   Gecko Security—none0/10

   Gecko Security is a vulnerability-scanning tool focused on code security review (SAST-style findings, PR checks, fixes), and none of the evidence mentions custom secret/token pattern detection rules or any mechanism for defining organization-specific secret formats.

   ⚑ flag

Secret scanning

1. developerScan a repository and its full git history for hardcoded secrets and credentials

   weight 3 · round drawn
   Snyk—none0/10

   The evidence pack covers Snyk Open Source (dependency vulnerabilities), Snyk Code (SAST), Snyk Container, and Snyk IaC, but contains no mention of secret/credential detection or git-history scanning for hardcoded secrets. This is a fair capability to expect from an AppSec scanning platform, but no evidence shows Snyk offers it.

   • [claimed-docs] “Scan, prioritize, and fix vulnerabilities in your code, open-source dependencies, container images, and cloud configurations.”
   • [claimed-docs] “Snyk Code is a developer-first static application security testing (SAST) solution. By scanning code in real-time and providing actionable i…”
   • [claimed-docs] “Snyk Open Source allows you to find and fix vulnerabilities in the open-source libraries used by your applications.”
   • [claimed-docs] “Snyk Container provides tools and integrations to quickly find and fix vulnerabilities. This allows you to create images that have security …”
   • [claimed-docs] “With Snyk IaC, you can: Write secure configurations for HashiCorp Terraform, AWS CloudFormation, Kubernetes, and Azure Resource Manager (ARM…”

   proof ↗⚑ flag

   Gecko Security—none0/10

   Gecko's documented capability is AI-driven vulnerability discovery (source-to-sink call chains, CVSS scoring, PoCs, patches) for application code, not secret/credential scanning or git-history scanning. Missing for 10: any mention of secret/credential detection, hardcoded key scanning, or scanning git commit history for leaked secrets.

   • [claimed-docs] “Each finding includes a severity (CVSS 4.0), a confidence score, the full source-to-sink call chain, a proof of concept, and a suggested pat…”
   • [claimed-docs] “Repository wiki: An AI-written map of your app: architecture, routing, and security model.”
   • [claimed-docs] “Compiler-accurate (precise cross-file analysis) | TypeScript · JavaScript · Python · Go · Java · Scala · C# · Rust”
   • [probe] “PROBE llms.txt: HTTP 200 at https://gecko.security/llms.txt # Gecko Security > Gecko Security is an AI security engineer that finds exploit…”

   proof ↗⚑ flag

Licenses

1. devops-leadDetect the licenses of my dependencies and enforce a license policy in scans

   weight 1 · round to Snyk
   Snyk~partialclaimed6/10

   Snyk Open Source explicitly scans dependency licenses for compliance against known licenses (snyk-docs-20), and the platform advertises general 'Policy management' features (snyk-docs-32), implying license policy enforcement capability. However, the evidence never details concrete license-policy enforcement mechanics (e.g., failing builds/PRs on license violations, custom license policy rules) and lacks independent/hands-on corroboration of this specific workflow. Missing for 10: explicit enforcement workflow details (build/PR gating on license violations), documented custom license policy configuration, and independent confirmation of license-policy enforcement in practice.

   • [claimed-docs] “Snyk Open Source can also scan your Projects for license compliance, checking against licenses known to Snyk.”
   • [claimed-docs] “Policy management”
   • [claimed-docs] “By using Snyk Open Source, you can reduce the risks introduced by open-source components.”

   proof ↗⚑ flag

   Gecko Security—none0/10

   Gecko is positioned as an AI vulnerability/exploit scanner (SAST-style call-chain analysis, patch suggestions) with no mention anywhere in docs, changelog, or API reference of dependency license detection, SBOM generation, or license policy enforcement.

   ⚑ flag

Sbom

1. security-engineerGenerate a software bill of materials (SPDX or CycloneDX) for my project or image

   weight 2 · round drawn
   Snyk—none0/10

   The evidence pack contains no mention of SBOM generation, SPDX, or CycloneDX anywhere across Snyk's docs, CLI, or MCP materials — only vulnerability scanning, fixing, and prioritization features are documented. Missing for 10: any documentation or CLI command output referencing SBOM export, SPDX format, or CycloneDX format.

   ⚑ flag

   Gecko Security—none0/10

   Gecko Security is an AI vulnerability scanner focused on code review, exploit findings, and auto-fix PRs; no evidence anywhere mentions generating an SBOM in SPDX or CycloneDX format for a project or container image.

   ⚑ flag

Ai triage

1. ai-native userUse AI to triage findings — explaining them, deduplicating, and flagging likely false positives

   weight 3 · round to Snyk
   Snyk~partialcommunity5/10

   Snyk documents AI-driven risk scoring (reachability, exploit maturity, popularity) and a Remediation Agent that scans, explains, and applies fixes, which supports explaining/prioritizing findings, but there is no explicit evidence of AI-based deduplication of findings or automated false-positive flagging — only manual 'ignore' suppression is documented. Community feedback (snyk-comm-10) also notes that legacy alerts remain largely 'noise' without context-aware triage, tempering confidence in how well false positives are actually filtered. Missing for 10: explicit deduplication feature, explicit AI false-positive detection/flagging, and independent hands-on confirmation that noise is meaningfully reduced.

   • [claimed-docs] “Sort by and prioritize the more important issues by incorporating factors such as issue prevalence, ease of fix, and risk factor into a sing…”
   • [claimed-docs] “DeepCode AI supports Snyk’s context-aware risk scoring, assessing the popularity of the affected package and the reachability of vulnerable …”
   • [claimed-docs] “The Remediation Agent is an LLM-driven tool that automates vulnerability remediation in your projects. It scans your Project with Snyk, gene…”
   • [claimed-docs] “Configure Snyk to ignore suggested fixes for an issue to suppress specific warnings.”
   • [community] “Until these alerts get better at understanding context of use of a dependency, they're going to remain mostly noise... No, I don't care abou…”

   proof ↗⚑ flag

   Gecko Security!disputedcontradicted4/10

   Docs describe confidence scores, severity, call chains, and PR-introduced vs pre-existing separation, plus an AI-tools API explicitly for triaging findings (gecko-security-docs-2, -9, -28), which maps to explaining/deduplicating/flagging false positives. However, hands-on community reports directly contradict the false-positive flagging claim: the tool assigned 100% confidence to two non-existent vulnerabilities and flagged an implausible 'RCE' in a CTF IRC bot script as high severity, while missing a real issue found by another scanner (gecko-security-comm-3, gecko-security-comm-5, gecko-security-comm-4). Missing for 10: independent validation that confidence scoring reliably suppresses false positives, and resolution of the reported inaccurate high-confidence findings.

   • [claimed-docs] “Each finding includes a severity (CVSS 4.0), a confidence score, the full source-to-sink call chain, a proof of concept, and a suggested pat…”
   • [claimed-docs] “Connect it to an AI tool and the tool can list your repositories and scans, pull full vulnerability evidence (proof of concept, call chain, …”
   • [claimed-docs] “Findings introduced by the PR are separated from pre-existing ones, so reviewers focus on what the change adds.”
   • [community] “Tool gave 100% confidence for two vulnerabilities (severity 9.4 and 6.5) that turned out not to be real.”
   • [community] “Uploaded a project (sydbox) to scan; took 10 minutes and found only one vulnerability, an 'RCE: IRC Message Command Execution Bypass' in an …”
   • [community] “Ran Gecko on a C project after Semgrep AI flagged a minor real issue; Gecko found nothing. After removing the fix and recreating the project…”

   proof ↗⚑ flag

Fp management

1. security-engineerBaseline or ignore existing and false-positive findings so only new issues alert

   weight 3 · round to Gecko Security
   Snyk~partialclaimed5/10

   Snyk docs mention configuring Snyk to ignore suggested fixes to suppress specific warnings (snyk-docs-5) and 'Policy management' as a platform capability (snyk-docs-32), which map to ignoring/baselining findings so only new issues surface. However, there's no detailed evidence of a dedicated baseline/false-positive triage workflow, ignore-policy scope (temporary vs permanent, org-wide), or CLI/API commands for it. missing for 10: explicit baseline scan feature, false-positive marking UI/API, ignore policy scoping and audit trail, independent confirmation of noise reduction effectiveness.

   • [claimed-docs] “Configure Snyk to ignore suggested fixes for an issue to suppress specific warnings.”
   • [claimed-docs] “Policy management”

   proof ↗⚑ flag

   Gecko Security~partialcommunity6/10

   Docs show PR scans separate pre-existing findings from new ones so reviewers focus on what changed (docs-28), and the API supports 'triage findings' (docs-10), implying some noise-reduction workflow. However, there's no explicit documentation of a false-positive marking/ignore mechanism or a persistent baseline snapshot feature, and community reports flag inaccurate high-confidence findings (comm-3) without confirming a suppression workflow exists. missing for 10: explicit 'mark as false positive'/dismiss UI, documented baseline snapshotting across scans, independent confirmation that dismissed findings stay suppressed on rescans.

   • [claimed-docs] “Findings introduced by the PR are separated from pre-existing ones, so reviewers focus on what the change adds.”
   • [claimed-docs] “Gecko's v1 API lets you trigger scans, read scan, repository, and vulnerability data, triage findings, manage scan schedules, rotate your Gi…”
   • [claimed-docs] “create a ticket in Jira, Linear, ClickUp, or Shortcut”
   • [community] “Tool gave 100% confidence for two vulnerabilities (severity 9.4 and 6.5) that turned out not to be real.”

   proof ↗⚑ flag

2. developerFilter and gate on findings by severity or confidence so low-value noise does not block builds

   weight 2 · round drawn
   Snyk~partialclaimed6/10

   Docs show severity reporting per issue (snyk-gh-6), risk-based prioritization scoring (snyk-docs-4, snyk-docs-17, snyk-docs-36), policy management (snyk-docs-32), and the ability to ignore/suppress specific findings (snyk-docs-5), which together support filtering low-value noise. However, there is no explicit documentation of a CI/CD severity-threshold gate flag or confidence-level filtering mechanism in the evidence pack. Missing for 10: explicit severity-threshold CLI/CI gating docs, confidence-score filtering mechanism, independent hands-on verification of gating behavior.

   • [github] “The report shows the vulnerabilities Snyk found in the package. For each issue found, Snyk reports the severity of the issue, provides a lin…”
   • [claimed-docs] “Sort by and prioritize the more important issues by incorporating factors such as issue prevalence, ease of fix, and risk factor into a sing…”
   • [claimed-docs] “Risk-based prioritization – – Reporting dashboard”
   • [claimed-docs] “DeepCode AI supports Snyk’s context-aware risk scoring, assessing the popularity of the affected package and the reachability of vulnerable …”
   • [claimed-docs] “Policy management”
   • [claimed-docs] “Configure Snyk to ignore suggested fixes for an issue to suppress specific warnings.”

   proof ↗⚑ flag

   Gecko Security~partialclaimed6/10

   Docs confirm a 'fail-on severity' threshold for PR checks (e.g., fail on High and above) and that findings carry both severity and confidence scores, supporting build-gating by severity. However, there's no documented mechanism to gate/filter specifically by confidence score, only severity threshold is described as configurable. Missing for 10: explicit confidence-based filtering/gating controls, and independent verification that severity gating reliably suppresses low-value noise.

   • [claimed-docs] “Each finding includes a severity (CVSS 4.0), a confidence score, the full source-to-sink call chain, a proof of concept, and a suggested pat…”
   • [claimed-docs] “**Fail-on severity**: the severity threshold that fails the check (for example, fail on High and above).”
   • [claimed-docs] “Findings introduced by the PR are separated from pre-existing ones, so reviewers focus on what the change adds.”

   proof ↗⚑ flag