Statsig vs GrowthBook

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round drawn
   Statsig✓fullprobed9/10

   Statsig publishes a working llms.txt (probed live, HTTP 200) and offers a dedicated Docs MCP server that lets AI clients read documentation directly, confirmed via a full keyless initialize handshake at /api/mcp. This directly satisfies pointing an agent at agent-oriented docs. Missing for 10: independent third-party confirmation of agents actually using llms.txt in practice beyond the probe.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.statsig.com/llms.txt # Statsig Documentation > Official documentation for Statsig feature flags, e…”
   • [claimed-docs] “Use the Statsig Docs MCP server to let supported AI clients read public Statsig documentation directly from /api/mcp.”
   • [claimed-docs] “Use the Statsig Docs MCP server to let supported AI clients read public Statsig documentation directly from /api/mcp”
   • [probe] “PROBE runtime (recorded 2026-09-05): the documented no-auth docs MCP server at https://docs.statsig.com/api/mcp completed a FULL keyless ini…”

   proof ↗⚑ flag

   GrowthBook✓fullprobed9/10

   A probe confirms llms.txt is live at docs.growthbook.io/llms.txt returning HTTP 200 with a structured index of 180 documentation pages, exactly matching the agent-oriented docs pattern; GrowthBook also documents an official MCP server for agent integration, reinforcing agent-friendliness. Missing for 10: no independent third-party confirmation of an agent successfully consuming llms.txt end-to-end beyond the probe.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.growthbook.io/llms.txt # GrowthBook Documentation - [Docs (180 pages)](https://docs.growthbook.io/…”
   • [claimed-docs] “Connect AI tools like Cursor, VS Code, and Claude to GrowthBook using the Model Context Protocol (MCP).”
   • [claimed-docs] “The official GrowthBook MCP server is a thin bridge: it loads GrowthBook Agent Skills (workflows and guardrails) and makes authenticated cal…”
   • [probe] “official MCP server documented at https://docs.growthbook.io/integrations/mcp”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round drawn
   Statsig✓fullprobed8/10

   Statsig ships an official CLI (@statsig/siggy) that runs keylessly and is explicitly documented and probe-verified for CI/CD automation of Feature Gates and Experiments, plus a Console API/HTTP API for headless programmatic control. Runtime probes confirm the CLI executes via npx in a scriptable, non-interactive way suitable for automation pipelines. Missing for 10: no independent third-party CI pipeline case study or deeper docs on CLI command coverage beyond gates/experiments.

   • [claimed-docs] “The Statsig CLI is a command-line interface that helps manage Feature Gates, Experiments, and Dynamic Configs within the Statsig platform.”
   • [claimed-docs] “You can integrate the CLI into CI/CD pipelines to automate the management of Feature Gates and Experiments.”
   • [claimed-docs] “The Console API is the CRUD API for performing the actions available on console.statsig.com without using the web UI.”
   • [claimed-docs] “Overview of the Statsig HTTP API for retrieving feature gate, experiment, and dynamic config values and logging events directly without an S…”
   • [probe] “official CLI documented at https://docs.statsig.com/statsigcli/introduction”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Statsig CLI (npm @statsig/siggy) ran keylessly via `npx -y @statsig/siggy --version`, prin…”

   proof ↗⚑ flag

   GrowthBook✓fullprobed8/10

   GrowthBook supports headless/CI usage via self-hosted Docker Compose deployment, a full REST API, SDK Connection Endpoints for programmatic flag evaluation, and inline server-side experiments requiring no third-party requests — all scriptable without a UI. A runtime probe confirms a real keyless self-host create-and-evaluate loop via API/docker, demonstrating genuine automation capability. missing for 10: no explicit CI/CD pipeline example or GitHub Actions integration doc, and no evidence of a dedicated CLI tool for automation workflows.

   • [claimed-docs] “Run GrowthBook on your own infrastructure with Docker Compose, including MongoDB, the API, and the front-end.”
   • [claimed-docs] “GrowthBook offers a full REST API for interacting with the application.”
   • [claimed-docs] “You can also run server-side experiments by using inline experiments directly with our SDK. This requires no 3rd party requests”
   • [claimed-docs] “The SDK Connection Endpoint provides readonly access to a subset of your feature flag data, just enough for the GrowthBook SDKs to assign va…”
   • [claimed-docs] “You can use docker compose to get started quickly... run docker compose up -d to start everything and view the app at http://localhost:3000”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official growthbook/growthbook docker image boot…”

   proof ↗⚑ flag

3. ai-native userConnect an agent via an official MCP server

   weight 3 · round drawn
   Statsig✓fullprobed9/10

   Statsig ships an official MCP server integrating with agent tools like Codex, Cursor, and Claude Code for querying Statsig data, plus a separate docs MCP server; runtime probes confirm both endpoints are live, with the main MCP server enforcing OAuth and the docs server completing a full handshake. missing for 10: independent third-party (non-vendor) hands-on report of using the MCP server in a real agent workflow beyond docs/probes.

   • [claimed-docs] “The Statsig MCP (Model Context Protocol) server brings the power of Statsig into tools like Codex, Cursor, and Claude Code. With this setup,…”
   • [claimed-docs] “The Statsig MCP (Model Context Protocol) server brings the power of Statsig into tools like Codex, Cursor, and Claude Code.”
   • [claimed-docs] “Use the Statsig Docs MCP server to let supported AI clients read public Statsig documentation directly from /api/mcp.”
   • [probe] “official MCP server documented at https://docs.statsig.com/integrations/mcp/overview”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.statsig.com/v1/mcp returned HTTP …”
   • [probe] “PROBE runtime (recorded 2026-09-05): the documented no-auth docs MCP server at https://docs.statsig.com/api/mcp completed a FULL keyless ini…”

   proof ↗⚑ flag

   GrowthBook✓fullprobed9/10

   GrowthBook documents an official MCP server that bridges AI tools (Cursor, VS Code, Claude) to GrowthBook via authenticated REST calls with Agent Skills/guardrails, and a runtime probe confirms the official npm package (@growthbook/mcp) completes a full stdio initialize handshake exposing read/write meta-tools. This is strong first-party plus hands-on evidence of a working official MCP server. Missing for 10: broader independent/community corroboration beyond the single runtime probe.

   • [claimed-docs] “Connect AI tools like Cursor, VS Code, and Claude to GrowthBook using the Model Context Protocol (MCP).”
   • [claimed-docs] “The official GrowthBook MCP server is a thin bridge: it loads GrowthBook Agent Skills (workflows and guardrails) and makes authenticated cal…”
   • [probe] “official MCP server documented at https://docs.growthbook.io/integrations/mcp”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official npm MCP server (`npx -y @growthbook/mcp`) completed a FULL keyless stdio initialize handsh…”

   proof ↗⚑ flag

4. ai-native userUse an official CLI

   weight 2 · round to Statsig
   Statsig✓fullprobed9/10

   Statsig ships an official CLI (Statsig CLI / @statsig/siggy) for managing Feature Gates, Experiments, and Dynamic Configs, documented for CI/CD automation, and a runtime probe confirms it executes keylessly via npx and is scriptable. missing for 10: no independent (non-Statsig) hands-on review of the CLI's full feature set beyond the version-check probe.

   • [claimed-docs] “The Statsig CLI is a command-line interface that helps manage Feature Gates, Experiments, and Dynamic Configs within the Statsig platform.”
   • [claimed-docs] “You can integrate the CLI into CI/CD pipelines to automate the management of Feature Gates and Experiments.”
   • [probe] “official CLI documented at https://docs.statsig.com/statsigcli/introduction”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Statsig CLI (npm @statsig/siggy) ran keylessly via `npx -y @statsig/siggy --version`, prin…”

   proof ↗⚑ flag

   GrowthBook—none0/10

   The evidence pack documents SDKs, REST API, self-host Docker, and an official MCP server, but there is no mention anywhere of an official standalone CLI tool for GrowthBook. Since a platform like GrowthBook could plausibly ship a CLI, the axis applies, but with no supporting evidence the verdict is none.

   ⚑ flag

5. ai-native userDrive the product through a documented public API

   weight 3 · round drawn
   Statsig✓fullprobed9/10

   Statsig documents multiple public API surfaces (HTTP API for gate/experiment retrieval and event logging, full CRUD Console API, and CLI) that let developers/agents drive the product programmatically, and runtime probes confirm the CLI works keylessly and the MCP servers respond live. missing for 10: a discoverable OpenAPI/Swagger spec (probe found 404s on standard OpenAPI paths), which would give full machine-readable API discoverability.

   • [claimed-docs] “Overview of the Statsig HTTP API for retrieving feature gate, experiment, and dynamic config values and logging events directly without an S…”
   • [claimed-docs] “The Console API is the CRUD API for performing the actions available on console.statsig.com without using the web UI.”
   • [claimed-docs] “The Statsig CLI is a command-line interface that helps manage Feature Gates, Experiments, and Dynamic Configs within the Statsig platform.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Statsig CLI (npm @statsig/siggy) ran keylessly via `npx -y @statsig/siggy --version`, prin…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.statsig.com/openapi.json, https://docs.statsig.com/swagger.json, https://docs.statsig.c…”

   proof ↗⚑ flag

   GrowthBook✓fullprobed9/10

   GrowthBook documents a full REST API for interacting with the application (growthbook-docs-12), including a readonly SDK Connection Endpoint (growthbook-docs-20) and unauthenticated public endpoints (growthbook-docs-41), and a runtime probe confirms an actual API-driven create-and-evaluate loop (creating org, feature flag, SDK connection, and reading it back) worked end-to-end (growthbook-probe-rt-2). Missing for 10: a discoverable OpenAPI/swagger spec file (all candidate paths 404'd per growthbook-probe-2), which would make the API more machine-discoverable for agentic tooling.

   • [claimed-docs] “GrowthBook offers a full REST API for interacting with the application.”
   • [claimed-docs] “The SDK Connection Endpoint provides readonly access to a subset of your feature flag data, just enough for the GrowthBook SDKs to assign va…”
   • [claimed-docs] “They are meant to be public and do not require authentication to view.”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official growthbook/growthbook docker image boot…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.growthbook.io/openapi.json, https://docs.growthbook.io/swagger.json, https://docs.growt…”

   proof ↗⚑ flag

6. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round to GrowthBook
   Statsig~partialprobed3/10

   Statsig documents general access-management/SSO/SCIM controls and shows its hosted MCP server requires an OAuth bearer token rather than being fully open (statsig-probe-rt-2), implying some authenticated, permissioned access surface an agent could use. However there is no explicit documentation of issuing scoped or least-privilege API keys/tokens specifically for agent use (e.g., read-only or project-scoped Console API keys). Missing for 10: explicit docs on creating scoped/limited-permission API keys, role-based key scoping for agents, and any guidance on least-privilege credential issuance for AI agents.

   • [claimed-docs] “In many enterprise environments, organizations use SSO and SCIM together to improve security and usability.”
   • [claimed-docs] “Basic settings include automatically adding new users with the same email domain to your project. Statsig also supports SSO to simplify invi…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.statsig.com/v1/mcp returned HTTP …”

   proof ↗⚑ flag

   GrowthBook~partialprobed5/10

   GrowthBook documents a read-only SDK Connection Endpoint that exposes only a subset of feature-flag data (least-privilege by design) and a full REST API, plus an 'Advanced access control' / 'Advanced permissioning' feature at higher pricing tiers, and the MCP server makes authenticated REST calls on behalf of an agent. However there is no explicit documentation of an admin flow for issuing a scoped, least-privilege API credential specifically for an AI agent (e.g., role-restricted API keys, token scoping UI, or agent-specific key generation). Missing for 10: documented API-key scoping/role assignment workflow, explicit agent-credential issuance guide, and evidence of granular per-tool permission limits for MCP/agent tokens beyond generic 'advanced access control' tier feature.

   • [claimed-docs] “The SDK Connection Endpoint provides readonly access to a subset of your feature flag data, just enough for the GrowthBook SDKs to assign va…”
   • [claimed-docs] “They are meant to be public and do not require authentication to view.”
   • [claimed-docs] “GrowthBook offers a full REST API for interacting with the application.”
   • [claimed-docs] “Pro, plus: Custom environments Ramp schedules Approval workflows Advanced access control SSO & SCIM provisioning Exportable audit logs”
   • [claimed-docs] “Custom environments * Ramp schedules * Approval workflows * Advanced access control * SSO & SCIM provisioning * Exportable audit l…”
   • [claimed-docs] “The official GrowthBook MCP server is a thin bridge: it loads GrowthBook Agent Skills (workflows and guardrails) and makes authenticated cal…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official npm MCP server (`npx -y @growthbook/mcp`) completed a FULL keyless stdio initialize handsh…”

   proof ↗⚑ flag

7. ai-native userBuild against official SDKs

   weight 2 · round to GrowthBook
   Statsig✓fullclaimed7/10

   Statsig's docs explicitly describe SDK architecture (client vs. server evaluation, on-device eval SDKs, deterministic evaluation across platforms) and even offer an HTTP API for cases without an SDK, confirming a first-party SDK ecosystem that AI-native developers can build against. Missing for 10: a concrete list/matrix of supported languages/platforms and independent hands-on confirmation of SDK usage beyond docs.

   • [claimed-docs] “Evaluation in Statsig is deterministic. Given the same user object and the same state of the experiment or feature gate, Statsig always retu…”
   • [claimed-docs] “Statsig's client-side On-Device Eval SDKs provide an alternate client-side architecture. The device keeps the definition of each experiment …”
   • [claimed-docs] “Evaluation in Statsig is deterministic. Given the same user object and the same state of the experiment or feature gate, Statsig always retu…”
   • [claimed-docs] “Statsig's client-side On-Device Eval SDKs provide an alternate client-side architecture. The device keeps the definition of each experiment …”
   • [claimed-docs] “Overview of the Statsig HTTP API for retrieving feature gate, experiment, and dynamic config values and logging events directly without an S…”

   proof ↗⚑ flag

   GrowthBook✓fullcommunity9/10

   GrowthBook ships 24 official language-specific SDKs (React, Python, Android, iOS, etc.) with documented quickstart code, local evaluation, targeting, and inline experiments, and this is corroborated by community hands-on reports of fast SDK integration. missing for 10: independent benchmarking across all 24 SDKs and more recent third-party validation beyond a single HN thread.

   • [github] “24 SDKs including React, Python, Android, and iOS.”
   • [claimed-docs] “Get GrowthBook feature flags and experiments running in your app in a few lines of code, then pick a language-specific SDK.”
   • [claimed-docs] “const showNewFeature = useFeatureIsOn("new-feature");”
   • [claimed-docs] “const showNewFeature = useFeatureIsOn("new-feature"); const buttonColor = useFeatureValue("button-color", "blue");”
   • [claimed-docs] “You can also run server-side experiments by using inline experiments directly with our SDK. This requires no 3rd party requests”
   • [community] “Been using GrowthBook for a few weeks now... I was shocked at how much setup time it took to get a simple feature flag into our app with oth…”
   • [community] “At Miggos we have implemented Growth book and we got FF and A/B in our Vue frontend in a few hours. Excited to see the progress of the platf…”

   proof ↗⚑ flag

8. ai-native userSubscribe to events via webhooks

   weight 2 · round drawn
   Statsig—none0/10

   The evidence pack contains no mention of webhooks or event subscription mechanisms anywhere in Statsig's docs, community posts, or probes — only feature gates, experiments, CLI, MCP, and access management are covered. Webhooks are a plausible axis for a flagging/analytics platform, so absence of evidence yields none rather than na.

   ⚑ flag

   GrowthBook—none0/10

   The evidence pack covers SDKs, REST API, MCP integration, self-hosting, and experimentation features but contains no mention of webhooks or event subscription mechanisms anywhere, so there is no evidence GrowthBook supports this capability.

   ⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round to Statsig
   Statsig~partialprobed5/10

   Statsig's MCP server explicitly lets AI clients (Claude Code, Cursor, Codex) 'ask questions, explore experiments, and access your Statsig data using AI,' which is a form of AI-generated insight retrieval, and this is confirmed live via a working OAuth-gated MCP endpoint. However this is an external-agent integration rather than a native in-product AI insights/suggestions feature (e.g., no evidence of built-in AI-generated experiment summaries, anomaly explanations, or suggestion widgets inside the Statsig console itself). Missing for 10: native in-app AI-generated insights/copilot UI, evidence of proactive AI suggestions surfaced directly in the console rather than via external MCP client.

   • [claimed-docs] “The Statsig MCP (Model Context Protocol) server brings the power of Statsig into tools like Codex, Cursor, and Claude Code. With this setup,…”
   • [claimed-docs] “The Statsig MCP (Model Context Protocol) server brings the power of Statsig into tools like Codex, Cursor, and Claude Code.”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.statsig.com/v1/mcp returned HTTP …”
   • [probe] “official MCP server documented at https://docs.statsig.com/integrations/mcp/overview”

   proof ↗⚑ flag

   GrowthBook~partialprobed4/10

   GrowthBook ships an official MCP server that lets AI tools like Cursor, VS Code, and Claude connect and make authenticated calls to pull experiment/feature data and get GrowthBook 'Agent Skills' workflows/guardrails, which can produce AI-generated insights and suggestions — this was verified working in a runtime probe. However, this capability lives entirely outside the product (in an external AI client) rather than as a native in-app AI insights feature; GrowthBook's own results UI only shows statistical outputs like 'Chance to Win', not LLM-generated narrative insights. Missing for 10: a native in-product AI insights/summarization feature surfaced in the GrowthBook UI itself, and evidence of automated suggestion-generation (e.g., 'this metric moved because X') without requiring an external MCP client.

   • [claimed-docs] “Connect AI tools like Cursor, VS Code, and Claude to GrowthBook using the Model Context Protocol (MCP).”
   • [claimed-docs] “The official GrowthBook MCP server is a thin bridge: it loads GrowthBook Agent Skills (workflows and guardrails) and makes authenticated cal…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official npm MCP server (`npx -y @growthbook/mcp`) completed a FULL keyless stdio initialize handsh…”
   • [claimed-docs] “Chance to Win" tells you the probability that the variation is better. Anything above 95%”

   proof ↗⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Statsig
   Statsig~partialprobed6/10

   Statsig supports background-running automations like Scheduled Rollouts (gradual gate rollout without manual intervention) and Autotune (autonomously shifts traffic toward winning variants), plus CLI integration into CI/CD pipelines for automated gate/experiment management. These are real autonomous background mechanisms, but they are experimentation/rollout automations rather than general-purpose AI-agent automations, and there's no evidence of scheduling/triggering arbitrary agentic workflows. Missing for 10: evidence of general-purpose scheduled/triggered agent workflows beyond rollout/autotune, and independent verification that these run reliably unattended in production.

   • [claimed-docs] “Gradually deploy a feature over time by setting up a Feature Gate as a Scheduled Rollout.”
   • [claimed-docs] “Unlike a standard A/B test, Autotune shifts traffic toward the winning variant to maximize a target metric.”
   • [claimed-docs] “Autotune shifts traffic toward the winning variant to maximize a target metric.”
   • [claimed-docs] “You can integrate the CLI into CI/CD pipelines to automate the management of Feature Gates and Experiments.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Statsig CLI (npm @statsig/siggy) ran keylessly via `npx -y @statsig/siggy --version`, prin…”

   proof ↗⚑ flag

   GrowthBook~partialprobed4/10

   GrowthBook has some automated background behaviors (automatic stale flag detection, ramp schedules, and 'safe rollouts' that presumably auto-guard rollouts), and its MCP server lets AI tools make authenticated calls to the product, but these are on-demand agent actions or passive detection features, not evidence of an AI agent configuring a persistent job that runs autonomously in the background on a schedule/trigger without human involvement. Missing for 10: a documented scheduler/trigger system for AI-configured autonomous jobs, evidence of MCP-driven automations persisting and executing unattended, and independent confirmation of any autonomous background workflow.

   • [claimed-docs] “GrowthBook can automatically detect these flags so you can clean them up.”
   • [claimed-docs] “How rules override a feature's default value — targeting users, rolling out gradually, experimenting — and how to scope, ramp, and schedule …”
   • [claimed-docs] “Pro, plus: Custom environments Ramp schedules Approval workflows Advanced access control SSO & SCIM provisioning Exportable audit logs”
   • [claimed-docs] “Custom environments * Ramp schedules * Approval workflows * Advanced access control * SSO & SCIM provisioning * Exportable audit l…”
   • [claimed-docs] “Starter, plus: Visual editor Multi-arm bandits Safe rollouts Customizable dashboards Advanced permissioning Power calculator Premium support”
   • [claimed-docs] “The official GrowthBook MCP server is a thin bridge: it loads GrowthBook Agent Skills (workflows and guardrails) and makes authenticated cal…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official npm MCP server (`npx -y @growthbook/mcp`) completed a FULL keyless stdio initialize handsh…”

   proof ↗⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round drawn
   Statsig—none0/10

   Evidence covers Statsig's MCP server, which lets external AI tools (Cursor, Claude Code, Codex) query Statsig data — the opposite direction of an in-product assistant that users delegate tasks to. No evidence of a built-in AI assistant embedded in the Statsig console itself.

   ⚑ flag

   GrowthBook—none0/10

   GrowthBook documents an MCP server that lets external AI tools (Cursor, VS Code, Claude) connect to and query GrowthBook — this is the opposite direction (GrowthBook as a data/tool source for external assistants), not a built-in AI assistant living inside the GrowthBook product itself. No evidence anywhere in the pack shows an in-app chat/assistant feature for delegating tasks within GrowthBook's UI.

   • [claimed-docs] “Connect AI tools like Cursor, VS Code, and Claude to GrowthBook using the Model Context Protocol (MCP).”
   • [claimed-docs] “The official GrowthBook MCP server is a thin bridge: it loads GrowthBook Agent Skills (workflows and guardrails) and makes authenticated cal…”
   • [probe] “official MCP server documented at https://docs.growthbook.io/integrations/mcp”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official npm MCP server (`npx -y @growthbook/mcp`) completed a FULL keyless stdio initialize handsh…”

   proof ↗⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round to GrowthBook
   Statsig~partialprobed6/10

   Statsig ships a first-party MCP server (verified live via runtime probe) that lets AI clients like Claude Code and Cursor query experiments, gates, and data using natural language, which is the core mechanism for natural-language operation. However, this is an agentic/AI-integration layer bolted onto a console/CLI/API-driven product rather than a native NL command interface across the whole product surface. missing for 10: evidence of natural-language command support within the core Statsig console UI itself (not just via external MCP-connected agents), independent hands-on validation of NL command accuracy/reliability, and coverage of write-actions (creating/modifying gates or experiments) via natural language rather than read-only querying.

   • [claimed-docs] “The Statsig MCP (Model Context Protocol) server brings the power of Statsig into tools like Codex, Cursor, and Claude Code. With this setup,…”
   • [claimed-docs] “The Statsig MCP (Model Context Protocol) server brings the power of Statsig into tools like Codex, Cursor, and Claude Code.”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.statsig.com/v1/mcp returned HTTP …”
   • [probe] “official MCP server documented at https://docs.statsig.com/integrations/mcp/overview”

   proof ↗⚑ flag

   GrowthBook✓fullprobed8/10

   GrowthBook ships an official MCP server that bridges AI tools like Cursor, VS Code, and Claude to GrowthBook, loading 'Agent Skills' (workflows/guardrails) and making authenticated REST calls so users can operate flags/experiments via natural language in an AI client. A runtime probe confirms the official npm MCP server completes a full keyless handshake exposing read/write meta-tools covering the whole API, corroborating the docs' claims beyond marketing copy. Missing for 10: an end-to-end example of an actual natural-language command executing a real write action (only the handshake/tool-discovery was verified, not a full command-to-result flow) and independent (non-vendor) hands-on confirmation.

   • [claimed-docs] “Connect AI tools like Cursor, VS Code, and Claude to GrowthBook using the Model Context Protocol (MCP).”
   • [claimed-docs] “The official GrowthBook MCP server is a thin bridge: it loads GrowthBook Agent Skills (workflows and guardrails) and makes authenticated cal…”
   • [probe] “official MCP server documented at https://docs.growthbook.io/integrations/mcp”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official npm MCP server (`npx -y @growthbook/mcp`) completed a FULL keyless stdio initialize handsh…”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   Statsig—none0/10

   Statsig documents an HTTP API and Console API but there is no evidence of an interactive API reference with runnable/try-it examples; explicit probes for OpenAPI/Swagger endpoints returned 404s, indicating no such interactive explorer exists.

   • [claimed-docs] “Overview of the Statsig HTTP API for retrieving feature gate, experiment, and dynamic config values and logging events directly without an S…”
   • [claimed-docs] “The Console API is the CRUD API for performing the actions available on console.statsig.com without using the web UI.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.statsig.com/openapi.json, https://docs.statsig.com/swagger.json, https://docs.statsig.c…”

   proof ↗⚑ flag

   GrowthBook—none0/10

   GrowthBook documents a REST API (growthbook-docs-12) but there is no evidence of an interactive, runnable API reference (e.g., Swagger/OpenAPI try-it console) — a direct probe for openapi/swagger specs returned 404 on all candidate paths, and no docs mention runnable code samples in an API explorer.

   • [claimed-docs] “GrowthBook offers a full REST API for interacting with the application.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.growthbook.io/openapi.json, https://docs.growthbook.io/swagger.json, https://docs.growt…”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   Statsig—none0/10

   Statsig documents a Console API and HTTP API but a direct probe for OpenAPI/swagger specs at common paths (openapi.json, swagger.json, api/openapi.json, .well-known/openapi.json) all returned 404, and no docs reference a downloadable machine-readable spec.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.statsig.com/openapi.json, https://docs.statsig.com/swagger.json, https://docs.statsig.c…”
   • [claimed-docs] “The Console API is the CRUD API for performing the actions available on console.statsig.com without using the web UI.”
   • [claimed-docs] “Overview of the Statsig HTTP API for retrieving feature gate, experiment, and dynamic config values and logging events directly without an S…”

   proof ↗⚑ flag

   GrowthBook—none0/10

   While GrowthBook documents a full REST API (growthbook-docs-12) and an SDK Connection Endpoint (growthbook-docs-20), the evidence pack shows an explicit probe for OpenAPI/Swagger spec files at expected paths that all returned 404, and no other evidence surfaces a downloadable OpenAPI or machine-readable spec.

   • [claimed-docs] “GrowthBook offers a full REST API for interacting with the application.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.growthbook.io/openapi.json, https://docs.growthbook.io/swagger.json, https://docs.growt…”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round to GrowthBook
   Statsig—none0/10

   The evidence pack documents feature gates, experiments, CLI, and MCP integrations, but nothing describes a distinct sandbox/staging environment or project isolated from production data for testing purposes — only production-oriented tools like emergency disable switches, gate overrides, and scheduled rollouts are mentioned. This is a fair question for a feature-flag/experimentation platform, but no evidence confirms a sandbox capability exists.

   • [claimed-docs] “Feature Gates, commonly known as feature flags, allow you to toggle the behavior of your product in real time without deploying new code.”
   • [claimed-docs] “Have an emergency disable switch that lets you immediately turn off a particular code branch for users in production”
   • [claimed-docs] “After your Feature Gate is live, view Feature Gate exposures in the Statsig console to monitor who encounters your gate.”
   • [claimed-docs] “set up Feature Gate overrides to allow specific users to bypass the gate.”
   • [claimed-docs] “Basic settings include automatically adding new users with the same email domain to your project. Statsig also supports SSO to simplify invi…”

   proof ↗⚑ flag

   GrowthBook✓fullprobed8/10

   GrowthBook supports separate environments (dev/staging/production) for feature flags, self-hosting via Docker Compose for a fully isolated instance, and a verified runtime probe confirms a keyless self-host roundtrip creating and evaluating flags without any production account/license — enabling true sandbox testing. missing for 10: no explicit vendor guidance on a dedicated 'sandbox mode' distinct from staging environments, and no independent (non-vendor) confirmation of environment data isolation guarantees.

   • [claimed-docs] “Use environments such as dev, staging, and production to enable or disable feature flags independently.”
   • [claimed-docs] “Run GrowthBook on your own infrastructure with Docker Compose, including MongoDB, the API, and the front-end.”
   • [claimed-docs] “You can use docker compose to get started quickly... run docker compose up -d to start everything and view the app at http://localhost:3000”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official growthbook/growthbook docker image boot…”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Statsig—none0/10

   Statsig documents an HTTP API and Console API but the evidence pack contains no mention of API versioning scheme or a documented deprecation policy; a probe for OpenAPI spec files (which would typically reveal version info) returned 404s across all candidate paths.

   • [claimed-docs] “Overview of the Statsig HTTP API for retrieving feature gate, experiment, and dynamic config values and logging events directly without an S…”
   • [claimed-docs] “The Console API is the CRUD API for performing the actions available on console.statsig.com without using the web UI.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.statsig.com/openapi.json, https://docs.statsig.com/swagger.json, https://docs.statsig.c…”

   proof ↗⚑ flag

   GrowthBook—none0/10

   GrowthBook documents a REST API and SDK Connection Endpoint, but no evidence pack item describes API versioning scheme or a documented deprecation policy; the openapi/swagger probe returned 404s, suggesting no formal API spec is even discoverable. Missing for 10: any mention of API version numbers, backward-compatibility guarantees, or a deprecation/sunset policy.

   • [claimed-docs] “GrowthBook offers a full REST API for interacting with the application.”
   • [claimed-docs] “The SDK Connection Endpoint provides readonly access to a subset of your feature flag data, just enough for the GrowthBook SDKs to assign va…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.growthbook.io/openapi.json, https://docs.growthbook.io/swagger.json, https://docs.growt…”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Statsig
   Statsig~partialprobed6/10

   Statsig's Console API provides a CRUD API for managing gates, experiments, and configs programmatically, and the CLI (confirmed runnable via npx) can be scripted/integrated into CI/CD, enabling automation across many items rather than one-by-one UI clicks. However, no evidence explicitly documents a bulk/batch endpoint or bulk-edit feature, so bulk operations must be assembled by looping API/CLI calls rather than a native bulk primitive; missing for 10: explicit bulk/batch API endpoints, documented multi-item transactional updates, and independent confirmation of large-scale automation use.

   • [claimed-docs] “The Console API is the CRUD API for performing the actions available on console.statsig.com without using the web UI.”
   • [claimed-docs] “The Statsig CLI is a command-line interface that helps manage Feature Gates, Experiments, and Dynamic Configs within the Statsig platform.”
   • [claimed-docs] “You can integrate the CLI into CI/CD pipelines to automate the management of Feature Gates and Experiments.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Statsig CLI (npm @statsig/siggy) ran keylessly via `npx -y @statsig/siggy --version`, prin…”

   proof ↗⚑ flag

   GrowthBook~partialprobed5/10

   GrowthBook exposes a full REST API (growthbook-docs-12) and an official MCP server that bridges AI tools to authenticated REST read/write calls across the whole API (growthbook-docs-21, growthbook-probe-rt-1), which could let an AI agent script operations across many flags/experiments programmatically. However, there is no documented bulk-specific endpoint, batch-update tool, or example of performing many-item operations in one call. Missing for 10: explicit bulk-create/bulk-update API endpoints, documented batch operation examples, and evidence of the MCP server or API actually handling many items in a single request rather than one-at-a-time calls.

   • [claimed-docs] “GrowthBook offers a full REST API for interacting with the application.”
   • [claimed-docs] “The official GrowthBook MCP server is a thin bridge: it loads GrowthBook Agent Skills (workflows and guardrails) and makes authenticated cal…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official npm MCP server (`npx -y @growthbook/mcp`) completed a FULL keyless stdio initialize handsh…”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to GrowthBook
   Statsig~partialclaimed4/10

   Statsig supports rule-based automation such as targeting rules on Feature Gates, Scheduled Rollouts, and Autotune (which automatically shifts traffic toward a winning variant based on live metrics) — these are forms of automatic rule-triggered behavior, but they are evaluation/targeting rules and traffic-shift algorithms rather than a general 'on event X, trigger action Y' automation engine. Missing for 10: documented event-triggered workflow/webhook automation (e.g., alert-to-action pipelines), explicit event-based trigger definitions distinct from targeting/eval rules, and independent hands-on evidence of such automation working end-to-end.

   • [claimed-docs] “Gradually deploy a feature over time by setting up a Feature Gate as a Scheduled Rollout.”
   • [claimed-docs] “Unlike a standard A/B test, Autotune shifts traffic toward the winning variant to maximize a target metric.”
   • [claimed-docs] “Schedule gradual feature rollouts to safely deploy new code”
   • [claimed-docs] “Autotune shifts traffic toward the winning variant to maximize a target metric.”
   • [claimed-docs] “set up Feature Gate overrides to allow specific users to bypass the gate.”
   • [claimed-docs] “Infra Analytics: Monitor and debug service health alongside product outcomes. Ingest metrics and traces with OpenTelemetry, search and group…”

   proof ↗⚑ flag

   GrowthBook~partialclaimed5/10

   GrowthBook lets users define feature-flag 'rules' that automatically apply based on targeting conditions, scheduling, and ramping (docs-22, docs-28, docs-2), and has automated stale-flag detection (docs-4) — these are rule-based automations tied to feature evaluation. However, this falls short of a general 'on event X, trigger action Y' automation engine; there's no evidence of webhook-driven arbitrary actions, external event ingestion, or an automation/workflow layer beyond flag targeting and rollout scheduling. Missing for 10: generic event-trigger/webhook automation outside feature-flag context, evidence of arbitrary action execution on external events, and independent corroboration of rule automation depth.

   • [claimed-docs] “How rules override a feature's default value — targeting users, rolling out gradually, experimenting — and how to scope, ramp, and schedule …”
   • [claimed-docs] “Add targeting conditions to a rule to control which values a user sees.”
   • [claimed-docs] “GrowthBook can automatically detect these flags so you can clean them up.”
   • [claimed-docs] “Add targeting conditions to a rule to control which values a user sees. Conditions are evaluated against attributes you pass into the SDK”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round drawn
   Statsig—none0/10

   Statsig only documents one-time 'Scheduled Rollouts' for feature gates and CLI/CI-CD scripting of flag management (statsig-docs-3, statsig-docs-25, statsig-docs-31); there is no evidence of a recurring job/workflow scheduler (e.g., cron-like recurring automation) that an AI-native user could set up. missing for 10: any documented recurring/cron-style job scheduling, workflow orchestration, or automated recurring task execution beyond a single scheduled feature rollout.

   • [claimed-docs] “Gradually deploy a feature over time by setting up a Feature Gate as a Scheduled Rollout.”
   • [claimed-docs] “Schedule gradual feature rollouts to safely deploy new code”
   • [claimed-docs] “You can integrate the CLI into CI/CD pipelines to automate the management of Feature Gates and Experiments.”

   proof ↗⚑ flag

   GrowthBook—none0/10

   Evidence only shows one-time scheduling of feature-flag rule start/stop dates and 'ramp schedules' for gradual rollouts (growthbook-docs-22, growthbook-docs-37) — nothing resembling recurring cron-like jobs or workflow automation that an AI-native user could schedule to run repeatedly. No mention of scheduled reports, recurring experiment analyses, or workflow automation triggers.

   • [claimed-docs] “How rules override a feature's default value — targeting users, rolling out gradually, experimenting — and how to scope, ramp, and schedule …”
   • [claimed-docs] “Custom environments * Ramp schedules * Approval workflows * Advanced access control * SSO & SCIM provisioning * Exportable audit l…”
   • [claimed-docs] “Pro, plus: Custom environments Ramp schedules Approval workflows Advanced access control SSO & SCIM provisioning Exportable audit logs”

   proof ↗⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round to GrowthBook
   Statsig~partialclaimed4/10

   Statsig's feature gates support rollback-like controls (emergency disable switch, scheduled rollouts) and CLI/Console API can be scripted into CI/CD for programmatic management, implying some versioned control over flag state, but the evidence never documents an explicit audit/version history, diffing, or approval-review workflow for these 'automations'. missing for 10: explicit version history/audit log for gate or experiment changes, a review/approval workflow before changes go live, and a documented one-click rollback to a prior version (only an on/off disable switch is shown).

   • [claimed-docs] “Have an emergency disable switch that lets you immediately turn off a particular code branch for users in production”
   • [claimed-docs] “Gradually deploy a feature over time by setting up a Feature Gate as a Scheduled Rollout.”
   • [claimed-docs] “The Statsig CLI is a command-line interface that helps manage Feature Gates, Experiments, and Dynamic Configs within the Statsig platform.”
   • [claimed-docs] “You can integrate the CLI into CI/CD pipelines to automate the management of Feature Gates and Experiments.”
   • [claimed-docs] “The Console API is the CRUD API for performing the actions available on console.statsig.com without using the web UI.”
   • [claimed-docs] “After your Feature Gate is live, view Feature Gate exposures in the Statsig console to monitor who encounters your gate.”

   proof ↗⚑ flag

   GrowthBook~partialcommunity6/10

   GrowthBook documents draft feature revisions, request-review, merge-conflict resolution, and required approval before publishing (docs-5, docs-22), and lists approval workflows plus exportable audit logs as paid-tier features (docs-37), which together enable versioning and review of flag/automation changes. However, no explicit rollback mechanism is documented and there's no independent/hands-on corroboration of the revision-history or rollback UI; a community comment even suggests config-as-code/git-based versioning isn't natively supported (growthbook-comm-11). Missing for 10: explicit rollback documentation, hands-on verification of revision history UI, and clarity on git-based config versioning.

   • [claimed-docs] “Draft feature revisions, request review, resolve merge conflicts, and require approval before publishing.”
   • [claimed-docs] “How rules override a feature's default value — targeting users, rolling out gradually, experimenting — and how to scope, ramp, and schedule …”
   • [claimed-docs] “Custom environments * Ramp schedules * Approval workflows * Advanced access control * SSO & SCIM provisioning * Exportable audit l…”
   • [community] “From the looks of it, it doesn't look like the configuration can be stored in the code repository itself. This is one of the key things to d…”

   proof ↗⚑ flag

Proxy

1. platform engineerRun a relay/edge proxy so flags stay served when the vendor is unreachable and SDK traffic stays inside my network

   weight 1 · round to GrowthBook
   Statsig—none0/10

   No evidence in the pack describes a self-hostable relay/edge proxy for offline or in-network flag serving; documentation covers SDKs, HTTP API, CLI, and MCP but nothing about a proxy component that caches flags locally when Statsig's servers are unreachable.

   ⚑ flag

   GrowthBook✓fullcommunity9/10

   GrowthBook explicitly documents a self-hostable Proxy that runs next to your app to cache SDK payloads, reduce latency, and keep serving flags if the API is down, directly matching the relay/edge proxy story, and this sits alongside full self-host Docker Compose deployment keeping SDK traffic in-network. Community evidence corroborates that evaluation happens locally within SDKs via a cacheable JSON payload, reinforcing offline resilience. Missing for 10: independent hands-on report specifically of the Proxy component (vs. general self-host) failing over during an outage.

   • [claimed-docs] “Run the GrowthBook Proxy next to your app to cache SDK payloads, reduce latency, and keep serving flags if the API is down.”
   • [claimed-docs] “Run GrowthBook on your own infrastructure with Docker Compose, including MongoDB, the API, and the front-end.”
   • [claimed-docs] “You can use docker compose to get started quickly... run docker compose up -d to start everything and view the app at http://localhost:3000”
   • [community] “With GrowthBook, there's a single cacheable JSON definition of all of the features and then evaluation happens locally within the SDKs... Th…”
   • [claimed-docs] “The SDK Connection Endpoint provides readonly access to a subset of your feature flag data, just enough for the GrowthBook SDKs to assign va…”

   proof ↗⚑ flag

Self host

1. platform engineerSelf-host the full flag platform from an open-source distribution, keeping evaluation data on my infrastructure

   weight 3 · round to GrowthBook
   Statsig—none0/10

   No evidence in the pack mentions an open-source self-hosted distribution of the full Statsig platform; all material describes the hosted SaaS console, SDKs, CLI, and MCP integrations that connect to Statsig's cloud API. Absence of evidence for this applicable axis means it is not delivered.

   ⚑ flag

   GrowthBook✓fullprobed9/10

   GrowthBook documents a self-host distribution via Docker Compose (MongoDB, API, front-end) and this is corroborated by a hands-on probe showing a full keyless self-host boot, feature creation, and SDK payload evaluation working end-to-end on local infrastructure, plus community confirmation of production self-hosted usage. missing for 10: independent third-party audit of data residency guarantees beyond vendor docs/probe.

   • [claimed-docs] “Run GrowthBook on your own infrastructure with Docker Compose, including MongoDB, the API, and the front-end.”
   • [claimed-docs] “You can use docker compose to get started quickly... run docker compose up -d to start everything and view the app at http://localhost:3000”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official growthbook/growthbook docker image boot…”
   • [community] “Growthbook is awesome! We have used the self-hosted version a few months in my company now, and are very happy about it. Talented people beh…”

   proof ↗⚑ flag

Agent ops

1. ai agentRead experiment configurations and results programmatically to summarize outcomes and recommend ship/rollback decisions

   weight 1 · round drawn
   Statsig✓fullprobed8/10

   Statsig provides a Console API (CRUD for experiments/gates), an HTTP API for retrieving experiment/gate values, a CLI scriptable in CI, and an official MCP server explicitly documented to let AI tools like Claude Code/Cursor/Codex 'ask questions, explore experiments, and access your Statsig data' — directly enabling an AI agent to programmatically read experiment configs/results. Runtime probes confirm the MCP server and CLI are live and reachable. Missing for 10: explicit documented example of an agent using these APIs to produce a ship/rollback recommendation (only general data-access capability is evidenced, not the recommendation workflow itself).

   • [claimed-docs] “The Console API is the CRUD API for performing the actions available on console.statsig.com without using the web UI.”
   • [claimed-docs] “Overview of the Statsig HTTP API for retrieving feature gate, experiment, and dynamic config values and logging events directly without an S…”
   • [claimed-docs] “The Statsig MCP (Model Context Protocol) server brings the power of Statsig into tools like Codex, Cursor, and Claude Code. With this setup,…”
   • [claimed-docs] “The Statsig CLI is a command-line interface that helps manage Feature Gates, Experiments, and Dynamic Configs within the Statsig platform.”
   • [claimed-docs] “You can integrate the CLI into CI/CD pipelines to automate the management of Feature Gates and Experiments.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Statsig CLI (npm @statsig/siggy) ran keylessly via `npx -y @statsig/siggy --version`, prin…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.statsig.com/v1/mcp returned HTTP …”

   proof ↗⚑ flag

   GrowthBook✓fullprobed8/10

   GrowthBook exposes a full REST API and an official MCP server that makes authenticated calls to that API, explicitly loading 'Agent Skills (workflows and guardrails)' so AI tools can query feature/experiment data — confirmed working end-to-end in a runtime probe. Experiment results expose statistical decision signals like 'Chance to Win' (>95% highlighted as a clear winner) that directly support ship/rollback recommendations, and metrics/fact-table docs show how results are computed from stored data for programmatic reading. Missing for 10: a concrete worked example of an agent actually generating a ship/rollback summary via MCP, and independent (non-vendor) confirmation of recommendation quality.

   • [claimed-docs] “GrowthBook offers a full REST API for interacting with the application.”
   • [claimed-docs] “Connect AI tools like Cursor, VS Code, and Claude to GrowthBook using the Model Context Protocol (MCP).”
   • [claimed-docs] “The official GrowthBook MCP server is a thin bridge: it loads GrowthBook Agent Skills (workflows and guardrails) and makes authenticated cal…”
   • [claimed-docs] “Chance to Win" tells you the probability that the variation is better. Anything above 95%”
   • [claimed-docs] “Chance to Win tells you the probability that the variation is better. Anything above 95% ... is highlighted green indicating a very clear wi…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official npm MCP server (`npx -y @growthbook/mcp`) completed a FULL keyless stdio initialize handsh…”
   • [probe] “official MCP server documented at https://docs.growthbook.io/integrations/mcp”

   proof ↗⚑ flag

Analysis

1. product managerTrust a documented statistics engine (Bayesian or frequentist, with variance-reduction options) behind experiment results

   weight 2 · round to GrowthBook
   Statsig~partialcommunity6/10

   Statsig documents a full experimentation suite with variance-reduction techniques (CUPED, stratified sampling, switchback tests) and hypothesis/metric scorecards, and community sentiment praises its 'stat rigor,' but the evidence never names or details whether the underlying engine is Bayesian, frequentist, or a documented hybrid methodology. missing for 10: explicit documentation of the statistical model (Bayesian vs frequentist), confidence-interval/p-value methodology details, and independent validation of statistical correctness beyond anecdotal praise.

   • [claimed-docs] “Statsig offers a full suite of experiment measurement tools, including CUPED, Stratified Sampling, Switchback Tests, and more.”
   • [claimed-docs] “The Scorecard feature makes this easy by letting you enter your hypothesis and select both primary and secondary metrics.”
   • [claimed-docs] “Statsig Warehouse Native is an experimentation platform that runs analysis directly in your data warehouse.”
   • [community] “Hats off to Statsig. They built a stellar product. Superior to many of their industry competitors like Optimizely. Back when I was on an int…”

   proof ↗⚑ flag

   GrowthBook✓fullcommunity9/10

   GrowthBook documents both Bayesian and Frequentist statistics engines with explicit UI outputs like 'Chance to Win', and its GitHub/community materials detail variance-reduction and rigor features (CUPED, Sequential testing, Post-Stratification, SRM checks) plus rationale for choosing Bayesian to mitigate peeking/Type I error. This is corroborated by both official docs and first-party community explanation, giving a PM strong documented trust in the underlying stats methodology; missing for 10: independent third-party statistical audit or academic validation of the engine's correctness.

   • [claimed-docs] “What you will see will depend a little bit on whether you are using our Bayesian or our Frequentist engine”
   • [claimed-docs] “Chance to Win" tells you the probability that the variation is better. Anything above 95%”
   • [claimed-docs] “Chance to Win tells you the probability that the variation is better. Anything above 95% ... is highlighted green indicating a very clear wi…”
   • [github] “World class experiment stats engine (CUPED, Sequential, Bayesian, Post-Strat, Bandits, SRM checks).”
   • [github] “🆎 World class experiment stats engine (CUPED, Sequential, Bayesian, Post-Strat, Bandits, SRM checks).”
   • [community] “We use a Bayesian statistics engine which is much more immune to peeking problems and Type I errors than frequentist approaches.”

   proof ↗⚑ flag

Experiments

1. product managerRun A/B and multivariate experiments on flags and see which variation wins on my metrics

   weight 3 · round to Statsig
   Statsig✓fullcommunity9/10

   Statsig's docs directly describe running A/B/n and multivariate experiments with primary/secondary metrics via the Scorecard, plus Layers for managing multiple experiments and Autotune for identifying/shifting toward winning variants, and community feedback corroborates its statistical rigor versus competitors. Missing for 10: independent hands-on validation of the results/analysis UI itself (e.g., screenshots or third-party review of the 'winning variation' scorecard output) rather than only docs.

   • [claimed-docs] “Statsig experimentation runs randomized controlled trials (A/B or A/B/n tests) that measure how product changes affect your key metrics.”
   • [claimed-docs] “The Scorecard feature makes this easy by letting you enter your hypothesis and select both primary and secondary metrics.”
   • [claimed-docs] “Unlike a standard A/B test, Autotune shifts traffic toward the winning variant to maximize a target metric.”
   • [claimed-docs] “Autotune shifts traffic toward the winning variant to maximize a target metric.”
   • [claimed-docs] “A Layer allows you to manage multiple experiments and feature flags together.”
   • [community] “Hats off to Statsig. They built a stellar product. Superior to many of their industry competitors like Optimizely. Back when I was on an int…”
   • [community] “Have implemented Statsig in two companies as alternatives to LaunchDarkly and loved it both times. Going to be interesting how their other b…”

   proof ↗⚑ flag

   GrowthBook✓fullcommunity8/10

   GrowthBook natively supports A/B and multivariate experiments attached to feature flags, with statistical engines (Bayesian/Frequentist, CUPED, Sequential, Bandits), templates for goal/secondary/guardrail metrics, and a results view showing 'Chance to Win' per variation to determine winners. Community reports corroborate quick setup and real-world use of feature flags plus A/B testing, though some note a gap between flags and full analytics requiring warehouse/metrics setup. missing for 10: independent hands-on validation of multivariate (>2 arm) experiment results and winner-declaration workflow specifically, beyond docs and community anecdotes.

   • [claimed-docs] “GrowthBook has a few different ways to run experiments or AB tests depending on your needs.”
   • [claimed-docs] “What you will see will depend a little bit on whether you are using our Bayesian or our Frequentist engine”
   • [claimed-docs] “Selecting a Template in the creation modal pre-fills settings like the data source, goal/secondary/guardrail metrics, targeting, and traffic”
   • [claimed-docs] “Define Fact Tables and metrics so GrowthBook can compute experiment results from your warehouse.”
   • [claimed-docs] “Chance to Win" tells you the probability that the variation is better. Anything above 95%”
   • [claimed-docs] “Run A/B tests with feature flags, the Visual Editor, URL redirects, or your own assignment, then analyze results in GrowthBook.”
   • [claimed-docs] “you can add an experiment rule to a feature that will randomly assign the users based on some hashing attribute into one of your experiment …”
   • [github] “World class experiment stats engine (CUPED, Sequential, Bayesian, Post-Strat, Bandits, SRM checks).”
   • [community] “With GrowthBook, there's a single cacheable JSON definition of all of the features and then evaluation happens locally within the SDKs... Th…”
   • [community] “We use a Bayesian statistics engine which is much more immune to peeking problems and Type I errors than frequentist approaches.”

   proof ↗⚑ flag

Metrics

1. product managerDefine experiment metrics from my own data — warehouse tables or ingested events — instead of a black-box metric store

   weight 2 · round to GrowthBook
   Statsig✓fullclaimed7/10

   Statsig Warehouse Native explicitly runs experiment analysis directly on the customer's own data warehouse rather than a black-box metric store, and Statsig's core event-ingestion pipeline lets PMs define metrics from ingested events with a full analysis toolkit (CUPED, stratified sampling, switchback tests). missing for 10: independent/hands-on corroboration of Warehouse Native in production and detail on the custom-metric-definition UI/workflow itself.

   • [claimed-docs] “Statsig Warehouse Native is an experimentation platform that runs analysis directly in your data warehouse.”
   • [claimed-docs] “Statsig offers a full suite of experiment measurement tools, including CUPED, Stratified Sampling, Switchback Tests, and more.”
   • [claimed-docs] “Statsig experimentation runs randomized controlled trials (A/B or A/B/n tests) that measure how product changes affect your key metrics.”
   • [claimed-docs] “Experiments enable you to run randomized controlled trials (A/B or A/B/n tests) to measure the impact of product changes on key metrics.”

   proof ↗⚑ flag

   GrowthBook✓fullcommunity8/10

   GrowthBook explicitly lets teams define Fact Tables and metrics computed directly from their own warehouse (BigQuery, Snowflake, Databricks, etc.) rather than relying on a proprietary metric store, and independent community commentary confirms this warehouse-native design as a key differentiator from black-box tools like Optimizely. missing for 10: hands-on PM-level walkthrough of defining a metric from ingested event data (vs. warehouse tables) and any independent report specifically validating metric-definition UX.

   • [claimed-docs] “Define Fact Tables and metrics so GrowthBook can compute experiment results from your warehouse.”
   • [github] “Warehouse Native. Query 11 data sources including BigQuery, Snowflake, and Databricks.”
   • [github] “❄️ Warehouse Native. Query 11 data sources including BigQuery, Snowflake, and Databricks.”
   • [community] “Re: Eppo - they're similar in setup but Growthbook is open source with an optional cloud SaaS offering. Optimizely is a walled garden, you n…”
   • [community] “a guide in your docs on how to use this with Matomo would go far. Growthbook provides Feature Flag and A/B Testing, but you have to bring yo…”

   proof ↗⚑ flag

Environments

1. developerManage separate environments (dev/staging/prod) with independent flag states and scoped SDK keys

   weight 2 · round to GrowthBook
   Statsig—none0/10

   The evidence pack covers feature gates, experiments, CLI, console API, and access management, but nowhere mentions environment tags (dev/staging/prod) or environment-scoped SDK keys — a core Statsig platform capability that isn't documented here. Missing for 10: explicit environment/project configuration docs, environment-scoped SDK key generation, and any mention of promoting flag state across environments.

   ⚑ flag

   GrowthBook✓fullprobed8/10

   Docs explicitly describe dedicated environments (dev/staging/production) with independently toggled flag states [growthbook-docs-3], and the SDK Connection Endpoint provides scoped, readonly keys per environment for SDKs [growthbook-docs-20], with runtime probes confirming a real create-flag/scoped-key/evaluate loop [growthbook-probe-rt-2]. Custom environments are also called out as a paid-tier feature, confirming the capability exists in-product [growthbook-docs-16, growthbook-docs-37]. Missing for 10: independent/community verification specifically of environment-scoped SDK key isolation (evidence is docs/probe only, not third-party confirmation) and more detail on default-environment limits on lower tiers.

   • [claimed-docs] “Use environments such as dev, staging, and production to enable or disable feature flags independently.”
   • [claimed-docs] “The SDK Connection Endpoint provides readonly access to a subset of your feature flag data, just enough for the GrowthBook SDKs to assign va…”
   • [claimed-docs] “Pro, plus: Custom environments Ramp schedules Approval workflows Advanced access control SSO & SCIM provisioning Exportable audit logs”
   • [claimed-docs] “Custom environments * Ramp schedules * Approval workflows * Advanced access control * SSO & SCIM provisioning * Exportable audit l…”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official growthbook/growthbook docker image boot…”

   proof ↗⚑ flag

Flags

1. developerCreate a feature flag and toggle it live in production within minutes of signing up

   weight 3 · round to GrowthBook
   Statsig✓fullcommunity8/10

   Docs provide a first-feature tutorial creating a gate, targeting users, and checking it live via SDK within a browser console (statsig-docs-24), plus a free-tier signup with feature flags included (statsig-docs-17), console-based exposure monitoring and overrides (statsig-docs-19, statsig-docs-29), and community praise confirming Statsig ships fast, developer-friendly tooling (statsig-comm-1, statsig-comm-2). Missing for 10: independent hands-on account of actual time-to-first-flag from signup and no third-party benchmark of onboarding speed.

   • [claimed-docs] “This tutorial creates a feature gate, targets mobile platforms and internal testers, then checks it live with the JavaScript SDK in your bro…”
   • [claimed-docs] “You get feature flags and 2 million metered events for free.”
   • [claimed-docs] “After your Feature Gate is live, view Feature Gate exposures in the Statsig console to monitor who encounters your gate.”
   • [claimed-docs] “set up Feature Gate overrides to allow specific users to bypass the gate.”
   • [community] “Hats off to Statsig. They built a stellar product. Superior to many of their industry competitors like Optimizely. Back when I was on an int…”
   • [community] “Have implemented Statsig in two companies as alternatives to LaunchDarkly and loved it both times. Going to be interesting how their other b…”

   proof ↗⚑ flag

   GrowthBook✓fullprobed9/10

   GrowthBook's quickstart docs show flag creation in a few lines of code with SDK hooks like useFeatureIsOn, and a runtime probe confirms a real keyless create-and-evaluate loop (create org, create flag, mint SDK connection, get correct value back) with no signup friction. Independent community testimony corroborates fast setup, with one user reporting a working flag in about 15 minutes and under 10 lines of code, and environments docs confirm dev/staging/prod toggling. missing for 10: a first-party or independent account of toggling a flag live specifically in a production environment end-to-end.

   • [claimed-docs] “Get GrowthBook feature flags and experiments running in your app in a few lines of code, then pick a language-specific SDK.”
   • [claimed-docs] “const showNewFeature = useFeatureIsOn("new-feature");”
   • [claimed-docs] “Use environments such as dev, staging, and production to enable or disable feature flags independently.”
   • [community] “Been using GrowthBook for a few weeks now... I was shocked at how much setup time it took to get a simple feature flag into our app with oth…”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official growthbook/growthbook docker image boot…”

   proof ↗⚑ flag

2. developerServe multivariate flags and dynamic configuration values (strings, numbers, JSON), not just booleans

   weight 2 · round to GrowthBook
   Statsig~partialclaimed6/10

   Statsig docs reference Dynamic Configs alongside Feature Gates and Experiments (e.g., statsig-docs-9, statsig-docs-10, statsig-docs-11, statsig-docs-31 mention 'Dynamic Configs' as a managed entity distinct from booleans), and experiments/layers imply multivariate parameter delivery (statsig-docs-4, statsig-docs-26, statsig-docs-30). However, no evidence explicitly confirms support for strings/numbers/JSON value types or shows a dynamic config JSON payload example. missing for 10: explicit documentation or example showing dynamic config value types (string/number/JSON), SDK code sample retrieving a non-boolean config value.

   • [claimed-docs] “The Statsig CLI is a command-line interface that helps manage Feature Gates, Experiments, and Dynamic Configs within the Statsig platform.”
   • [claimed-docs] “Overview of the Statsig HTTP API for retrieving feature gate, experiment, and dynamic config values and logging events directly without an S…”
   • [claimed-docs] “The Console API is the CRUD API for performing the actions available on console.statsig.com without using the web UI.”
   • [claimed-docs] “You can integrate the CLI into CI/CD pipelines to automate the management of Feature Gates and Experiments.”
   • [claimed-docs] “A Layer allows you to manage multiple experiments and feature flags together.”
   • [claimed-docs] “Statsig experimentation runs randomized controlled trials (A/B or A/B/n tests) that measure how product changes affect your key metrics.”

   proof ↗⚑ flag

   GrowthBook✓fullprobed7/10

   GrowthBook's SDK docs explicitly show non-boolean flag values via useFeatureValue (string example 'button-color') alongside useFeatureIsOn for booleans, and its targeting/rules docs describe rules that 'control which values a user sees' beyond simple on/off, plus a real runtime probe confirms a flag payload with a typed defaultValue served via the API. Missing for 10: explicit documented example of numeric and JSON-typed flag values (only string is shown in the quickstart snippet), and no independent hands-on confirmation of JSON config usage.

   • [claimed-docs] “const showNewFeature = useFeatureIsOn("new-feature"); const buttonColor = useFeatureValue("button-color", "blue");”
   • [claimed-docs] “How rules override a feature's default value — targeting users, rolling out gradually, experimenting — and how to scope, ramp, and schedule …”
   • [claimed-docs] “Add targeting conditions to a rule to control which values a user sees.”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official growthbook/growthbook docker image boot…”
   • [github] “24 SDKs including React, Python, Android, and iOS.”

   proof ↗⚑ flag

Lifecycle

1. platform engineerFind stale flags and code references so temporary flags actually get removed from the codebase

   weight 2 · round to GrowthBook
   Statsig—none0/10

   The evidence pack covers feature gates, experiments, rollout scheduling, overrides, CLI, and MCP integrations, but nothing documents stale-flag detection, flag usage/code-reference scanning, or cleanup workflows. This is a reasonable axis for a feature-flag platform, but no capability is evidenced.

   ⚑ flag

   GrowthBook✓fullclaimed8/10

   GrowthBook's docs explicitly describe automatic stale flag detection ('GrowthBook can automatically detect these flags so you can clean them up') as a dedicated feature, directly addressing the story of finding stale flags for removal. This is corroborated by the product's broader flag-management infrastructure (environments, rules, revisions) but lacks detail on code-reference scanning across repos or independent/hands-on verification of the stale-detection feature itself. Missing for 10: evidence of code-reference/usage scanning across the codebase, and independent/community confirmation that stale detection works reliably in practice.

   • [claimed-docs] “GrowthBook can automatically detect these flags so you can clean them up.”
   • [claimed-docs] “How rules override a feature's default value — targeting users, rolling out gradually, experimenting — and how to scope, ramp, and schedule …”
   • [claimed-docs] “Use environments such as dev, staging, and production to enable or disable feature flags independently.”

   proof ↗⚑ flag

Targeting

1. developerTarget or exclude specific individual users for a flag (allowlists, beta testers, internal accounts)

   weight 1 · round to Statsig
   Statsig✓fullclaimed8/10

   Docs explicitly cover overrides for targeting/excluding specific users and a guided tutorial that targets internal testers/mobile platforms, directly matching the allowlist/beta-tester/internal-account use case. missing for 10: no independent hands-on confirmation of override UI behavior beyond first-party docs.

   • [claimed-docs] “set up Feature Gate overrides to allow specific users to bypass the gate.”
   • [claimed-docs] “This tutorial creates a feature gate, targets mobile platforms and internal testers, then checks it live with the JavaScript SDK in your bro…”
   • [claimed-docs] “After your Feature Gate is live, view Feature Gate exposures in the Statsig console to monitor who encounters your gate.”

   proof ↗⚑ flag

   GrowthBook✓fullclaimed7/10

   GrowthBook's rule/targeting system lets you write conditions against arbitrary user attributes (e.g., user ID, email) to include or exclude specific users, which covers allowlisting beta testers or internal accounts (growthbook-docs-2, growthbook-docs-22, growthbook-docs-18, growthbook-docs-28). This is a first-class part of its rules engine (scope, ramp, schedule). missing for 10: an explicit doc example/tutorial titled 'allowlist' or 'beta testers' walkthrough, and independent hands-on confirmation of exclusion-list behavior.

   • [claimed-docs] “Add targeting conditions to a rule to control which values a user sees. Conditions are evaluated against attributes you pass into the SDK”
   • [claimed-docs] “How rules override a feature's default value — targeting users, rolling out gradually, experimenting — and how to scope, ramp, and schedule …”
   • [claimed-docs] “The actual values of targeting attributes (e.g., user IDs, emails) are never sent to GrowthBook. They are only stored in memory locally with…”
   • [claimed-docs] “Add targeting conditions to a rule to control which values a user sees.”

   proof ↗⚑ flag

2. developerTarget flags with attribute-based rules and reusable segments so the right users see the right variation

   weight 3 · round to GrowthBook
   Statsig~partialclaimed5/10

   Docs confirm Feature Gates support attribute-based targeting (e.g., targeting mobile platforms and internal testers) and per-user overrides, indicating rule-based flag targeting is supported. However, the evidence pack never mentions a dedicated 'segments' or reusable audience-list feature, so reusable, cross-flag targeting groups are unevidenced. missing for 10: explicit documentation of reusable segment/audience definitions, evidence of segment reuse across multiple flags/experiments, and independent/hands-on confirmation of complex rule builders.

   • [claimed-docs] “This tutorial creates a feature gate, targets mobile platforms and internal testers, then checks it live with the JavaScript SDK in your bro…”
   • [claimed-docs] “set up Feature Gate overrides to allow specific users to bypass the gate.”
   • [claimed-docs] “Feature Gates, commonly known as feature flags, allow you to toggle the behavior of your product in real time without deploying new code.”

   proof ↗⚑ flag

   GrowthBook~partialclaimed6/10

   GrowthBook clearly supports attribute-based targeting rules (conditions evaluated against SDK-passed attributes, rule scoping/ramping/scheduling) as shown in growthbook-docs-2, -22, -28, and -18. However, the evidence pack never mentions a distinct 'segments' feature (reusable named audience definitions) that developers could apply across multiple flags/rules, so the reusable-segment half of the story is unevidenced. missing for 10: explicit segment/audience-reuse documentation, evidence of applying the same segment across multiple flags, independent confirmation of segment behavior.

   • [claimed-docs] “Add targeting conditions to a rule to control which values a user sees. Conditions are evaluated against attributes you pass into the SDK”
   • [claimed-docs] “How rules override a feature's default value — targeting users, rolling out gradually, experimenting — and how to scope, ramp, and schedule …”
   • [claimed-docs] “Add targeting conditions to a rule to control which values a user sees.”
   • [claimed-docs] “The actual values of targeting attributes (e.g., user IDs, emails) are never sent to GrowthBook. They are only stored in memory locally with…”

   proof ↗⚑ flag

Access

1. platform engineerRestrict who can change which flags with roles, permissions, and scoped API tokens

   weight 2 · round to GrowthBook
   Statsig~partialclaimed4/10

   Statsig docs reference enterprise access management with SSO/SCIM and a Console API for programmatic control, implying some access-control layer exists, but no evidence details role definitions, granular permissions, or scoped/limited API tokens for restricting who can change specific flags. Missing for 10: explicit RBAC documentation, permission granularity for flag edits, and scoped/read-vs-write API token support.

   • [claimed-docs] “In many enterprise environments, organizations use SSO and SCIM together to improve security and usability.”
   • [claimed-docs] “Basic settings include automatically adding new users with the same email domain to your project. Statsig also supports SSO to simplify invi…”
   • [claimed-docs] “The Console API is the CRUD API for performing the actions available on console.statsig.com without using the web UI.”

   proof ↗⚑ flag

   GrowthBook~partialclaimed5/10

   Pricing page confirms GrowthBook has role/permission features ("Advanced permissioning", "Advanced access control", SSO & SCIM) gated behind paid tiers, implying RBAC exists, but there is no docs page detailing how roles map to flag-level permissions or how API tokens can be scoped to restrict flag changes; the API docs even note the SDK connection endpoint is public/unauthenticated, which is a different, more limited access model. Missing for 10: a dedicated permissions/roles doc, explicit description of scoped/restricted API tokens for flag mutation, and independent confirmation of enforcement.

   • [claimed-docs] “Starter, plus: Visual editor Multi-arm bandits Safe rollouts Customizable dashboards Advanced permissioning Power calculator Premium support”
   • [claimed-docs] “Pro, plus: Custom environments Ramp schedules Approval workflows Advanced access control SSO & SCIM provisioning Exportable audit logs”
   • [claimed-docs] “Visual editor * Multi-arm bandits * Safe rollouts * Customizable dashboards * Advanced permissioning * Power calculator”
   • [claimed-docs] “Custom environments * Ramp schedules * Approval workflows * Advanced access control * SSO & SCIM provisioning * Exportable audit l…”
   • [claimed-docs] “GrowthBook offers a full REST API for interacting with the application.”
   • [claimed-docs] “They are meant to be public and do not require authentication to view.”

   proof ↗⚑ flag

Agent ops

1. ai agentCreate and toggle flags through documented APIs, CLIs, or MCP — and the platform can force my changes through approval workflows instead of letting me write to production unreviewed

   weight 3 · round to GrowthBook
   Statsig~partialprobed4/10

   Statsig documents multiple agent-friendly interfaces for creating/toggling flags — a CLI (also usable in CI/CD), a Console CRUD API, and an official MCP server reachable via OAuth — giving strong coverage of the 'documented APIs, CLIs, or MCP' half of the story. However, there is no evidence of an approval/review workflow that intercepts or blocks agent-initiated changes before they hit production; access-management docs mention SSO/SCIM for identity but not change-approval gating. Missing for 10: documented approval-workflow/change-review gating for flag mutations, evidence that CLI/API/MCP writes can be routed through a review step rather than applied directly.

   • [claimed-docs] “The Statsig CLI is a command-line interface that helps manage Feature Gates, Experiments, and Dynamic Configs within the Statsig platform.”
   • [claimed-docs] “The Console API is the CRUD API for performing the actions available on console.statsig.com without using the web UI.”
   • [claimed-docs] “The Statsig MCP (Model Context Protocol) server brings the power of Statsig into tools like Codex, Cursor, and Claude Code. With this setup,…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Statsig CLI (npm @statsig/siggy) ran keylessly via `npx -y @statsig/siggy --version`, prin…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.statsig.com/v1/mcp returned HTTP …”
   • [claimed-docs] “In many enterprise environments, organizations use SSO and SCIM together to improve security and usability.”

   proof ↗⚑ flag

   GrowthBook~partialprobed6/10

   GrowthBook documents an official MCP server that makes authenticated REST calls (growthbook-docs-13/21, growthbook-probe-3/rt-1) and a full REST API (growthbook-docs-12), and separately documents 'Draft feature revisions, request review... require approval before publishing' as a Pro+ approval-workflow feature (growthbook-docs-5, growthbook-docs-16/37). However, the runtime probe (growthbook-probe-rt-2) shows a flag created and immediately live via the API with no review step, and approval workflows appear gated behind a paid plan rather than proven to force-block an AI agent's writes by default. Missing for 10: a documented/verified CLI channel, and hands-on evidence that MCP/API writes are actually intercepted and blocked pending human approval rather than merely available as an opt-in paid feature.

   • [claimed-docs] “Connect AI tools like Cursor, VS Code, and Claude to GrowthBook using the Model Context Protocol (MCP).”
   • [claimed-docs] “The official GrowthBook MCP server is a thin bridge: it loads GrowthBook Agent Skills (workflows and guardrails) and makes authenticated cal…”
   • [probe] “official MCP server documented at https://docs.growthbook.io/integrations/mcp”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official npm MCP server (`npx -y @growthbook/mcp`) completed a FULL keyless stdio initialize handsh…”
   • [claimed-docs] “Draft feature revisions, request review, resolve merge conflicts, and require approval before publishing.”
   • [claimed-docs] “Pro, plus: Custom environments Ramp schedules Approval workflows Advanced access control SSO & SCIM provisioning Exportable audit logs”
   • [claimed-docs] “Custom environments * Ramp schedules * Approval workflows * Advanced access control * SSO & SCIM provisioning * Exportable audit l…”
   • [claimed-docs] “GrowthBook offers a full REST API for interacting with the application.”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official growthbook/growthbook docker image boot…”

   proof ↗⚑ flag

Approvals

1. platform engineerRequire approvals or change requests before production flag changes go live

   weight 3 · round to GrowthBook
   Statsig—none0/10

   No evidence describes an approval workflow, change request, or review gate before flag changes go live in production; docs mention scheduled rollouts, overrides, and audit-adjacent features like exposure logging but not a governance approval mechanism.

   ⚑ flag

   GrowthBook✓fullclaimed8/10

   GrowthBook's docs explicitly describe drafting feature revisions, requesting review, resolving merge conflicts, and requiring approval before publishing changes (growthbook-docs-5), and this is packaged as a named 'Approval workflows' feature on paid tiers (growthbook-docs-16, growthbook-docs-37). This directly matches the story of gating production flag changes behind approvals. Missing for 10: independent/hands-on confirmation of the approval workflow in practice (only vendor docs cited) and detail on how approvals interact with environment-specific promotion.

   • [claimed-docs] “Draft feature revisions, request review, resolve merge conflicts, and require approval before publishing.”
   • [claimed-docs] “Pro, plus: Custom environments Ramp schedules Approval workflows Advanced access control SSO & SCIM provisioning Exportable audit logs”
   • [claimed-docs] “Custom environments * Ramp schedules * Approval workflows * Advanced access control * SSO & SCIM provisioning * Exportable audit l…”
   • [claimed-docs] “Use environments such as dev, staging, and production to enable or disable feature flags independently.”

   proof ↗⚑ flag

Audit

1. platform engineerEvery flag change is recorded in an audit log — who changed what, when, and to which value

   weight 3 · round to GrowthBook
   Statsig—none0/10

   The evidence pack covers feature gates, experiments, SSO/SCIM, CLI, Console API, and MCP integrations, but nowhere mentions an audit log or change history capturing who changed a flag, when, or to what value. This is a standard governance capability for feature-flag platforms, so its complete absence from the docs pack means the axis applies but is unsupported.

   ⚑ flag

   GrowthBook~partialclaimed5/10

   Pricing docs confirm GrowthBook ships an audit log feature, with 'Exportable audit logs' listed as a Pro-tier capability, implying built-in tracking of changes for governance purposes. However, there is no first-party documentation detailing exactly what is captured (who/when/old-new value) or how the audit trail is presented/queried, and the capability is gated behind a paid plan rather than universally available. Missing for 10: dedicated audit-log documentation describing captured fields, retention, and access/export mechanics, plus confirmation it's available outside paid tiers or independent corroboration of its behavior.

   • [claimed-docs] “Pro, plus: Custom environments Ramp schedules Approval workflows Advanced access control SSO & SCIM provisioning Exportable audit logs”
   • [claimed-docs] “Custom environments * Ramp schedules * Approval workflows * Advanced access control * SSO & SCIM provisioning * Exportable audit l…”
   • [claimed-docs] “SSO & SCIM provisioning * Exportable audit logs”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Statsig
   Statsig✓fullprobed8/10

   Statsig documents a full Console API that is explicitly the CRUD API for 'performing the actions available on console.statsig.com without using the web UI,' plus a separate HTTP API for gate/experiment evaluation and event logging, and an official CLI wrapping these for CI/CD scripting — together covering feature gates, experiments, dynamic configs, and access management via API/CLI rather than only the UI. MCP servers and CLI were confirmed live/reachable in runtime probes, reinforcing that programmatic parity is real and operational, not just documented aspiration. Missing for 10: no independent third-party audit confirming 1:1 UI/API feature parity (e.g., some newer UI features like Autotune or Warehouse Native scorecards aren't explicitly confirmed as API-exposed), and no public OpenAPI spec was found (404s on standard paths).

   • [claimed-docs] “The Console API is the CRUD API for performing the actions available on console.statsig.com without using the web UI.”
   • [claimed-docs] “Overview of the Statsig HTTP API for retrieving feature gate, experiment, and dynamic config values and logging events directly without an S…”
   • [claimed-docs] “The Statsig CLI is a command-line interface that helps manage Feature Gates, Experiments, and Dynamic Configs within the Statsig platform.”
   • [claimed-docs] “You can integrate the CLI into CI/CD pipelines to automate the management of Feature Gates and Experiments.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Statsig CLI (npm @statsig/siggy) ran keylessly via `npx -y @statsig/siggy --version`, prin…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.statsig.com/openapi.json, https://docs.statsig.com/swagger.json, https://docs.statsig.c…”

   proof ↗⚑ flag

   GrowthBook✓fullprobed7/10

   GrowthBook documents a 'full REST API for interacting with the application' (growthbook-docs-12) and a runtime probe confirms it: a self-hosted instance created an org, user, and feature flag purely via API calls and served it through the SDK payload endpoint (growthbook-probe-rt-2), and the official MCP server exposes authenticated read/write meta-tools covering the whole REST API (growthbook-docs-21, growthbook-probe-rt-1). Missing for 10: an explicit enumerated API-vs-UI parity list (e.g. confirmation that experiment approval workflows, visual editor changes, and dashboard configuration are all API-exposed, not just feature flags) and independent third-party corroboration of API completeness.

   • [claimed-docs] “GrowthBook offers a full REST API for interacting with the application.”
   • [claimed-docs] “The official GrowthBook MCP server is a thin bridge: it loads GrowthBook Agent Skills (workflows and guardrails) and makes authenticated cal…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official npm MCP server (`npx -y @growthbook/mcp`) completed a FULL keyless stdio initialize handsh…”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official growthbook/growthbook docker image boot…”
   • [claimed-docs] “The SDK Connection Endpoint provides readonly access to a subset of your feature flag data, just enough for the GrowthBook SDKs to assign va…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round to GrowthBook
   Statsig~partialclaimed4/10

   Statsig provides a Console API and CLI for programmatic access to configuration data (feature gates, experiments, dynamic configs), which supports exporting configuration/setup data in a scriptable way, but there is no evidence of a bulk data export feature for user-level event/analytics data in open/standard formats, nor documentation of data portability or account deletion/export workflows for users wanting to leave the platform. missing for 10: documented bulk export of analytics/event data in open formats (CSV/JSON/Parquet), explicit data portability or 'leave the platform' export tooling, evidence of exporting experiment results/metrics data, and any community confirmation of successful full data export.

   • [claimed-docs] “The Console API is the CRUD API for performing the actions available on console.statsig.com without using the web UI.”
   • [claimed-docs] “The Statsig CLI is a command-line interface that helps manage Feature Gates, Experiments, and Dynamic Configs within the Statsig platform.”
   • [claimed-docs] “You can integrate the CLI into CI/CD pipelines to automate the management of Feature Gates and Experiments.”

   proof ↗⚑ flag

   GrowthBook~partialcommunity5/10

   GrowthBook is open-source and self-hostable with its data stored in a standard MongoDB instance, and it exposes a full REST API for programmatic access to feature flags, experiments, and metrics, which together give a technically-inclined user a path to extract all their data without vendor lock-in. However, there is no explicit documented 'export all data' or bulk-export feature/wizard, and 'exportable audit logs' is gated behind an Enterprise tier rather than being a general open-export capability. Missing for 10: an explicit documented full-data export/backup tool or format, confirmation that experiment results/metrics history export in an open format, and evidence this works outside self-hosting for cloud customers.

   • [claimed-docs] “GrowthBook offers a full REST API for interacting with the application.”
   • [claimed-docs] “Run GrowthBook on your own infrastructure with Docker Compose, including MongoDB, the API, and the front-end.”
   • [claimed-docs] “You can use docker compose to get started quickly... run docker compose up -d to start everything and view the app at http://localhost:3000”
   • [claimed-docs] “Custom environments * Ramp schedules * Approval workflows * Advanced access control * SSO & SCIM provisioning * Exportable audit l…”
   • [community] “Re: Eppo - they're similar in setup but Growthbook is open source with an optional cloud SaaS offering. Optimizely is a walled garden, you n…”

   proof ↗⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round to GrowthBook
   Statsig—none0/10

   No evidence in the pack indicates Statsig's core platform or source code is released under an open license; only docs, CLI, MCP servers, and SaaS features are mentioned, none of which speak to open-source licensing of the product's source.

   ⚑ flag

   GrowthBook~partialcommunity5/10

   Community evidence explicitly states GrowthBook is open source with an optional cloud SaaS offering, and a public GitHub repository (github.com/growthbook/growthbook) is cited, supporting that source is readable. However, no evidence pack item specifies the actual license type (e.g., MIT/Apache) or licensing terms, so the 'open license' claim isn't directly documented. Missing for 10: explicit license identification/text, first-party docs confirming license terms, independent confirmation of license permissiveness.

   • [community] “Re: Eppo - they're similar in setup but Growthbook is open source with an optional cloud SaaS offering. Optimizely is a walled garden, you n…”
   • [github] “24 SDKs including React, Python, Android, and iOS.”
   • [claimed-docs] “Run GrowthBook on your own infrastructure with Docker Compose, including MongoDB, the API, and the front-end.”

   proof ↗⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round to GrowthBook
   Statsig—none0/10

   No evidence anywhere in the pack mentions self-hosting or an on-prem/open-source deployment of the core Statsig platform; all documentation describes a hosted SaaS console, hosted MCP/API endpoints, and cloud-based warehouse-native analysis. Self-hosting is a fair ask for this category (competitors offer it), so absence of evidence means 'none' rather than 'na'.

   ⚑ flag

   GrowthBook✓fullprobed10/10

   GrowthBook is documented and independently confirmed (via runtime probe) to self-host fully via Docker Compose including MongoDB, API, and front-end, with a real keyless create-and-evaluate flag loop working end-to-end; community reviews also corroborate long-term self-hosted usage.

   • [claimed-docs] “Run GrowthBook on your own infrastructure with Docker Compose, including MongoDB, the API, and the front-end.”
   • [claimed-docs] “You can use docker compose to get started quickly... run docker compose up -d to start everything and view the app at http://localhost:3000”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official growthbook/growthbook docker image boot…”
   • [community] “Growthbook is awesome! We have used the self-hosted version a few months in my company now, and are very happy about it. Talented people beh…”

   proof ↗⚑ flag

Pricing

1. product managerSee published pricing and understand what drives cost (seats, MAUs, events, requests) before committing

   weight 2 · round to GrowthBook
   Statsig~partialclaimed3/10

   Docs mention a free tier tied to 'events' as a metered unit, giving a partial glimpse of cost drivers, but there is no evidence of a full published pricing page detailing seats, MAUs, or request-based costs across paid tiers. missing for 10: published pricing tiers/page, seat-based pricing, MAU-based pricing, request-based pricing, enterprise pricing details.

   • [claimed-docs] “You get feature flags and 2 million metered events for free.”

   proof ↗⚑ flag

   GrowthBook~partialclaimed6/10

   GrowthBook publishes a tiered pricing page (Starter/Pro/Enterprise-like tiers) listing feature gates (visual editor, approval workflows, SSO/SCIM, etc.) and calls out unlimited flags/experiments/traffic and per-user caps like 'Up to 3 users', but none of the evidence shows usage-based cost drivers like MAUs, events, or API requests being priced or metered — the plans appear seat/feature-gated rather than usage-metered. missing for 10: explicit cost drivers tied to MAUs/events/requests, a pricing calculator or usage-based cost breakdown, and independent confirmation that published prices match real invoices.

   • [claimed-docs] “Starter, plus: Visual editor Multi-arm bandits Safe rollouts Customizable dashboards Advanced permissioning Power calculator Premium support”
   • [claimed-docs] “Pro, plus: Custom environments Ramp schedules Approval workflows Advanced access control SSO & SCIM provisioning Exportable audit logs”
   • [claimed-docs] “Unlimited feature flags * Unlimited experiments * Unlimited traffic”
   • [claimed-docs] “Starter, plus: * Visual editor * Multi-arm bandits * Safe rollouts * Customizable dashboards”
   • [claimed-docs] “Visual editor * Multi-arm bandits * Safe rollouts * Customizable dashboards * Advanced permissioning * Power calculator”
   • [claimed-docs] “Custom environments * Ramp schedules * Approval workflows * Advanced access control * SSO & SCIM provisioning * Exportable audit l…”
   • [claimed-docs] “Up to 3 users * 1 project * Unlimited feature flags * Unlimited experiments * Unlimited traffic”
   • [claimed-docs] “SSO & SCIM provisioning * Exportable audit logs”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round to GrowthBook
   Statsig—none0/10

   No evidence in the pack mentions data residency, regional storage options, or geographic data controls for Statsig's platform; the documentation excerpts cover feature flags, experiments, MCP, CLI, and access management but never address where data is stored or user choice of region.

   ⚑ flag

   GrowthBook~partialprobed6/10

   GrowthBook can be fully self-hosted via Docker Compose (own MongoDB, API, and front-end), which lets an AI-native user control exactly where all data lives, and a probe confirms a working self-hosted flag create/evaluate loop with no third-party data storage. However, there is no documented cloud-hosted region/residency selection (e.g., EU vs US SaaS options) for users who don't want to self-host. Missing for 10: explicit cloud multi-region hosting options, formal data-residency/compliance certifications, and documentation letting SaaS customers pick a storage region without self-hosting.

   • [claimed-docs] “Run GrowthBook on your own infrastructure with Docker Compose, including MongoDB, the API, and the front-end.”
   • [claimed-docs] “You can use docker compose to get started quickly... run docker compose up -d to start everything and view the app at http://localhost:3000”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official growthbook/growthbook docker image boot…”
   • [claimed-docs] “The actual values of targeting attributes (e.g., user IDs, emails) are never sent to GrowthBook. They are only stored in memory locally with…”

   proof ↗⚑ flag

2. ai-native userControl data retention and deletion

   weight 2 · round to GrowthBook
   Statsig—none0/10

   The evidence pack covers feature flags, experimentation, MCP integrations, and access management (SSO/SCIM), but contains no mention of data retention policies, deletion controls, or GDPR/CCPA-style data lifecycle management. This is a fair axis for a SaaS analytics/experimentation platform handling user event data, so absence of evidence yields 'none' rather than 'na'.

   ⚑ flag

   GrowthBook~partialclaimed4/10

   GrowthBook's privacy-by-design targeting (attribute values never sent to GrowthBook, kept in-memory in the SDK) and the option to self-host on your own infrastructure/MongoDB give users indirect control over what data is retained and how it's deleted, since they own the database. However, there is no explicit documentation of a data retention policy, a deletion API/endpoint, or GDPR-style 'delete my data' tooling in the evidence pack. Missing for 10: explicit retention-period settings, a documented deletion/erasure API or workflow, and any compliance certification (e.g., GDPR/CCPA) statements.

   • [claimed-docs] “The actual values of targeting attributes (e.g., user IDs, emails) are never sent to GrowthBook. They are only stored in memory locally with…”
   • [claimed-docs] “Run GrowthBook on your own infrastructure with Docker Compose, including MongoDB, the API, and the front-end.”
   • [claimed-docs] “You can use docker compose to get started quickly... run docker compose up -d to start everything and view the app at http://localhost:3000”

   proof ↗⚑ flag

3. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Statsig—none0/10

   No evidence in the pack addresses whether Statsig's own tools (CLI, SDKs, MCP server, docs site) collect telemetry/usage data or provide an opt-out mechanism; the pack only covers Statsig's core flagging/experimentation features, not its own instrumentation privacy controls.

   ⚑ flag

   GrowthBook—none0/10

   The evidence pack shows GrowthBook is self-hostable and that targeting-attribute values are never sent to the vendor, but there is no documentation of any product telemetry/usage-tracking opt-out setting or flag for the self-hosted or cloud application itself. Missing for 10: any explicit telemetry/usage-analytics toggle, docs on what data is phoned home from self-hosted instances, or a documented opt-out mechanism.

   ⚑ flag

Rollouts

1. platform engineerGuard a rollout with metrics so a regression is detected and the release is rolled back automatically

   weight 2 · round to Statsig
   Statsig~partialclaimed5/10

   Statsig supports gradual/scheduled rollouts, feature-gate emergency kill switches, and metric-based experimentation with alerting for regressions (Infra Analytics 'set alerts to catch regressions'), giving the pieces needed to guard a rollout with metrics. However, none of the evidence shows an automated rollback action triggered by a detected metric regression — alerts and kill switches appear to require manual intervention rather than an automatic release rollback loop. Missing for 10: documented automatic rollback/kill-switch triggered by metric threshold breach, closed-loop guardrail-to-rollback automation, and independent confirmation this works in production.

   • [claimed-docs] “Have an emergency disable switch that lets you immediately turn off a particular code branch for users in production”
   • [claimed-docs] “Gradually deploy a feature over time by setting up a Feature Gate as a Scheduled Rollout.”
   • [claimed-docs] “Infra Analytics: Monitor and debug service health alongside product outcomes. Ingest metrics and traces with OpenTelemetry, search and group…”
   • [claimed-docs] “Monitor and debug service health alongside product outcomes. Ingest metrics and traces with OpenTelemetry, search and group logs, and set al…”
   • [claimed-docs] “Schedule gradual feature rollouts to safely deploy new code”
   • [claimed-docs] “The Scorecard feature makes this easy by letting you enter your hypothesis and select both primary and secondary metrics.”

   proof ↗⚑ flag

   GrowthBook~partialclaimed4/10

   GrowthBook's pricing pages list a named 'Safe rollouts' feature (Pro+ tier) alongside guardrail-metric support in experiment templates, implying automated regression detection and rollback, but the evidence pack contains no docs explaining how the automatic rollback mechanism actually works (thresholds, trigger logic, rollback speed) nor any hands-on confirmation it fires reliably in production. Missing for 10: detailed docs/description of the Safe Rollouts mechanism and guardrail-triggered auto-rollback, plus independent or hands-on evidence it works as claimed.

   • [claimed-docs] “Starter, plus: Visual editor Multi-arm bandits Safe rollouts Customizable dashboards Advanced permissioning Power calculator Premium support”
   • [claimed-docs] “Starter, plus: * Visual editor * Multi-arm bandits * Safe rollouts * Customizable dashboards”
   • [claimed-docs] “Visual editor * Multi-arm bandits * Safe rollouts * Customizable dashboards * Advanced permissioning * Power calculator”
   • [claimed-docs] “Visual editor * Multi-arm bandits * Safe rollouts * Customizable dashboards”
   • [claimed-docs] “Selecting a Template in the creation modal pre-fills settings like the data source, goal/secondary/guardrail metrics, targeting, and traffic”

   proof ↗⚑ flag

2. developerRoll a flag out progressively by percentage with consistent bucketing, ramping from 1% to 100% without redeploying

   weight 3 · round drawn
   Statsig✓fullclaimed8/10

   Statsig docs explicitly describe scheduled/gradual rollouts to safely deploy new code without redeploying, and deterministic evaluation ensures consistent bucketing across evaluations for the same user given the same gate state. Percentage-based ramping (1%→100%) is a core documented Feature Gate/Scheduled Rollout capability, backed by consistent hashing via deterministic evaluation. missing for 10: explicit mention of percentage granularity/UI slider mechanics and independent hands-on confirmation of ramp behavior.

   • [claimed-docs] “Gradually deploy a feature over time by setting up a Feature Gate as a Scheduled Rollout.”
   • [claimed-docs] “Schedule gradual feature rollouts to safely deploy new code”
   • [claimed-docs] “Evaluation in Statsig is deterministic. Given the same user object and the same state of the experiment or feature gate, Statsig always retu…”
   • [claimed-docs] “Evaluation in Statsig is deterministic. Given the same user object and the same state of the experiment or feature gate, Statsig always retu…”
   • [claimed-docs] “Feature Gates, commonly known as feature flags, allow you to toggle the behavior of your product in real time without deploying new code.”

   proof ↗⚑ flag

   GrowthBook✓fullprobed8/10

   GrowthBook's rules docs explicitly cover 'rolling out gradually' and describe how rules can 'scope, ramp, and schedule' percentage rollouts, with experiment/flag assignment based on a hashing attribute for consistent bucketing, and SDKs evaluate the cached JSON payload client-side with no redeploy needed. Pricing tiers also list dedicated 'Ramp schedules' as a named capability, confirming this is a first-class shipped feature, and probe evidence confirms a live flag create/evaluate loop via the SDK payload endpoint. missing for 10: independent/hands-on confirmation of an actual 1%→100% ramp in practice, and detail on the specific hashing algorithm ensuring bucket stability across ramp changes

   • [claimed-docs] “How rules override a feature's default value — targeting users, rolling out gradually, experimenting — and how to scope, ramp, and schedule …”
   • [claimed-docs] “you can add an experiment rule to a feature that will randomly assign the users based on some hashing attribute into one of your experiment …”
   • [claimed-docs] “When the client loads your application, our SDK will check to see if the user should be part of any experiments, and if so, assign them and …”
   • [claimed-docs] “Pro, plus: Custom environments Ramp schedules Approval workflows Advanced access control SSO & SCIM provisioning Exportable audit logs”
   • [claimed-docs] “Custom environments * Ramp schedules * Approval workflows * Advanced access control * SSO & SCIM provisioning * Exportable audit l…”
   • [claimed-docs] “GrowthBook supports several ways to do this: Feature Flags (most common)... URL Redirects... Our Visual Editor... Your own custom variation …”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official growthbook/growthbook docker image boot…”

   proof ↗⚑ flag

Scheduling

1. developerSchedule flag changes and releases to happen at a specific future time

   weight 1 · round drawn
   Statsig~partialclaimed6/10

   Docs confirm Statsig supports 'Scheduled Rollout' for Feature Gates, letting you gradually deploy a feature over time rather than instantly ('Gradually deploy a feature over time by setting up a Feature Gate as a Scheduled Rollout', 'Schedule gradual feature rollouts to safely deploy new code'), which implies future-time scheduling of flag state changes. However, the pack lacks detail on setting an exact future timestamp for a release event, calendar/date-picker UI specifics, or independent confirmation of this exact workflow. Missing for 10: explicit documentation of specifying an exact date/time for a rollout step, screenshots or hands-on walkthrough, and third-party confirmation that scheduled changes fire reliably.

   • [claimed-docs] “Gradually deploy a feature over time by setting up a Feature Gate as a Scheduled Rollout.”
   • [claimed-docs] “Schedule gradual feature rollouts to safely deploy new code”

   proof ↗⚑ flag

   GrowthBook~partialclaimed6/10

   GrowthBook explicitly documents that feature rules can be 'scoped, ramped, and scheduled,' and its pricing page lists 'Ramp schedules' as a distinct capability, confirming a future-time scheduling mechanism for flag rollouts. However, this appears gated to paid plans (Pro tier) and the evidence pack gives no detailed walkthrough of how scheduling is configured or any independent/hands-on confirmation it works as described. Missing for 10: detailed documentation of the scheduling UI/API, hands-on or community verification, and clarity on whether it applies broadly to all rule types.

   • [claimed-docs] “How rules override a feature's default value — targeting users, rolling out gradually, experimenting — and how to scope, ramp, and schedule …”
   • [claimed-docs] “Pro, plus: Custom environments Ramp schedules Approval workflows Advanced access control SSO & SCIM provisioning Exportable audit logs”
   • [claimed-docs] “Custom environments * Ramp schedules * Approval workflows * Advanced access control * SSO & SCIM provisioning * Exportable audit l…”

   proof ↗⚑ flag

Edge

1. platform engineerEvaluate flags at the edge (CDN workers or an edge/relay layer) close to users

   weight 2 · round to GrowthBook
   Statsig—none0/10

   The evidence pack covers Statsig's feature gates, on-device eval SDKs for client-side, server SDKs, and CLI/API surfaces, but nothing addresses evaluating flags specifically at CDN/edge workers or an edge/relay layer close to users. Missing for 10: any mention of Cloudflare Workers, edge runtime SDKs, or a proxy/relay layer for edge evaluation.

   • [claimed-docs] “Statsig's client-side On-Device Eval SDKs provide an alternate client-side architecture. The device keeps the definition of each experiment …”
   • [claimed-docs] “Statsig's client-side On-Device Eval SDKs provide an alternate client-side architecture. The device keeps the definition of each experiment …”
   • [claimed-docs] “Evaluation in Statsig is deterministic. Given the same user object and the same state of the experiment or feature gate, Statsig always retu…”

   proof ↗⚑ flag

   GrowthBook✓fullclaimed8/10

   GrowthBook documents a dedicated Cloudflare Workers edge SDK that evaluates feature flags and Visual Editor experiments at the edge to cut flicker before HTML ships, plus a self-hostable Proxy layer that caches SDK payloads near users and keeps serving flags if the API is down. Missing for 10: independent hands-on evidence specifically of the edge/Cloudflare SDK in production (community evidence covers general SDK/self-host use but not edge-worker deployments) and broader edge-platform coverage beyond Cloudflare.

   • [claimed-docs] “Evaluate GrowthBook feature flags and Visual Editor experiments at the edge on Cloudflare Workers to cut flicker before HTML ships.”
   • [claimed-docs] “Run the GrowthBook Proxy next to your app to cache SDK payloads, reduce latency, and keep serving flags if the API is down.”
   • [claimed-docs] “The SDK Connection Endpoint provides readonly access to a subset of your feature flag data, just enough for the GrowthBook SDKs to assign va…”

   proof ↗⚑ flag

Evaluation

1. platform engineerMy server SDKs evaluate flags locally from a cached ruleset — microsecond decisions with no network call per flag check

   weight 3 · round to GrowthBook
   Statsig~partialclaimed5/10

   Docs confirm deterministic evaluation 'even across different platforms (client or server)' implying server SDKs replicate rule logic locally, and explicitly describe on-device eval SDKs keeping definitions in memory for fast evaluation — but this on-device architecture is documented only for client SDKs, not explicitly server SDKs. Missing for 10: explicit server-SDK docs on background polling/streaming to a local cache, no-network-call guarantee per check, and microsecond-latency benchmarks specific to server SDKs.

   • [claimed-docs] “Evaluation in Statsig is deterministic. Given the same user object and the same state of the experiment or feature gate, Statsig always retu…”
   • [claimed-docs] “Statsig's client-side On-Device Eval SDKs provide an alternate client-side architecture. The device keeps the definition of each experiment …”
   • [claimed-docs] “Evaluation in Statsig is deterministic. Given the same user object and the same state of the experiment or feature gate, Statsig always retu…”
   • [claimed-docs] “Statsig's client-side On-Device Eval SDKs provide an alternate client-side architecture. The device keeps the definition of each experiment …”

   proof ↗⚑ flag

   GrowthBook✓fullprobed9/10

   GrowthBook's core architecture is a cacheable JSON ruleset that SDKs evaluate locally, confirmed by docs (SDK Connection Endpoint providing readonly flag data for local assignment, in-memory attribute evaluation) and corroborated by a community founder comment describing exactly this model plus a runtime probe showing the keyless payload endpoint serving flag data for local SDK evaluation. Missing for 10: explicit published latency/microsecond benchmarks or independent third-party performance testing.

   • [claimed-docs] “The SDK Connection Endpoint provides readonly access to a subset of your feature flag data, just enough for the GrowthBook SDKs to assign va…”
   • [claimed-docs] “The actual values of targeting attributes (e.g., user IDs, emails) are never sent to GrowthBook. They are only stored in memory locally with…”
   • [community] “With GrowthBook, there's a single cacheable JSON definition of all of the features and then evaluation happens locally within the SDKs... Th…”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official growthbook/growthbook docker image boot…”
   • [claimed-docs] “Run the GrowthBook Proxy next to your app to cache SDK payloads, reduce latency, and keep serving flags if the API is down.”

   proof ↗⚑ flag

Sdks

1. developerUse official SDKs across my whole stack — backend, web, and mobile — with consistent flag behavior

   weight 2 · round to GrowthBook
   Statsig~partialclaimed5/10

   Statsig documents deterministic evaluation guaranteeing identical results 'across different platforms (client or server)' and offers On-Device Eval client SDKs plus a JS SDK tutorial that also targets mobile platforms, implying a consistent cross-stack SDK model. However, the pack never enumerates specific backend/web/mobile SDK names or shows hands-on parity testing across a real multi-platform stack. Missing for 10: an explicit list of officially supported language/platform SDKs, and independent/hands-on evidence confirming identical flag behavior when used together across backend, web, and mobile in a real project.

   • [claimed-docs] “Evaluation in Statsig is deterministic. Given the same user object and the same state of the experiment or feature gate, Statsig always retu…”
   • [claimed-docs] “Evaluation in Statsig is deterministic. Given the same user object and the same state of the experiment or feature gate, Statsig always retu…”
   • [claimed-docs] “Statsig's client-side On-Device Eval SDKs provide an alternate client-side architecture. The device keeps the definition of each experiment …”
   • [claimed-docs] “Statsig's client-side On-Device Eval SDKs provide an alternate client-side architecture. The device keeps the definition of each experiment …”
   • [claimed-docs] “This tutorial creates a feature gate, targets mobile platforms and internal testers, then checks it live with the JavaScript SDK in your bro…”

   proof ↗⚑ flag

   GrowthBook✓fullcommunity7/10

   GrowthBook offers 24 official SDKs spanning backend, web/React, and mobile (Android/iOS) per GitHub README, with docs showing consistent flag-evaluation patterns (useFeatureIsOn, useFeatureValue, targeting attributes evaluated locally) and community reports of quick multi-stack integration (Vue frontend in hours). An older 2021 community comment noted no native mobile SDKs yet, but this is superseded by the more recent GitHub claim explicitly listing Android/iOS SDKs, so it's not a live contradiction. Missing for 10: independent verification of identical targeting/rollout behavior across all 24 SDKs, and more recent hands-on confirmation of mobile SDK maturity/parity with backend/web SDKs.

   • [github] “24 SDKs including React, Python, Android, and iOS.”
   • [claimed-docs] “Get GrowthBook feature flags and experiments running in your app in a few lines of code, then pick a language-specific SDK.”
   • [claimed-docs] “const showNewFeature = useFeatureIsOn("new-feature");”
   • [claimed-docs] “const showNewFeature = useFeatureIsOn("new-feature"); const buttonColor = useFeatureValue("button-color", "blue");”
   • [claimed-docs] “Add targeting conditions to a rule to control which values a user sees.”
   • [community] “At Miggos we have implemented Growth book and we got FF and A/B in our Vue frontend in a few hours. Excited to see the progress of the platf…”
   • [community] “We don't have native mobile SDKs yet, but it's something we want to support in the future. Mobile is a little tricky since you either need t…”

   proof ↗⚑ flag

Standards

1. platform engineerUse the vendor through OpenFeature providers so my flag code isn't locked to one vendor's SDK API

   weight 1 · round drawn
   Statsig—none0/10

   No evidence pack item mentions OpenFeature or a provider implementation for it; all SDK references are to Statsig's own proprietary SDKs, CLI, HTTP API, and Console API. This is a fair axis for a feature-flag/experimentation platform, so absence of evidence yields 'none'.

   ⚑ flag

   GrowthBook—none0/10

   The evidence pack describes GrowthBook's own 24 SDKs, REST API, and proxy, but contains no mention of OpenFeature or an OpenFeature provider integration anywhere. Since this axis (avoiding vendor lock-in via OpenFeature) plausibly applies to a flagging platform, the absence of any supporting evidence yields 'none' rather than 'na'.

   • [github] “24 SDKs including React, Python, Android, and iOS.”
   • [claimed-docs] “Get GrowthBook feature flags and experiments running in your app in a few lines of code, then pick a language-specific SDK.”
   • [claimed-docs] “Get GrowthBook feature flags and experiments running in your app in a few lines of code”

   proof ↗⚑ flag

Streaming

1. developerFlag changes propagate to connected SDKs in seconds via streaming or fast polling — a kill switch actually kills

   weight 2 · round to Statsig
   Statsig~partialclaimed7/10

   Statsig's docs explicitly describe real-time toggling and an 'emergency disable switch' to immediately turn off a code branch in production, plus deterministic evaluation and on-device SDKs for fast client updates — directly supporting the kill-switch and fast-propagation story. However, there is no explicit documentation of streaming vs polling mechanics, propagation latency in seconds, or independent/hands-on verification of SDK update speed. Missing for 10: documented streaming/polling interval specifics, third-party benchmark or hands-on confirmation of sub-second/seconds-level propagation.

   • [claimed-docs] “Feature Gates, commonly known as feature flags, allow you to toggle the behavior of your product in real time without deploying new code.”
   • [claimed-docs] “Have an emergency disable switch that lets you immediately turn off a particular code branch for users in production”
   • [claimed-docs] “Evaluation in Statsig is deterministic. Given the same user object and the same state of the experiment or feature gate, Statsig always retu…”
   • [claimed-docs] “Statsig's client-side On-Device Eval SDKs provide an alternate client-side architecture. The device keeps the definition of each experiment …”

   proof ↗⚑ flag

   GrowthBook~partialcommunity4/10

   GrowthBook's docs confirm a single cacheable feature-flag JSON payload that SDKs evaluate locally (growthbook-comm-6), a Proxy that caches payloads to reduce latency and keep serving flags if the API is down (growthbook-docs-11), and environment/rule toggles that could function as kill switches (growthbook-docs-3, growthbook-docs-22). However, no evidence in the pack documents actual streaming/SSE or fast-polling update latency in seconds, nor confirms a tested kill-switch effect at runtime. Missing for 10: explicit documentation of push/streaming update mechanism and propagation-time guarantees, and a hands-on/community test showing a kill switch immediately disabling a flag across connected SDKs.

   • [claimed-docs] “Run the GrowthBook Proxy next to your app to cache SDK payloads, reduce latency, and keep serving flags if the API is down.”
   • [community] “With GrowthBook, there's a single cacheable JSON definition of all of the features and then evaluation happens locally within the SDKs... Th…”
   • [claimed-docs] “Use environments such as dev, staging, and production to enable or disable feature flags independently.”
   • [claimed-docs] “How rules override a feature's default value — targeting users, rolling out gradually, experimenting — and how to scope, ramp, and schedule …”
   • [claimed-docs] “The SDK Connection Endpoint provides readonly access to a subset of your feature flag data, just enough for the GrowthBook SDKs to assign va…”

   proof ↗⚑ flag