GrowthBook vs Flagsmith

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to GrowthBook
   GrowthBook✓fullprobed9/10

   A probe confirms llms.txt is live at docs.growthbook.io/llms.txt returning HTTP 200 with a structured index of 180 documentation pages, exactly matching the agent-oriented docs pattern; GrowthBook also documents an official MCP server for agent integration, reinforcing agent-friendliness. Missing for 10: no independent third-party confirmation of an agent successfully consuming llms.txt end-to-end beyond the probe.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.growthbook.io/llms.txt # GrowthBook Documentation - [Docs (180 pages)](https://docs.growthbook.io/…”
   • [claimed-docs] “Connect AI tools like Cursor, VS Code, and Claude to GrowthBook using the Model Context Protocol (MCP).”
   • [claimed-docs] “The official GrowthBook MCP server is a thin bridge: it loads GrowthBook Agent Skills (workflows and guardrails) and makes authenticated cal…”
   • [probe] “official MCP server documented at https://docs.growthbook.io/integrations/mcp”

   proof ↗⚑ flag

   Flagsmith~partialprobed5/10

   Flagsmith has no llms.txt file (confirmed 404 via direct probe), but it does publish agent-oriented documentation — a dedicated docs page describing the MCP server 'for AI assistants and agents' to programmatically access the Admin API, plus a documented CLI. This is genuine agent-facing documentation, just not in the llms.txt convention. Missing for 10: an actual llms.txt or equivalent machine-readable agent doc index, and independent confirmation an agent can successfully consume the MCP docs page as a pointer target.

   • [probe] “PROBE llms.txt: HTTP 404 at https://docs.flagsmith.com/llms.txt”
   • [claimed-docs] “The Flagsmith MCP Server gives AI assistants and agents programmatic access to the Flagsmith Admin API through the Model Context Protocol”
   • [probe] “official MCP server documented at https://docs.flagsmith.com/integrating-with-flagsmith/mcp-server”
   • [claimed-docs] “The Flagsmith CLI lets you manage flags, segments, features, projects and environments from your terminal, and evaluate flags the way an SDK…”
   • [probe] “official CLI documented at https://docs.flagsmith.com/integrating-with-flagsmith/CLI”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round drawn
   GrowthBook✓fullprobed8/10

   GrowthBook supports headless/CI usage via self-hosted Docker Compose deployment, a full REST API, SDK Connection Endpoints for programmatic flag evaluation, and inline server-side experiments requiring no third-party requests — all scriptable without a UI. A runtime probe confirms a real keyless self-host create-and-evaluate loop via API/docker, demonstrating genuine automation capability. missing for 10: no explicit CI/CD pipeline example or GitHub Actions integration doc, and no evidence of a dedicated CLI tool for automation workflows.

   • [claimed-docs] “Run GrowthBook on your own infrastructure with Docker Compose, including MongoDB, the API, and the front-end.”
   • [claimed-docs] “GrowthBook offers a full REST API for interacting with the application.”
   • [claimed-docs] “You can also run server-side experiments by using inline experiments directly with our SDK. This requires no 3rd party requests”
   • [claimed-docs] “The SDK Connection Endpoint provides readonly access to a subset of your feature flag data, just enough for the GrowthBook SDKs to assign va…”
   • [claimed-docs] “You can use docker compose to get started quickly... run docker compose up -d to start everything and view the app at http://localhost:3000”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official growthbook/growthbook docker image boot…”

   proof ↗⚑ flag

   Flagsmith✓fullprobed8/10

   Flagsmith ships an official CLI and Admin API that can create/evaluate flags entirely headlessly, and a probe confirms a full keyless self-hosted Docker create-and-evaluate loop with no UI involved, plus server-side SDKs for automated environments. missing for 10: explicit CI/CD pipeline examples (e.g., GitHub Actions) and independent third-party CI usage reports.

   • [claimed-docs] “The Flagsmith CLI lets you manage flags, segments, features, projects and environments from your terminal, and evaluate flags the way an SDK…”
   • [claimed-docs] “Anything you can do in the Flagsmith dashboard, you can also do via the Admin API.”
   • [claimed-docs] “This is the API you use to programmatically manage your Flagsmith projects. Anything you can do in the Flagsmith dashboard, you can also do …”
   • [claimed-docs] “You can use Docker to set up an entire Flagsmith Feature Flag environment locally”
   • [claimed-docs] “You can use Docker to set up an entire [Flagsmith Feature Flag](https://www.flagsmith.com) environment locally”
   • [probe] “official CLI documented at https://docs.flagsmith.com/integrating-with-flagsmith/CLI”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official flagsmith/flagsmith unified docker imag…”

   proof ↗⚑ flag

3. ai-native userConnect an agent via an official MCP server

   weight 3 · round drawn
   GrowthBook✓fullprobed9/10

   GrowthBook documents an official MCP server that bridges AI tools (Cursor, VS Code, Claude) to GrowthBook via authenticated REST calls with Agent Skills/guardrails, and a runtime probe confirms the official npm package (@growthbook/mcp) completes a full stdio initialize handshake exposing read/write meta-tools. This is strong first-party plus hands-on evidence of a working official MCP server. Missing for 10: broader independent/community corroboration beyond the single runtime probe.

   • [claimed-docs] “Connect AI tools like Cursor, VS Code, and Claude to GrowthBook using the Model Context Protocol (MCP).”
   • [claimed-docs] “The official GrowthBook MCP server is a thin bridge: it loads GrowthBook Agent Skills (workflows and guardrails) and makes authenticated cal…”
   • [probe] “official MCP server documented at https://docs.growthbook.io/integrations/mcp”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official npm MCP server (`npx -y @growthbook/mcp`) completed a FULL keyless stdio initialize handsh…”

   proof ↗⚑ flag

   Flagsmith✓fullprobed9/10

   Flagsmith documents an official MCP Server that gives AI assistants/agents programmatic access to the Admin API via the Model Context Protocol, and a runtime probe confirms a live hosted MCP endpoint (mcp.flagsmith.com) responding to JSON-RPC initialize calls (key-gated). This is first-party, dedicated, and independently verified as functioning. Missing for 10: no independent/community hands-on report of actually using an agent through the MCP server in a real workflow.

   • [claimed-docs] “The Flagsmith MCP Server gives AI assistants and agents programmatic access to the Flagsmith Admin API through the Model Context Protocol”
   • [claimed-docs] “The Flagsmith MCP Server gives AI assistants and agents programmatic access to the Flagsmith Admin API”
   • [probe] “official MCP server documented at https://docs.flagsmith.com/integrating-with-flagsmith/mcp-server”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the documented hosted MCP server https://mcp.flagsmith.com returned…”

   proof ↗⚑ flag

4. ai-native userUse an official CLI

   weight 2 · round to Flagsmith
   GrowthBook—none0/10

   The evidence pack documents SDKs, REST API, self-host Docker, and an official MCP server, but there is no mention anywhere of an official standalone CLI tool for GrowthBook. Since a platform like GrowthBook could plausibly ship a CLI, the axis applies, but with no supporting evidence the verdict is none.

   ⚑ flag

   Flagsmith✓fullprobed8/10

   Flagsmith documents an official CLI that manages flags, segments, features, projects and environments and can evaluate flags like an SDK, confirmed live at the documented URL. missing for 10: independent/hands-on community verification of the CLI's functionality and details on installation/versioning maturity.

   • [claimed-docs] “The Flagsmith CLI lets you manage flags, segments, features, projects and environments from your terminal, and evaluate flags the way an SDK…”
   • [probe] “official CLI documented at https://docs.flagsmith.com/integrating-with-flagsmith/CLI”

   proof ↗⚑ flag

5. ai-native userDrive the product through a documented public API

   weight 3 · round to GrowthBook
   GrowthBook✓fullprobed9/10

   GrowthBook documents a full REST API for interacting with the application (growthbook-docs-12), including a readonly SDK Connection Endpoint (growthbook-docs-20) and unauthenticated public endpoints (growthbook-docs-41), and a runtime probe confirms an actual API-driven create-and-evaluate loop (creating org, feature flag, SDK connection, and reading it back) worked end-to-end (growthbook-probe-rt-2). Missing for 10: a discoverable OpenAPI/swagger spec file (all candidate paths 404'd per growthbook-probe-2), which would make the API more machine-discoverable for agentic tooling.

   • [claimed-docs] “GrowthBook offers a full REST API for interacting with the application.”
   • [claimed-docs] “The SDK Connection Endpoint provides readonly access to a subset of your feature flag data, just enough for the GrowthBook SDKs to assign va…”
   • [claimed-docs] “They are meant to be public and do not require authentication to view.”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official growthbook/growthbook docker image boot…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.growthbook.io/openapi.json, https://docs.growthbook.io/swagger.json, https://docs.growt…”

   proof ↗⚑ flag

   Flagsmith✓fullprobed8/10

   Flagsmith exposes a well-documented Admin API ('Anything you can do in the Flagsmith dashboard, you can also do via the Admin API'), a public CLI, and an official MCP server explicitly built for AI assistants/agents, with a runtime probe confirming the hosted MCP endpoint is live and key-gated and a self-hosted API create/evaluate roundtrip working keylessly. This directly satisfies driving the product via documented public API for an AI-native user. Missing for 10: a discoverable OpenAPI/swagger spec or llms.txt (both probed and returned 404), and independent/third-party corroboration of the MCP server's real-world agent usage beyond first-party docs.

   • [claimed-docs] “The Flagsmith MCP Server gives AI assistants and agents programmatic access to the Flagsmith Admin API through the Model Context Protocol”
   • [claimed-docs] “The Flagsmith MCP Server gives AI assistants and agents programmatic access to the Flagsmith Admin API”
   • [claimed-docs] “Anything you can do in the Flagsmith dashboard, you can also do via the Admin API.”
   • [claimed-docs] “This is the API you use to programmatically manage your Flagsmith projects. Anything you can do in the Flagsmith dashboard, you can also do …”
   • [claimed-docs] “The Flagsmith CLI lets you manage flags, segments, features, projects and environments from your terminal, and evaluate flags the way an SDK…”
   • [probe] “official MCP server documented at https://docs.flagsmith.com/integrating-with-flagsmith/mcp-server”
   • [probe] “official CLI documented at https://docs.flagsmith.com/integrating-with-flagsmith/CLI”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the documented hosted MCP server https://mcp.flagsmith.com returned…”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official flagsmith/flagsmith unified docker imag…”
   • [probe] “PROBE llms.txt: HTTP 404 at https://docs.flagsmith.com/llms.txt”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.flagsmith.com/openapi.json, https://docs.flagsmith.com/swagger.json, https://docs.flags…”

   proof ↗⚑ flag

6. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round drawn
   GrowthBook~partialprobed5/10

   GrowthBook documents a read-only SDK Connection Endpoint that exposes only a subset of feature-flag data (least-privilege by design) and a full REST API, plus an 'Advanced access control' / 'Advanced permissioning' feature at higher pricing tiers, and the MCP server makes authenticated REST calls on behalf of an agent. However there is no explicit documentation of an admin flow for issuing a scoped, least-privilege API credential specifically for an AI agent (e.g., role-restricted API keys, token scoping UI, or agent-specific key generation). Missing for 10: documented API-key scoping/role assignment workflow, explicit agent-credential issuance guide, and evidence of granular per-tool permission limits for MCP/agent tokens beyond generic 'advanced access control' tier feature.

   • [claimed-docs] “The SDK Connection Endpoint provides readonly access to a subset of your feature flag data, just enough for the GrowthBook SDKs to assign va…”
   • [claimed-docs] “They are meant to be public and do not require authentication to view.”
   • [claimed-docs] “GrowthBook offers a full REST API for interacting with the application.”
   • [claimed-docs] “Pro, plus: Custom environments Ramp schedules Approval workflows Advanced access control SSO & SCIM provisioning Exportable audit logs”
   • [claimed-docs] “Custom environments * Ramp schedules * Approval workflows * Advanced access control * SSO & SCIM provisioning * Exportable audit l…”
   • [claimed-docs] “The official GrowthBook MCP server is a thin bridge: it loads GrowthBook Agent Skills (workflows and guardrails) and makes authenticated cal…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official npm MCP server (`npx -y @growthbook/mcp`) completed a FULL keyless stdio initialize handsh…”

   proof ↗⚑ flag

   Flagsmith~partialprobed5/10

   Flagsmith exposes an Admin API with RBAC for fine-grained access control, and its documented MCP server requires an API key to authenticate agent access (confirmed via a live 401 probe), showing credential-gated agent access. However, there is no explicit documentation of issuing scoped, least-privilege tokens specifically for agents (e.g., read-only or resource-limited keys) beyond general user-role RBAC. Missing for 10: explicit agent-specific scoped/short-lived API key issuance, documented token scopes (read/write/resource-limited), and independent confirmation of least-privilege enforcement for MCP/agent credentials.

   • [claimed-docs] “Role-based access control (RBAC) provides fine-grained access management of Flagsmith resources.”
   • [claimed-docs] “Only allow certain users to modify your production environments.”
   • [claimed-docs] “Purpose: Creating, updating, and deleting projects, environments, flags, segments, and users.”
   • [claimed-docs] “The Flagsmith MCP Server gives AI assistants and agents programmatic access to the Flagsmith Admin API through the Model Context Protocol”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the documented hosted MCP server https://mcp.flagsmith.com returned…”
   • [probe] “official MCP server documented at https://docs.flagsmith.com/integrating-with-flagsmith/mcp-server”

   proof ↗⚑ flag

7. ai-native userBuild against official SDKs

   weight 2 · round to GrowthBook
   GrowthBook✓fullcommunity9/10

   GrowthBook ships 24 official language-specific SDKs (React, Python, Android, iOS, etc.) with documented quickstart code, local evaluation, targeting, and inline experiments, and this is corroborated by community hands-on reports of fast SDK integration. missing for 10: independent benchmarking across all 24 SDKs and more recent third-party validation beyond a single HN thread.

   • [github] “24 SDKs including React, Python, Android, and iOS.”
   • [claimed-docs] “Get GrowthBook feature flags and experiments running in your app in a few lines of code, then pick a language-specific SDK.”
   • [claimed-docs] “const showNewFeature = useFeatureIsOn("new-feature");”
   • [claimed-docs] “const showNewFeature = useFeatureIsOn("new-feature"); const buttonColor = useFeatureValue("button-color", "blue");”
   • [claimed-docs] “You can also run server-side experiments by using inline experiments directly with our SDK. This requires no 3rd party requests”
   • [community] “Been using GrowthBook for a few weeks now... I was shocked at how much setup time it took to get a simple feature flag into our app with oth…”
   • [community] “At Miggos we have implemented Growth book and we got FF and A/B in our Vue frontend in a few hours. Excited to see the progress of the platf…”

   proof ↗⚑ flag

   Flagsmith~partialclaimed6/10

   Flagsmith advertises official SDKs across 15+ languages and frameworks (React, Next.js, .NET, Java, etc.) and positions itself for web, mobile, and server-side integration, giving AI-native developers concrete SDKs to build against. However the evidence pack lacks SDK-specific documentation depth (API references, code samples, changelogs) beyond a marketing blurb. Missing for 10: dedicated per-SDK docs/examples, versioning/maintenance evidence, and independent corroboration of SDK quality.

   • [github] “Choose from 15+ popular languages like Typescript, .NET, Java, and more. Integrate with any framework, including React, Next.js, and more”
   • [claimed-docs] “Manage feature flags and remote config across web, mobile, and server-side apps.”

   proof ↗⚑ flag

8. ai-native userSubscribe to events via webhooks

   weight 2 · round to Flagsmith
   GrowthBook—none0/10

   The evidence pack covers SDKs, REST API, MCP integration, self-hosting, and experimentation features but contains no mention of webhooks or event subscription mechanisms anywhere, so there is no evidence GrowthBook supports this capability.

   ⚑ flag

   Flagsmith~partialclaimed5/10

   Flagsmith documents Audit Log Webhooks that let you stream events (flag/segment/identity changes) into your own infrastructure, which is a real webhook subscription mechanism. However, evidence only covers audit-log-triggered webhooks, not a broader event/webhook API for arbitrary flag or environment change subscriptions, and there's no independent corroboration of reliability. Missing for 10: documentation of a general-purpose webhook subscription API beyond audit logs, and any hands-on/independent verification of webhook delivery in production use.

   • [claimed-docs] “You can stream your Audit Logs into your own infrastructure using Audit Log Webhooks.”
   • [claimed-docs] “Every action taken within the Flagsmith administration application is tracked and logged.”

   proof ↗⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round to GrowthBook
   GrowthBook~partialprobed4/10

   GrowthBook ships an official MCP server that lets AI tools like Cursor, VS Code, and Claude connect and make authenticated calls to pull experiment/feature data and get GrowthBook 'Agent Skills' workflows/guardrails, which can produce AI-generated insights and suggestions — this was verified working in a runtime probe. However, this capability lives entirely outside the product (in an external AI client) rather than as a native in-app AI insights feature; GrowthBook's own results UI only shows statistical outputs like 'Chance to Win', not LLM-generated narrative insights. Missing for 10: a native in-product AI insights/summarization feature surfaced in the GrowthBook UI itself, and evidence of automated suggestion-generation (e.g., 'this metric moved because X') without requiring an external MCP client.

   • [claimed-docs] “Connect AI tools like Cursor, VS Code, and Claude to GrowthBook using the Model Context Protocol (MCP).”
   • [claimed-docs] “The official GrowthBook MCP server is a thin bridge: it loads GrowthBook Agent Skills (workflows and guardrails) and makes authenticated cal…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official npm MCP server (`npx -y @growthbook/mcp`) completed a FULL keyless stdio initialize handsh…”
   • [claimed-docs] “Chance to Win" tells you the probability that the variation is better. Anything above 95%”

   proof ↗⚑ flag

   Flagsmith—none0/10

   Flagsmith provides an MCP server for external AI agents to call its Admin API (docs-17/29) and a 'release pipeline' described as an 'intelligent release assistant' (docs-37), but this is rule-based automation and API access, not evidence of the product itself generating AI insights or suggestions from data (e.g., anomaly detection, usage analysis, recommended flags/segments). No documented AI/ML-driven insight or suggestion feature is present in the evidence pack.

   • [claimed-docs] “Release Pipelines act as your intelligent release assistant, automating your entire feature flag deployment process.”
   • [claimed-docs] “The Flagsmith MCP Server gives AI assistants and agents programmatic access to the Flagsmith Admin API through the Model Context Protocol”
   • [claimed-docs] “The Flagsmith MCP Server gives AI assistants and agents programmatic access to the Flagsmith Admin API”

   proof ↗⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Flagsmith
   GrowthBook~partialprobed4/10

   GrowthBook has some automated background behaviors (automatic stale flag detection, ramp schedules, and 'safe rollouts' that presumably auto-guard rollouts), and its MCP server lets AI tools make authenticated calls to the product, but these are on-demand agent actions or passive detection features, not evidence of an AI agent configuring a persistent job that runs autonomously in the background on a schedule/trigger without human involvement. Missing for 10: a documented scheduler/trigger system for AI-configured autonomous jobs, evidence of MCP-driven automations persisting and executing unattended, and independent confirmation of any autonomous background workflow.

   • [claimed-docs] “GrowthBook can automatically detect these flags so you can clean them up.”
   • [claimed-docs] “How rules override a feature's default value — targeting users, rolling out gradually, experimenting — and how to scope, ramp, and schedule …”
   • [claimed-docs] “Pro, plus: Custom environments Ramp schedules Approval workflows Advanced access control SSO & SCIM provisioning Exportable audit logs”
   • [claimed-docs] “Custom environments * Ramp schedules * Approval workflows * Advanced access control * SSO & SCIM provisioning * Exportable audit l…”
   • [claimed-docs] “Starter, plus: Visual editor Multi-arm bandits Safe rollouts Customizable dashboards Advanced permissioning Power calculator Premium support”
   • [claimed-docs] “The official GrowthBook MCP server is a thin bridge: it loads GrowthBook Agent Skills (workflows and guardrails) and makes authenticated cal…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official npm MCP server (`npx -y @growthbook/mcp`) completed a FULL keyless stdio initialize handsh…”

   proof ↗⚑ flag

   Flagsmith~partialprobed7/10

   Flagsmith documents concrete autonomous background automation: Scheduled Flags apply changes at a future time without manual intervention, and Release Pipelines define stages with triggers/actions that automatically progress rollouts to environments/audiences, generating change requests on promotion. The MCP server also lets AI agents programmatically drive these flows, and a runtime probe confirms the hosted MCP endpoint is live (though key-gated, un-exercised end-to-end). Missing for 10: independent/hands-on verification that Release Pipelines actually execute staged autonomous rollouts in practice, and a verified authenticated MCP-driven automation run rather than just a 401 probe.

   • [claimed-docs] “Scheduled flags allow you to queue and automatically apply changes to feature flags at a specified future time, eliminating the need for man…”
   • [claimed-docs] “Scheduled flags allow you to queue and automatically apply changes to feature flags at a specified future time”
   • [claimed-docs] “Define stages that your flags progress through automatically, with triggers and actions that control rollouts to selected environments and a…”
   • [claimed-docs] “Release Pipelines can automatically generate change requests when promoting flags to production, eliminating manual effort and streamlining …”
   • [claimed-docs] “Release Pipelines act as your intelligent release assistant, automating your entire feature flag deployment process.”
   • [claimed-docs] “The Flagsmith MCP Server gives AI assistants and agents programmatic access to the Flagsmith Admin API through the Model Context Protocol”
   • [claimed-docs] “The Flagsmith MCP Server gives AI assistants and agents programmatic access to the Flagsmith Admin API”
   • [probe] “official MCP server documented at https://docs.flagsmith.com/integrating-with-flagsmith/mcp-server”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the documented hosted MCP server https://mcp.flagsmith.com returned…”

   proof ↗⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round drawn
   GrowthBook—none0/10

   GrowthBook documents an MCP server that lets external AI tools (Cursor, VS Code, Claude) connect to and query GrowthBook — this is the opposite direction (GrowthBook as a data/tool source for external assistants), not a built-in AI assistant living inside the GrowthBook product itself. No evidence anywhere in the pack shows an in-app chat/assistant feature for delegating tasks within GrowthBook's UI.

   • [claimed-docs] “Connect AI tools like Cursor, VS Code, and Claude to GrowthBook using the Model Context Protocol (MCP).”
   • [claimed-docs] “The official GrowthBook MCP server is a thin bridge: it loads GrowthBook Agent Skills (workflows and guardrails) and makes authenticated cal…”
   • [probe] “official MCP server documented at https://docs.growthbook.io/integrations/mcp”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official npm MCP server (`npx -y @growthbook/mcp`) completed a FULL keyless stdio initialize handsh…”

   proof ↗⚑ flag

   Flagsmith—none0/10

   Flagsmith exposes an MCP server so external AI agents can call its Admin API, and calls its automated release pipeline an 'intelligent release assistant,' but neither is a built-in AI assistant inside the Flagsmith product that a user can converse with or delegate tasks to — the MCP server is a server-side integration point for external agents, not a first-party in-app assistant.

   • [claimed-docs] “The Flagsmith MCP Server gives AI assistants and agents programmatic access to the Flagsmith Admin API through the Model Context Protocol”
   • [claimed-docs] “The Flagsmith MCP Server gives AI assistants and agents programmatic access to the Flagsmith Admin API”
   • [claimed-docs] “Release Pipelines act as your intelligent release assistant, automating your entire feature flag deployment process.”
   • [probe] “official MCP server documented at https://docs.flagsmith.com/integrating-with-flagsmith/mcp-server”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the documented hosted MCP server https://mcp.flagsmith.com returned…”

   proof ↗⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round to GrowthBook
   GrowthBook✓fullprobed8/10

   GrowthBook ships an official MCP server that bridges AI tools like Cursor, VS Code, and Claude to GrowthBook, loading 'Agent Skills' (workflows/guardrails) and making authenticated REST calls so users can operate flags/experiments via natural language in an AI client. A runtime probe confirms the official npm MCP server completes a full keyless handshake exposing read/write meta-tools covering the whole API, corroborating the docs' claims beyond marketing copy. Missing for 10: an end-to-end example of an actual natural-language command executing a real write action (only the handshake/tool-discovery was verified, not a full command-to-result flow) and independent (non-vendor) hands-on confirmation.

   • [claimed-docs] “Connect AI tools like Cursor, VS Code, and Claude to GrowthBook using the Model Context Protocol (MCP).”
   • [claimed-docs] “The official GrowthBook MCP server is a thin bridge: it loads GrowthBook Agent Skills (workflows and guardrails) and makes authenticated cal…”
   • [probe] “official MCP server documented at https://docs.growthbook.io/integrations/mcp”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official npm MCP server (`npx -y @growthbook/mcp`) completed a FULL keyless stdio initialize handsh…”

   proof ↗⚑ flag

   Flagsmith~partialprobed7/10

   Flagsmith ships an official hosted MCP server (mcp.flagsmith.com) exposing the Admin API to AI assistants/agents, confirmed live via a runtime probe (401 requiring auth, not a failure), which enables natural-language operation through an MCP-compatible AI client, plus a CLI for terminal-based flag management. However, this is agent-mediated (via MCP client) rather than native NLP in the product itself, and there's no hands-on evidence of a full natural-language command actually being executed end-to-end through an AI assistant. Missing for 10: a demonstrated end-to-end natural-language interaction transcript, and independent (non-vendor) corroboration of the MCP server working with a real AI agent.

   • [claimed-docs] “The Flagsmith MCP Server gives AI assistants and agents programmatic access to the Flagsmith Admin API through the Model Context Protocol”
   • [claimed-docs] “The Flagsmith MCP Server gives AI assistants and agents programmatic access to the Flagsmith Admin API”
   • [probe] “official MCP server documented at https://docs.flagsmith.com/integrating-with-flagsmith/mcp-server”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the documented hosted MCP server https://mcp.flagsmith.com returned…”
   • [claimed-docs] “The Flagsmith CLI lets you manage flags, segments, features, projects and environments from your terminal, and evaluate flags the way an SDK…”
   • [probe] “official CLI documented at https://docs.flagsmith.com/integrating-with-flagsmith/CLI”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   GrowthBook—none0/10

   GrowthBook documents a REST API (growthbook-docs-12) but there is no evidence of an interactive, runnable API reference (e.g., Swagger/OpenAPI try-it console) — a direct probe for openapi/swagger specs returned 404 on all candidate paths, and no docs mention runnable code samples in an API explorer.

   • [claimed-docs] “GrowthBook offers a full REST API for interacting with the application.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.growthbook.io/openapi.json, https://docs.growthbook.io/swagger.json, https://docs.growt…”

   proof ↗⚑ flag

   Flagsmith—none0/10

   The evidence pack explicitly shows probes for an OpenAPI/Swagger spec and llms.txt returning 404s, and no citation describes an interactive, runnable API reference (e.g., Swagger UI, Postman collection, or live code playground). Only static API overview docs and an Admin API description are mentioned, none of which are runnable/interactive.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.flagsmith.com/openapi.json, https://docs.flagsmith.com/swagger.json, https://docs.flags…”
   • [probe] “PROBE llms.txt: HTTP 404 at https://docs.flagsmith.com/llms.txt”
   • [claimed-docs] “Authentication: Uses a public, non-secret Environment Key. Security: Open by design. The Environment Key can be exposed in client-side code.”
   • [claimed-docs] “This is the API you use to programmatically manage your Flagsmith projects. Anything you can do in the Flagsmith dashboard, you can also do …”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   GrowthBook—none0/10

   While GrowthBook documents a full REST API (growthbook-docs-12) and an SDK Connection Endpoint (growthbook-docs-20), the evidence pack shows an explicit probe for OpenAPI/Swagger spec files at expected paths that all returned 404, and no other evidence surfaces a downloadable OpenAPI or machine-readable spec.

   • [claimed-docs] “GrowthBook offers a full REST API for interacting with the application.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.growthbook.io/openapi.json, https://docs.growthbook.io/swagger.json, https://docs.growt…”

   proof ↗⚑ flag

   Flagsmith—none0/10

   Flagsmith documents a full Admin API and CLI/MCP integrations, but explicit probes for a machine-readable spec (openapi.json, swagger.json, .well-known/openapi.json) all returned 404, and no evidence pack item points to a downloadable OpenAPI/Swagger spec.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.flagsmith.com/openapi.json, https://docs.flagsmith.com/swagger.json, https://docs.flags…”
   • [claimed-docs] “This is the API you use to programmatically manage your Flagsmith projects. Anything you can do in the Flagsmith dashboard, you can also do …”
   • [claimed-docs] “Purpose: Creating, updating, and deleting projects, environments, flags, segments, and users.”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round to GrowthBook
   GrowthBook✓fullprobed8/10

   GrowthBook supports separate environments (dev/staging/production) for feature flags, self-hosting via Docker Compose for a fully isolated instance, and a verified runtime probe confirms a keyless self-host roundtrip creating and evaluating flags without any production account/license — enabling true sandbox testing. missing for 10: no explicit vendor guidance on a dedicated 'sandbox mode' distinct from staging environments, and no independent (non-vendor) confirmation of environment data isolation guarantees.

   • [claimed-docs] “Use environments such as dev, staging, and production to enable or disable feature flags independently.”
   • [claimed-docs] “Run GrowthBook on your own infrastructure with Docker Compose, including MongoDB, the API, and the front-end.”
   • [claimed-docs] “You can use docker compose to get started quickly... run docker compose up -d to start everything and view the app at http://localhost:3000”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official growthbook/growthbook docker image boot…”

   proof ↗⚑ flag

   Flagsmith~partialprobed7/10

   Flagsmith supports multiple environments (dev/staging/prod) and fully self-hostable Docker setups that let users create, toggle, and evaluate flags without touching production, and a probe confirms a full local create-and-evaluate flag loop working offline; RBAC also lets teams restrict who can modify production. However, none of this is framed specifically as an 'AI-native sandbox' testing workflow, and the MCP/agent-facing docs don't call out sandbox vs. production isolation explicitly. Missing for 10: an explicit sandbox/staging environment guide oriented at AI agents, and confirmation that the MCP server itself supports pointing at a non-prod environment safely.

   • [github] “Just wrap a section of code with a flag, and then use Flagsmith to toggle that feature on or off for different environments, users or user s…”
   • [claimed-docs] “Self-hosting Flagsmith allows you to run the complete Flagsmith platform within your own infrastructure, giving you full control over your d…”
   • [claimed-docs] “You can use Docker to set up an entire Flagsmith Feature Flag environment locally”
   • [claimed-docs] “You can use Docker to set up an entire [Flagsmith Feature Flag](https://www.flagsmith.com) environment locally”
   • [claimed-docs] “Only allow certain users to modify your production environments.”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official flagsmith/flagsmith unified docker imag…”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   GrowthBook—none0/10

   GrowthBook documents a REST API and SDK Connection Endpoint, but no evidence pack item describes API versioning scheme or a documented deprecation policy; the openapi/swagger probe returned 404s, suggesting no formal API spec is even discoverable. Missing for 10: any mention of API version numbers, backward-compatibility guarantees, or a deprecation/sunset policy.

   • [claimed-docs] “GrowthBook offers a full REST API for interacting with the application.”
   • [claimed-docs] “The SDK Connection Endpoint provides readonly access to a subset of your feature flag data, just enough for the GrowthBook SDKs to assign va…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.growthbook.io/openapi.json, https://docs.growthbook.io/swagger.json, https://docs.growt…”

   proof ↗⚑ flag

   Flagsmith—none0/10

   The evidence pack documents Flagsmith's Admin API, SDKs, CLI, and MCP server, but nowhere mentions API version numbers, a versioning scheme, or a documented deprecation policy; probes even show openapi.json/spec endpoints returning 404, suggesting no discoverable API contract documentation. Missing for 10: any statement of API versioning strategy, deprecation timelines, or migration guides for breaking changes.

   • [claimed-docs] “Authentication: Uses a public, non-secret Environment Key. Security: Open by design. The Environment Key can be exposed in client-side code.”
   • [claimed-docs] “Purpose: Creating, updating, and deleting projects, environments, flags, segments, and users.”
   • [claimed-docs] “Anything you can do in the Flagsmith dashboard, you can also do via the Admin API.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.flagsmith.com/openapi.json, https://docs.flagsmith.com/swagger.json, https://docs.flags…”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Flagsmith
   GrowthBook~partialprobed5/10

   GrowthBook exposes a full REST API (growthbook-docs-12) and an official MCP server that bridges AI tools to authenticated REST read/write calls across the whole API (growthbook-docs-21, growthbook-probe-rt-1), which could let an AI agent script operations across many flags/experiments programmatically. However, there is no documented bulk-specific endpoint, batch-update tool, or example of performing many-item operations in one call. Missing for 10: explicit bulk-create/bulk-update API endpoints, documented batch operation examples, and evidence of the MCP server or API actually handling many items in a single request rather than one-at-a-time calls.

   • [claimed-docs] “GrowthBook offers a full REST API for interacting with the application.”
   • [claimed-docs] “The official GrowthBook MCP server is a thin bridge: it loads GrowthBook Agent Skills (workflows and guardrails) and makes authenticated cal…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official npm MCP server (`npx -y @growthbook/mcp`) completed a FULL keyless stdio initialize handsh…”

   proof ↗⚑ flag

   Flagsmith~partialprobed6/10

   Flagsmith's Admin API, CLI, and MCP server expose full CRUD over flags, segments, projects, environments and users programmatically, which lets an AI-native user script operations across many items in a loop (docs-16, docs-18, docs-17/29, probe-3/4). However, there is no documented explicit bulk/batch endpoint, bulk import/export tool, or multi-item transactional operation — only single-resource CRUD calls that must be iterated. Missing for 10: dedicated bulk/batch API endpoints, bulk import/export documentation, evidence of atomic multi-item operations.

   • [claimed-docs] “Purpose: Creating, updating, and deleting projects, environments, flags, segments, and users.”
   • [claimed-docs] “The Flagsmith CLI lets you manage flags, segments, features, projects and environments from your terminal, and evaluate flags the way an SDK…”
   • [claimed-docs] “The Flagsmith MCP Server gives AI assistants and agents programmatic access to the Flagsmith Admin API through the Model Context Protocol”
   • [claimed-docs] “The Flagsmith MCP Server gives AI assistants and agents programmatic access to the Flagsmith Admin API”
   • [probe] “official MCP server documented at https://docs.flagsmith.com/integrating-with-flagsmith/mcp-server”
   • [probe] “official CLI documented at https://docs.flagsmith.com/integrating-with-flagsmith/CLI”
   • [claimed-docs] “This is the API you use to programmatically manage your Flagsmith projects. Anything you can do in the Flagsmith dashboard, you can also do …”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to Flagsmith
   GrowthBook~partialclaimed5/10

   GrowthBook lets users define feature-flag 'rules' that automatically apply based on targeting conditions, scheduling, and ramping (docs-22, docs-28, docs-2), and has automated stale-flag detection (docs-4) — these are rule-based automations tied to feature evaluation. However, this falls short of a general 'on event X, trigger action Y' automation engine; there's no evidence of webhook-driven arbitrary actions, external event ingestion, or an automation/workflow layer beyond flag targeting and rollout scheduling. Missing for 10: generic event-trigger/webhook automation outside feature-flag context, evidence of arbitrary action execution on external events, and independent corroboration of rule automation depth.

   • [claimed-docs] “How rules override a feature's default value — targeting users, rolling out gradually, experimenting — and how to scope, ramp, and schedule …”
   • [claimed-docs] “Add targeting conditions to a rule to control which values a user sees.”
   • [claimed-docs] “GrowthBook can automatically detect these flags so you can clean them up.”
   • [claimed-docs] “Add targeting conditions to a rule to control which values a user sees. Conditions are evaluated against attributes you pass into the SDK”

   proof ↗⚑ flag

   Flagsmith~partialclaimed6/10

   Release Pipelines let users define stages with 'triggers and actions' that automatically progress flags/rollouts to environments and audiences, and Scheduled Flags apply changes automatically at a specified time — both are rule-based automated actions. Audit Log Webhooks can stream internal events for downstream automation. However, evidence covers only time/stage-based and rollout triggers rather than generic arbitrary-event-triggered actions (e.g., inbound webhook conditions tied to external signals), and there's no independent/hands-on corroboration of the pipeline automation working as described. Missing for 10: broader event-source flexibility (e.g., external event ingestion triggering flag changes), and independent verification of release-pipeline automation in practice.

   • [claimed-docs] “Define stages that your flags progress through automatically, with triggers and actions that control rollouts to selected environments and a…”
   • [claimed-docs] “Release Pipelines can automatically generate change requests when promoting flags to production, eliminating manual effort and streamlining …”
   • [claimed-docs] “Release Pipelines act as your intelligent release assistant, automating your entire feature flag deployment process.”
   • [claimed-docs] “Scheduled flags allow you to queue and automatically apply changes to feature flags at a specified future time, eliminating the need for man…”
   • [claimed-docs] “Scheduled flags allow you to queue and automatically apply changes to feature flags at a specified future time”
   • [claimed-docs] “You can stream your Audit Logs into your own infrastructure using Audit Log Webhooks.”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round to Flagsmith
   GrowthBook—none0/10

   Evidence only shows one-time scheduling of feature-flag rule start/stop dates and 'ramp schedules' for gradual rollouts (growthbook-docs-22, growthbook-docs-37) — nothing resembling recurring cron-like jobs or workflow automation that an AI-native user could schedule to run repeatedly. No mention of scheduled reports, recurring experiment analyses, or workflow automation triggers.

   • [claimed-docs] “How rules override a feature's default value — targeting users, rolling out gradually, experimenting — and how to scope, ramp, and schedule …”
   • [claimed-docs] “Custom environments * Ramp schedules * Approval workflows * Advanced access control * SSO & SCIM provisioning * Exportable audit l…”
   • [claimed-docs] “Pro, plus: Custom environments Ramp schedules Approval workflows Advanced access control SSO & SCIM provisioning Exportable audit logs”

   proof ↗⚑ flag

   Flagsmith~partialclaimed4/10

   Flagsmith supports 'Scheduled flags' to queue a one-time future flag change, and 'Release Pipelines' that automatically progress flags through stages with triggers/actions — both provide some automation depth, but neither is documented as a recurring/cron-style job scheduler that repeats on an interval. Missing for 10: explicit recurring/cron scheduling, repeatable automated workflow triggers beyond a single future timestamp, and any AI-agent-driven orchestration of repeating tasks.

   • [claimed-docs] “Scheduled flags allow you to queue and automatically apply changes to feature flags at a specified future time, eliminating the need for man…”
   • [claimed-docs] “Scheduled flags allow you to queue and automatically apply changes to feature flags at a specified future time”
   • [claimed-docs] “Define stages that your flags progress through automatically, with triggers and actions that control rollouts to selected environments and a…”
   • [claimed-docs] “Release Pipelines can automatically generate change requests when promoting flags to production, eliminating manual effort and streamlining …”
   • [claimed-docs] “Release Pipelines act as your intelligent release assistant, automating your entire feature flag deployment process.”

   proof ↗⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round to Flagsmith
   GrowthBook~partialcommunity6/10

   GrowthBook documents draft feature revisions, request-review, merge-conflict resolution, and required approval before publishing (docs-5, docs-22), and lists approval workflows plus exportable audit logs as paid-tier features (docs-37), which together enable versioning and review of flag/automation changes. However, no explicit rollback mechanism is documented and there's no independent/hands-on corroboration of the revision-history or rollback UI; a community comment even suggests config-as-code/git-based versioning isn't natively supported (growthbook-comm-11). Missing for 10: explicit rollback documentation, hands-on verification of revision history UI, and clarity on git-based config versioning.

   • [claimed-docs] “Draft feature revisions, request review, resolve merge conflicts, and require approval before publishing.”
   • [claimed-docs] “How rules override a feature's default value — targeting users, rolling out gradually, experimenting — and how to scope, ramp, and schedule …”
   • [claimed-docs] “Custom environments * Ramp schedules * Approval workflows * Advanced access control * SSO & SCIM provisioning * Exportable audit l…”
   • [community] “From the looks of it, it doesn't look like the configuration can be stored in the code repository itself. This is one of the key things to d…”

   proof ↗⚑ flag

   Flagsmith✓fullclaimed7/10

   Flagsmith provides a four-eyes Change Request workflow analogous to PR review (docs-10), full audit logs tracking every flag/segment/identity change over time (docs-8, docs-39), and Release Pipelines with automated stage-gated rollouts and auto-generated change requests (docs-6, docs-34, docs-37), plus instant toggle-off to revert behavior without redeploying (flagsmith-gh-1). Together these cover versioning, review, and effective rollback of flag-driven automations. missing for 10: an explicit named 'rollback to previous flag version' feature (evidence only shows audit trail + manual re-toggle, not one-click revert), and independent/hands-on confirmation of the change-request/pipeline workflow in practice.

   • [claimed-docs] “Every action taken within the Flagsmith administration application is tracked and logged. This allows you to easily retrace the events and v…”
   • [claimed-docs] “Feature Change Requests help creating a four-eyes workflow (create, approve, publish) to updating feature flags, similarly to Pull Requests …”
   • [claimed-docs] “Define stages that your flags progress through automatically, with triggers and actions that control rollouts to selected environments and a…”
   • [claimed-docs] “Release Pipelines can automatically generate change requests when promoting flags to production, eliminating manual effort and streamlining …”
   • [claimed-docs] “Release Pipelines act as your intelligent release assistant, automating your entire feature flag deployment process.”
   • [github] “Make changes remotely: Easily toggle individual features on and off, and make changes without deploying new code”
   • [claimed-docs] “Every action taken within the Flagsmith administration application is tracked and logged.”

   proof ↗⚑ flag

Proxy

1. platform engineerRun a relay/edge proxy so flags stay served when the vendor is unreachable and SDK traffic stays inside my network

   weight 1 · round to GrowthBook
   GrowthBook✓fullcommunity9/10

   GrowthBook explicitly documents a self-hostable Proxy that runs next to your app to cache SDK payloads, reduce latency, and keep serving flags if the API is down, directly matching the relay/edge proxy story, and this sits alongside full self-host Docker Compose deployment keeping SDK traffic in-network. Community evidence corroborates that evaluation happens locally within SDKs via a cacheable JSON payload, reinforcing offline resilience. Missing for 10: independent hands-on report specifically of the Proxy component (vs. general self-host) failing over during an outage.

   • [claimed-docs] “Run the GrowthBook Proxy next to your app to cache SDK payloads, reduce latency, and keep serving flags if the API is down.”
   • [claimed-docs] “Run GrowthBook on your own infrastructure with Docker Compose, including MongoDB, the API, and the front-end.”
   • [claimed-docs] “You can use docker compose to get started quickly... run docker compose up -d to start everything and view the app at http://localhost:3000”
   • [community] “With GrowthBook, there's a single cacheable JSON definition of all of the features and then evaluation happens locally within the SDKs... Th…”
   • [claimed-docs] “The SDK Connection Endpoint provides readonly access to a subset of your feature flag data, just enough for the GrowthBook SDKs to assign va…”

   proof ↗⚑ flag

   Flagsmith✓fullclaimed8/10

   Flagsmith documents a self-hosted Edge Proxy service that runs the Flagsmith Engine within your own infrastructure and is connected to directly by server-side SDKs, keeping evaluation traffic local and resilient to vendor outages. This directly matches the platform-engineer story of running a relay/edge proxy for network-local flag serving. Missing for 10: independent hands-on validation of failover behavior when the vendor API is unreachable, and more detail on sync/refresh mechanics between proxy and main API.

   • [claimed-docs] “The Flagsmith Edge Proxy is a service that you host yourself, that allows you to run an instance of the Flagsmith Engine close to your serve…”
   • [claimed-docs] “Run the Edge Proxy within in your own infrastructure and connect to it from your server-side SDKs”
   • [claimed-docs] “Self-hosting Flagsmith allows you to run the complete Flagsmith platform within your own infrastructure, giving you full control over your d…”

   proof ↗⚑ flag

Self host

1. platform engineerSelf-host the full flag platform from an open-source distribution, keeping evaluation data on my infrastructure

   weight 3 · round drawn
   GrowthBook✓fullprobed9/10

   GrowthBook documents a self-host distribution via Docker Compose (MongoDB, API, front-end) and this is corroborated by a hands-on probe showing a full keyless self-host boot, feature creation, and SDK payload evaluation working end-to-end on local infrastructure, plus community confirmation of production self-hosted usage. missing for 10: independent third-party audit of data residency guarantees beyond vendor docs/probe.

   • [claimed-docs] “Run GrowthBook on your own infrastructure with Docker Compose, including MongoDB, the API, and the front-end.”
   • [claimed-docs] “You can use docker compose to get started quickly... run docker compose up -d to start everything and view the app at http://localhost:3000”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official growthbook/growthbook docker image boot…”
   • [community] “Growthbook is awesome! We have used the self-hosted version a few months in my company now, and are very happy about it. Talented people beh…”

   proof ↗⚑ flag

   Flagsmith✓fullprobed9/10

   Flagsmith's docs explicitly describe self-hosting the complete platform for full data/security control, with a Docker-based setup guide, and this was independently verified via a real keyless self-host probe that created an org/project/environment/flag and successfully evaluated it against the local instance. missing for 10: no independent third-party audit of production-scale self-hosted deployments or HA/scaling guidance beyond the basic Docker quickstart.

   • [claimed-docs] “Self-hosting Flagsmith allows you to run the complete Flagsmith platform within your own infrastructure, giving you full control over your d…”
   • [claimed-docs] “You can use Docker to set up an entire Flagsmith Feature Flag environment locally”
   • [claimed-docs] “You can use Docker to set up an entire [Flagsmith Feature Flag](https://www.flagsmith.com) environment locally”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official flagsmith/flagsmith unified docker imag…”

   proof ↗⚑ flag

Agent ops

1. ai agentRead experiment configurations and results programmatically to summarize outcomes and recommend ship/rollback decisions

   weight 1 · round to GrowthBook
   GrowthBook✓fullprobed8/10

   GrowthBook exposes a full REST API and an official MCP server that makes authenticated calls to that API, explicitly loading 'Agent Skills (workflows and guardrails)' so AI tools can query feature/experiment data — confirmed working end-to-end in a runtime probe. Experiment results expose statistical decision signals like 'Chance to Win' (>95% highlighted as a clear winner) that directly support ship/rollback recommendations, and metrics/fact-table docs show how results are computed from stored data for programmatic reading. Missing for 10: a concrete worked example of an agent actually generating a ship/rollback summary via MCP, and independent (non-vendor) confirmation of recommendation quality.

   • [claimed-docs] “GrowthBook offers a full REST API for interacting with the application.”
   • [claimed-docs] “Connect AI tools like Cursor, VS Code, and Claude to GrowthBook using the Model Context Protocol (MCP).”
   • [claimed-docs] “The official GrowthBook MCP server is a thin bridge: it loads GrowthBook Agent Skills (workflows and guardrails) and makes authenticated cal…”
   • [claimed-docs] “Chance to Win" tells you the probability that the variation is better. Anything above 95%”
   • [claimed-docs] “Chance to Win tells you the probability that the variation is better. Anything above 95% ... is highlighted green indicating a very clear wi…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official npm MCP server (`npx -y @growthbook/mcp`) completed a FULL keyless stdio initialize handsh…”
   • [probe] “official MCP server documented at https://docs.growthbook.io/integrations/mcp”

   proof ↗⚑ flag

   Flagsmith~partialprobed5/10

   Flagsmith documents an Experimentation feature with a Bayesian statistics engine for reading A/B test results (flagsmith-docs-7/25/35/38), and separately an official MCP server plus Admin API giving agents programmatic access to flags/segments/environments (flagsmith-docs-17/29, probe-3, probe-rt-1). However, no evidence confirms the MCP/Admin API surface exposes the experiment analytics/results data (vs. just flag/segment CRUD), so an agent's ability to programmatically pull experiment outcomes (not just configs) for ship/rollback recommendations is unconfirmed. Missing for 10: documented API/MCP endpoint for reading experiment statistics results, and any example of an agent consuming that data to produce a recommendation.

   • [claimed-docs] “serve variations of a feature with a multivariate flag, collect events from your application into a managed data warehouse, and read the res…”
   • [claimed-docs] “The Flagsmith MCP Server gives AI assistants and agents programmatic access to the Flagsmith Admin API through the Model Context Protocol”
   • [claimed-docs] “The Flagsmith MCP Server gives AI assistants and agents programmatic access to the Flagsmith Admin API”
   • [probe] “official MCP server documented at https://docs.flagsmith.com/integrating-with-flagsmith/mcp-server”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the documented hosted MCP server https://mcp.flagsmith.com returned…”
   • [claimed-docs] “Flagsmith Experimentation lets you run A/B tests end to end on the platform: serve variations of a feature with a multivariate flag, collect…”
   • [claimed-docs] “Purpose: Creating, updating, and deleting projects, environments, flags, segments, and users.”

   proof ↗⚑ flag

Analysis

1. product managerTrust a documented statistics engine (Bayesian or frequentist, with variance-reduction options) behind experiment results

   weight 2 · round to GrowthBook
   GrowthBook✓fullcommunity9/10

   GrowthBook documents both Bayesian and Frequentist statistics engines with explicit UI outputs like 'Chance to Win', and its GitHub/community materials detail variance-reduction and rigor features (CUPED, Sequential testing, Post-Stratification, SRM checks) plus rationale for choosing Bayesian to mitigate peeking/Type I error. This is corroborated by both official docs and first-party community explanation, giving a PM strong documented trust in the underlying stats methodology; missing for 10: independent third-party statistical audit or academic validation of the engine's correctness.

   • [claimed-docs] “What you will see will depend a little bit on whether you are using our Bayesian or our Frequentist engine”
   • [claimed-docs] “Chance to Win" tells you the probability that the variation is better. Anything above 95%”
   • [claimed-docs] “Chance to Win tells you the probability that the variation is better. Anything above 95% ... is highlighted green indicating a very clear wi…”
   • [github] “World class experiment stats engine (CUPED, Sequential, Bayesian, Post-Strat, Bandits, SRM checks).”
   • [github] “🆎 World class experiment stats engine (CUPED, Sequential, Bayesian, Post-Strat, Bandits, SRM checks).”
   • [community] “We use a Bayesian statistics engine which is much more immune to peeking problems and Type I errors than frequentist approaches.”

   proof ↗⚑ flag

   Flagsmith~partialclaimed6/10

   Flagsmith explicitly documents an experimentation feature with a 'built-in Bayesian statistics engine' reading results from multivariate flags and warehoused events, giving product managers a named statistical method behind results. However, the docs give no detail on the underlying model, confidence intervals, sample-ratio checks, or frequentist alternative, and there is no independent/hands-on corroboration of the engine's correctness or variance-reduction techniques. Missing for 10: documentation of specific statistical methodology/assumptions, variance-reduction (e.g., CUPED) support, and third-party validation of the engine's results.

   • [claimed-docs] “serve variations of a feature with a multivariate flag, collect events from your application into a managed data warehouse, and read the res…”
   • [claimed-docs] “read the results with a built-in Bayesian statistics engine”
   • [claimed-docs] “collect events from your application into a managed data warehouse, and read the results with a built-in Bayesian statistics engine.”
   • [claimed-docs] “Flagsmith Experimentation lets you run A/B tests end to end on the platform: serve variations of a feature with a multivariate flag, collect…”

   proof ↗⚑ flag

Experiments

1. product managerRun A/B and multivariate experiments on flags and see which variation wins on my metrics

   weight 3 · round to GrowthBook
   GrowthBook✓fullcommunity8/10

   GrowthBook natively supports A/B and multivariate experiments attached to feature flags, with statistical engines (Bayesian/Frequentist, CUPED, Sequential, Bandits), templates for goal/secondary/guardrail metrics, and a results view showing 'Chance to Win' per variation to determine winners. Community reports corroborate quick setup and real-world use of feature flags plus A/B testing, though some note a gap between flags and full analytics requiring warehouse/metrics setup. missing for 10: independent hands-on validation of multivariate (>2 arm) experiment results and winner-declaration workflow specifically, beyond docs and community anecdotes.

   • [claimed-docs] “GrowthBook has a few different ways to run experiments or AB tests depending on your needs.”
   • [claimed-docs] “What you will see will depend a little bit on whether you are using our Bayesian or our Frequentist engine”
   • [claimed-docs] “Selecting a Template in the creation modal pre-fills settings like the data source, goal/secondary/guardrail metrics, targeting, and traffic”
   • [claimed-docs] “Define Fact Tables and metrics so GrowthBook can compute experiment results from your warehouse.”
   • [claimed-docs] “Chance to Win" tells you the probability that the variation is better. Anything above 95%”
   • [claimed-docs] “Run A/B tests with feature flags, the Visual Editor, URL redirects, or your own assignment, then analyze results in GrowthBook.”
   • [claimed-docs] “you can add an experiment rule to a feature that will randomly assign the users based on some hashing attribute into one of your experiment …”
   • [github] “World class experiment stats engine (CUPED, Sequential, Bayesian, Post-Strat, Bandits, SRM checks).”
   • [community] “With GrowthBook, there's a single cacheable JSON definition of all of the features and then evaluation happens locally within the SDKs... Th…”
   • [community] “We use a Bayesian statistics engine which is much more immune to peeking problems and Type I errors than frequentist approaches.”

   proof ↗⚑ flag

   Flagsmith✓fullclaimed7/10

   Flagsmith's Experimentation feature explicitly supports serving multivariate (A/B/n) flag variations, collecting events into a managed data warehouse, and reading results via a built-in Bayesian statistics engine, directly matching the PM story of running A/B/MVT tests and seeing which variation wins on metrics. Missing for 10: independent/hands-on evidence of the experimentation UI in action (e.g., screenshots or third-party reports of actual win/loss determination), and detail on which metrics/goals can be tracked or how statistical significance is surfaced to a non-technical PM.

   • [claimed-docs] “serve variations of a feature with a multivariate flag, collect events from your application into a managed data warehouse, and read the res…”
   • [claimed-docs] “Flagsmith Experimentation lets you run A/B tests end to end on the platform: serve variations of a feature with a multivariate flag, collect…”
   • [claimed-docs] “Multivariate flags let you define multiple variants with percentage weightings (A/B/n).”
   • [claimed-docs] “read the results with a built-in Bayesian statistics engine”
   • [claimed-docs] “collect events from your application into a managed data warehouse, and read the results with a built-in Bayesian statistics engine.”

   proof ↗⚑ flag

Metrics

1. product managerDefine experiment metrics from my own data — warehouse tables or ingested events — instead of a black-box metric store

   weight 2 · round to GrowthBook
   GrowthBook✓fullcommunity8/10

   GrowthBook explicitly lets teams define Fact Tables and metrics computed directly from their own warehouse (BigQuery, Snowflake, Databricks, etc.) rather than relying on a proprietary metric store, and independent community commentary confirms this warehouse-native design as a key differentiator from black-box tools like Optimizely. missing for 10: hands-on PM-level walkthrough of defining a metric from ingested event data (vs. warehouse tables) and any independent report specifically validating metric-definition UX.

   • [claimed-docs] “Define Fact Tables and metrics so GrowthBook can compute experiment results from your warehouse.”
   • [github] “Warehouse Native. Query 11 data sources including BigQuery, Snowflake, and Databricks.”
   • [github] “❄️ Warehouse Native. Query 11 data sources including BigQuery, Snowflake, and Databricks.”
   • [community] “Re: Eppo - they're similar in setup but Growthbook is open source with an optional cloud SaaS offering. Optimizely is a walled garden, you n…”
   • [community] “a guide in your docs on how to use this with Matomo would go far. Growthbook provides Feature Flag and A/B Testing, but you have to bring yo…”

   proof ↗⚑ flag

   Flagsmith—none0/10

   Flagsmith's experimentation feature explicitly funnels events into its own 'managed data warehouse' and evaluates them via a built-in Bayesian engine, the opposite of letting a PM define metrics from their own warehouse tables or ingested events — there is no evidence of BYO-warehouse or custom event-source metric definition.

   • [claimed-docs] “serve variations of a feature with a multivariate flag, collect events from your application into a managed data warehouse, and read the res…”
   • [claimed-docs] “collect events from your application into a managed data warehouse, and read the results with a built-in Bayesian statistics engine.”
   • [claimed-docs] “Flagsmith Experimentation lets you run A/B tests end to end on the platform: serve variations of a feature with a multivariate flag, collect…”
   • [claimed-docs] “read the results with a built-in Bayesian statistics engine”

   proof ↗⚑ flag

Environments

1. developerManage separate environments (dev/staging/prod) with independent flag states and scoped SDK keys

   weight 2 · round drawn
   GrowthBook✓fullprobed8/10

   Docs explicitly describe dedicated environments (dev/staging/production) with independently toggled flag states [growthbook-docs-3], and the SDK Connection Endpoint provides scoped, readonly keys per environment for SDKs [growthbook-docs-20], with runtime probes confirming a real create-flag/scoped-key/evaluate loop [growthbook-probe-rt-2]. Custom environments are also called out as a paid-tier feature, confirming the capability exists in-product [growthbook-docs-16, growthbook-docs-37]. Missing for 10: independent/community verification specifically of environment-scoped SDK key isolation (evidence is docs/probe only, not third-party confirmation) and more detail on default-environment limits on lower tiers.

   • [claimed-docs] “Use environments such as dev, staging, and production to enable or disable feature flags independently.”
   • [claimed-docs] “The SDK Connection Endpoint provides readonly access to a subset of your feature flag data, just enough for the GrowthBook SDKs to assign va…”
   • [claimed-docs] “Pro, plus: Custom environments Ramp schedules Approval workflows Advanced access control SSO & SCIM provisioning Exportable audit logs”
   • [claimed-docs] “Custom environments * Ramp schedules * Approval workflows * Advanced access control * SSO & SCIM provisioning * Exportable audit l…”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official growthbook/growthbook docker image boot…”

   proof ↗⚑ flag

   Flagsmith✓fullprobed8/10

   Flagsmith's core model is organized around projects with multiple environments (dev/staging/prod), each with its own independent flag states and its own scoped Environment/API key that's safe to expose client-side, confirmed by docs and a live probe showing environment-scoped key evaluation working end to end. RBAC further lets teams restrict who can modify production environments specifically. missing for 10: no explicit doc snippet walking through creating multiple named environments side-by-side or promoting flag state between them, and no independent/community corroboration beyond vendor docs and the probe.

   • [claimed-docs] “Authentication: Uses a public, non-secret Environment Key. Security: Open by design. The Environment Key can be exposed in client-side code.”
   • [claimed-docs] “Purpose: Creating, updating, and deleting projects, environments, flags, segments, and users.”
   • [claimed-docs] “Only allow certain users to modify your production environments.”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official flagsmith/flagsmith unified docker imag…”
   • [claimed-docs] “Manage feature flags and remote config across web, mobile, and server-side apps.”

   proof ↗⚑ flag

Flags

1. developerCreate a feature flag and toggle it live in production within minutes of signing up

   weight 3 · round drawn
   GrowthBook✓fullprobed9/10

   GrowthBook's quickstart docs show flag creation in a few lines of code with SDK hooks like useFeatureIsOn, and a runtime probe confirms a real keyless create-and-evaluate loop (create org, create flag, mint SDK connection, get correct value back) with no signup friction. Independent community testimony corroborates fast setup, with one user reporting a working flag in about 15 minutes and under 10 lines of code, and environments docs confirm dev/staging/prod toggling. missing for 10: a first-party or independent account of toggling a flag live specifically in a production environment end-to-end.

   • [claimed-docs] “Get GrowthBook feature flags and experiments running in your app in a few lines of code, then pick a language-specific SDK.”
   • [claimed-docs] “const showNewFeature = useFeatureIsOn("new-feature");”
   • [claimed-docs] “Use environments such as dev, staging, and production to enable or disable feature flags independently.”
   • [community] “Been using GrowthBook for a few weeks now... I was shocked at how much setup time it took to get a simple feature flag into our app with oth…”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official growthbook/growthbook docker image boot…”

   proof ↗⚑ flag

   Flagsmith✓fullprobed9/10

   Docs and a hands-on runtime probe confirm the core flow: create a feature via the dashboard/API, name it, toggle it on/off per environment without deploying code, and evaluate it live via SDK/API within minutes — the probe even demonstrates a full keyless create-and-evaluate loop from scratch. Docker quickstart and clear UI steps (Create Feature, toggle) further support fast time-to-value. missing for 10: no independent third-party timing benchmark or new-user onboarding video confirming the 'within minutes' claim beyond docs/probe.

   • [claimed-docs] “Go to the **Features** section in your dashboard. Click **Create Feature**.”
   • [claimed-docs] “Enter a descriptive name for your flag (e.g., new_ui_enabled). Fill in the available fields according to the specifications of your feature.”
   • [github] “Make changes remotely: Easily toggle individual features on and off, and make changes without deploying new code”
   • [github] “Easily toggle individual features on and off, and make changes without deploying new code”
   • [claimed-docs] “You can use Docker to set up an entire Flagsmith Feature Flag environment locally”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official flagsmith/flagsmith unified docker imag…”

   proof ↗⚑ flag

2. developerServe multivariate flags and dynamic configuration values (strings, numbers, JSON), not just booleans

   weight 2 · round to Flagsmith
   GrowthBook✓fullprobed7/10

   GrowthBook's SDK docs explicitly show non-boolean flag values via useFeatureValue (string example 'button-color') alongside useFeatureIsOn for booleans, and its targeting/rules docs describe rules that 'control which values a user sees' beyond simple on/off, plus a real runtime probe confirms a flag payload with a typed defaultValue served via the API. Missing for 10: explicit documented example of numeric and JSON-typed flag values (only string is shown in the quickstart snippet), and no independent hands-on confirmation of JSON config usage.

   • [claimed-docs] “const showNewFeature = useFeatureIsOn("new-feature"); const buttonColor = useFeatureValue("button-color", "blue");”
   • [claimed-docs] “How rules override a feature's default value — targeting users, rolling out gradually, experimenting — and how to scope, ramp, and schedule …”
   • [claimed-docs] “Add targeting conditions to a rule to control which values a user sees.”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official growthbook/growthbook docker image boot…”
   • [github] “24 SDKs including React, Python, Android, and iOS.”

   proof ↗⚑ flag

   Flagsmith✓fullprobed9/10

   Flagsmith docs explicitly describe multivariate flags with percentage-weighted variants and remote config values, and core management docs show configurable flag fields (not just boolean toggles) plus experimentation using multivariate flags. This is corroborated by a self-hosted probe showing a real flag create/evaluate roundtrip. missing for 10: explicit doc confirmation of string/number/JSON config value types beyond variant weighting, and independent hands-on confirmation of non-boolean config value retrieval via SDK.

   • [claimed-docs] “Multivariate flags let you define multiple variants with percentage weightings (A/B/n).”
   • [claimed-docs] “serve variations of a feature with a multivariate flag, collect events from your application into a managed data warehouse, and read the res…”
   • [claimed-docs] “Enter a descriptive name for your flag (e.g., new_ui_enabled). Fill in the available fields according to the specifications of your feature.”
   • [claimed-docs] “Go to the **Features** section in your dashboard. Click **Create Feature**.”
   • [claimed-docs] “Flagsmith Experimentation lets you run A/B tests end to end on the platform: serve variations of a feature with a multivariate flag, collect…”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official flagsmith/flagsmith unified docker imag…”

   proof ↗⚑ flag

Lifecycle

1. platform engineerFind stale flags and code references so temporary flags actually get removed from the codebase

   weight 2 · round to GrowthBook
   GrowthBook✓fullclaimed8/10

   GrowthBook's docs explicitly describe automatic stale flag detection ('GrowthBook can automatically detect these flags so you can clean them up') as a dedicated feature, directly addressing the story of finding stale flags for removal. This is corroborated by the product's broader flag-management infrastructure (environments, rules, revisions) but lacks detail on code-reference scanning across repos or independent/hands-on verification of the stale-detection feature itself. Missing for 10: evidence of code-reference/usage scanning across the codebase, and independent/community confirmation that stale detection works reliably in practice.

   • [claimed-docs] “GrowthBook can automatically detect these flags so you can clean them up.”
   • [claimed-docs] “How rules override a feature's default value — targeting users, rolling out gradually, experimenting — and how to scope, ramp, and schedule …”
   • [claimed-docs] “Use environments such as dev, staging, and production to enable or disable feature flags independently.”

   proof ↗⚑ flag

   Flagsmith—none0/10

   The evidence pack covers targeting, rollouts, experimentation, audit logs, RBAC, and change requests, but contains no mention of stale-flag detection, code reference scanning, or flag lifecycle/cleanup tooling that would help identify and remove unused flags from a codebase. This is a fair capability to expect from a mature flag-management platform (competitors ship code-reference scanners), so absence of evidence means 'none' rather than 'na'.

   ⚑ flag

Targeting

1. developerTarget or exclude specific individual users for a flag (allowlists, beta testers, internal accounts)

   weight 1 · round to Flagsmith
   GrowthBook✓fullclaimed7/10

   GrowthBook's rule/targeting system lets you write conditions against arbitrary user attributes (e.g., user ID, email) to include or exclude specific users, which covers allowlisting beta testers or internal accounts (growthbook-docs-2, growthbook-docs-22, growthbook-docs-18, growthbook-docs-28). This is a first-class part of its rules engine (scope, ramp, schedule). missing for 10: an explicit doc example/tutorial titled 'allowlist' or 'beta testers' walkthrough, and independent hands-on confirmation of exclusion-list behavior.

   • [claimed-docs] “Add targeting conditions to a rule to control which values a user sees. Conditions are evaluated against attributes you pass into the SDK”
   • [claimed-docs] “How rules override a feature's default value — targeting users, rolling out gradually, experimenting — and how to scope, ramp, and schedule …”
   • [claimed-docs] “The actual values of targeting attributes (e.g., user IDs, emails) are never sent to GrowthBook. They are only stored in memory locally with…”
   • [claimed-docs] “Add targeting conditions to a rule to control which values a user sees.”

   proof ↗⚑ flag

   Flagsmith✓fullclaimed9/10

   Flagsmith explicitly supports overriding feature flags for individual users via 'identities', documented for internal testing, QA, and personalized experiences, plus segment-based targeting for group-based allowlists (e.g., beta testers). This is a well-documented core feature directly matching the story. Missing for 10: independent third-party hands-on confirmation of identity-level overrides specifically (probes verify flag creation/eval but not the identity-override workflow itself).

   • [claimed-docs] “Override feature flags for individual users ("identities"). Enables internal testing, QA, customer support, or personalised experiences at t…”
   • [claimed-docs] “Override feature flags for individual users ("identities").”
   • [claimed-docs] “Define **segments**—groups of users matching rules based on traits (e.g., location, plan, app version, usage).”
   • [claimed-docs] “Define segments—groups of users matching rules based on traits (e.g., location, plan, app version, usage). Override feature flags for all us…”

   proof ↗⚑ flag

2. developerTarget flags with attribute-based rules and reusable segments so the right users see the right variation

   weight 3 · round to Flagsmith
   GrowthBook~partialclaimed6/10

   GrowthBook clearly supports attribute-based targeting rules (conditions evaluated against SDK-passed attributes, rule scoping/ramping/scheduling) as shown in growthbook-docs-2, -22, -28, and -18. However, the evidence pack never mentions a distinct 'segments' feature (reusable named audience definitions) that developers could apply across multiple flags/rules, so the reusable-segment half of the story is unevidenced. missing for 10: explicit segment/audience-reuse documentation, evidence of applying the same segment across multiple flags, independent confirmation of segment behavior.

   • [claimed-docs] “Add targeting conditions to a rule to control which values a user sees. Conditions are evaluated against attributes you pass into the SDK”
   • [claimed-docs] “How rules override a feature's default value — targeting users, rolling out gradually, experimenting — and how to scope, ramp, and schedule …”
   • [claimed-docs] “Add targeting conditions to a rule to control which values a user sees.”
   • [claimed-docs] “The actual values of targeting attributes (e.g., user IDs, emails) are never sent to GrowthBook. They are only stored in memory locally with…”

   proof ↗⚑ flag

   Flagsmith✓fullprobed9/10

   Docs clearly describe trait-based targeting (geography, subscription, device, version), reusable segments defined by trait rules, per-identity overrides, percentage-split rollouts within segments, and multivariate flags — all core to attribute-based targeting and segment-driven variation delivery, corroborated by a runtime probe confirming a real flag create/evaluate loop. Missing for 10: independent (non-vendor) third-party validation of segment/attribute targeting behavior at scale beyond the single self-host probe.

   • [claimed-docs] “Target features to users with specific traits (such as geography, subscription level, device type, or application version).”
   • [claimed-docs] “Define **segments**—groups of users matching rules based on traits (e.g., location, plan, app version, usage).”
   • [claimed-docs] “Define segments—groups of users matching rules based on traits (e.g., location, plan, app version, usage). Override feature flags for all us…”
   • [claimed-docs] “Override feature flags for individual users ("identities").”
   • [claimed-docs] “Add a rule defined with the % Split condition. Specify a value between 1 and 100 to define what percentage of your user base is included wit…”
   • [claimed-docs] “Multivariate flags let you define multiple variants with percentage weightings (A/B/n).”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official flagsmith/flagsmith unified docker imag…”

   proof ↗⚑ flag

Access

1. platform engineerRestrict who can change which flags with roles, permissions, and scoped API tokens

   weight 2 · round to Flagsmith
   GrowthBook~partialclaimed5/10

   Pricing page confirms GrowthBook has role/permission features ("Advanced permissioning", "Advanced access control", SSO & SCIM) gated behind paid tiers, implying RBAC exists, but there is no docs page detailing how roles map to flag-level permissions or how API tokens can be scoped to restrict flag changes; the API docs even note the SDK connection endpoint is public/unauthenticated, which is a different, more limited access model. Missing for 10: a dedicated permissions/roles doc, explicit description of scoped/restricted API tokens for flag mutation, and independent confirmation of enforcement.

   • [claimed-docs] “Starter, plus: Visual editor Multi-arm bandits Safe rollouts Customizable dashboards Advanced permissioning Power calculator Premium support”
   • [claimed-docs] “Pro, plus: Custom environments Ramp schedules Approval workflows Advanced access control SSO & SCIM provisioning Exportable audit logs”
   • [claimed-docs] “Visual editor * Multi-arm bandits * Safe rollouts * Customizable dashboards * Advanced permissioning * Power calculator”
   • [claimed-docs] “Custom environments * Ramp schedules * Approval workflows * Advanced access control * SSO & SCIM provisioning * Exportable audit l…”
   • [claimed-docs] “GrowthBook offers a full REST API for interacting with the application.”
   • [claimed-docs] “They are meant to be public and do not require authentication to view.”

   proof ↗⚑ flag

   Flagsmith✓fullclaimed9/10

   Flagsmith documents fine-grained RBAC to restrict who can modify resources (e.g., 'only allow certain users to modify your production environments'), plus a four-eyes change-request workflow for approvals, and clearly distinguishes scoped API tokens (public non-secret client-side environment keys vs. privileged Admin API keys for CRUD operations). SSO/SAML and 2FA further support access governance. Missing for 10: no explicit documentation of scoped/granular API token permissions (e.g., per-project or read-only server tokens) beyond the client/server key distinction, and no independent/community validation of RBAC granularity in practice.

   • [claimed-docs] “Role-based access control (RBAC) provides fine-grained access management of Flagsmith resources.”
   • [claimed-docs] “Only allow certain users to modify your production environments.”
   • [claimed-docs] “Feature Change Requests help creating a four-eyes workflow (create, approve, publish) to updating feature flags, similarly to Pull Requests …”
   • [claimed-docs] “Purpose: Creating, updating, and deleting projects, environments, flags, segments, and users.”
   • [claimed-docs] “Authentication: Uses a public, non-secret Environment Key. Security: Open by design. The Environment Key can be exposed in client-side code.”
   • [claimed-docs] “SAML/SSO Authentication”
   • [claimed-docs] “Two-Factor Authentication (2FA)”

   proof ↗⚑ flag

Agent ops

1. ai agentCreate and toggle flags through documented APIs, CLIs, or MCP — and the platform can force my changes through approval workflows instead of letting me write to production unreviewed

   weight 3 · round to Flagsmith
   GrowthBook~partialprobed6/10

   GrowthBook documents an official MCP server that makes authenticated REST calls (growthbook-docs-13/21, growthbook-probe-3/rt-1) and a full REST API (growthbook-docs-12), and separately documents 'Draft feature revisions, request review... require approval before publishing' as a Pro+ approval-workflow feature (growthbook-docs-5, growthbook-docs-16/37). However, the runtime probe (growthbook-probe-rt-2) shows a flag created and immediately live via the API with no review step, and approval workflows appear gated behind a paid plan rather than proven to force-block an AI agent's writes by default. Missing for 10: a documented/verified CLI channel, and hands-on evidence that MCP/API writes are actually intercepted and blocked pending human approval rather than merely available as an opt-in paid feature.

   • [claimed-docs] “Connect AI tools like Cursor, VS Code, and Claude to GrowthBook using the Model Context Protocol (MCP).”
   • [claimed-docs] “The official GrowthBook MCP server is a thin bridge: it loads GrowthBook Agent Skills (workflows and guardrails) and makes authenticated cal…”
   • [probe] “official MCP server documented at https://docs.growthbook.io/integrations/mcp”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official npm MCP server (`npx -y @growthbook/mcp`) completed a FULL keyless stdio initialize handsh…”
   • [claimed-docs] “Draft feature revisions, request review, resolve merge conflicts, and require approval before publishing.”
   • [claimed-docs] “Pro, plus: Custom environments Ramp schedules Approval workflows Advanced access control SSO & SCIM provisioning Exportable audit logs”
   • [claimed-docs] “Custom environments * Ramp schedules * Approval workflows * Advanced access control * SSO & SCIM provisioning * Exportable audit l…”
   • [claimed-docs] “GrowthBook offers a full REST API for interacting with the application.”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official growthbook/growthbook docker image boot…”

   proof ↗⚑ flag

   Flagsmith✓fullprobed7/10

   Flagsmith documents an official MCP server, CLI, and Admin API that let an AI agent create/read/toggle flags (flagsmith-docs-17/29, probe-3, probe-rt-1, docs-18/40), and separately documents a four-eyes Change Request workflow plus RBAC that can restrict who may modify production flags, similar to PR review (flagsmith-docs-10, flagsmith-docs-26). This gives both halves of the story: programmatic write access and an approval-gate mechanism. missing for 10: explicit documentation/hands-on proof that API/MCP/CLI-originated writes are automatically routed through Change Requests (rather than change requests being a dashboard-only workflow), and no independent verification that RBAC blocks an agent's direct API write to production.

   • [claimed-docs] “The Flagsmith MCP Server gives AI assistants and agents programmatic access to the Flagsmith Admin API through the Model Context Protocol”
   • [probe] “official MCP server documented at https://docs.flagsmith.com/integrating-with-flagsmith/mcp-server”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the documented hosted MCP server https://mcp.flagsmith.com returned…”
   • [claimed-docs] “The Flagsmith CLI lets you manage flags, segments, features, projects and environments from your terminal, and evaluate flags the way an SDK…”
   • [claimed-docs] “This is the API you use to programmatically manage your Flagsmith projects. Anything you can do in the Flagsmith dashboard, you can also do …”
   • [claimed-docs] “Feature Change Requests help creating a four-eyes workflow (create, approve, publish) to updating feature flags, similarly to Pull Requests …”
   • [claimed-docs] “Only allow certain users to modify your production environments.”
   • [claimed-docs] “Every action taken within the Flagsmith administration application is tracked and logged. This allows you to easily retrace the events and v…”

   proof ↗⚑ flag

Approvals

1. platform engineerRequire approvals or change requests before production flag changes go live

   weight 3 · round to Flagsmith
   GrowthBook✓fullclaimed8/10

   GrowthBook's docs explicitly describe drafting feature revisions, requesting review, resolving merge conflicts, and requiring approval before publishing changes (growthbook-docs-5), and this is packaged as a named 'Approval workflows' feature on paid tiers (growthbook-docs-16, growthbook-docs-37). This directly matches the story of gating production flag changes behind approvals. Missing for 10: independent/hands-on confirmation of the approval workflow in practice (only vendor docs cited) and detail on how approvals interact with environment-specific promotion.

   • [claimed-docs] “Draft feature revisions, request review, resolve merge conflicts, and require approval before publishing.”
   • [claimed-docs] “Pro, plus: Custom environments Ramp schedules Approval workflows Advanced access control SSO & SCIM provisioning Exportable audit logs”
   • [claimed-docs] “Custom environments * Ramp schedules * Approval workflows * Advanced access control * SSO & SCIM provisioning * Exportable audit l…”
   • [claimed-docs] “Use environments such as dev, staging, and production to enable or disable feature flags independently.”

   proof ↗⚑ flag

   Flagsmith✓fullclaimed9/10

   Flagsmith documents a native four-eyes Feature Change Request workflow (create, approve, publish) explicitly analogous to GitHub PRs, plus Release Pipelines that can auto-generate change requests when promoting flags to production, and RBAC to restrict who can modify production environments. This directly satisfies the governance requirement for approvals before production flag changes go live. Missing for 10: independent/hands-on verification of the approval gating in practice (only first-party docs cited).

   • [claimed-docs] “Feature Change Requests help creating a four-eyes workflow (create, approve, publish) to updating feature flags, similarly to Pull Requests …”
   • [claimed-docs] “Release Pipelines can automatically generate change requests when promoting flags to production, eliminating manual effort and streamlining …”
   • [claimed-docs] “Release Pipelines act as your intelligent release assistant, automating your entire feature flag deployment process.”
   • [claimed-docs] “Only allow certain users to modify your production environments.”
   • [claimed-docs] “Role-based access control (RBAC) provides fine-grained access management of Flagsmith resources.”

   proof ↗⚑ flag

Audit

1. platform engineerEvery flag change is recorded in an audit log — who changed what, when, and to which value

   weight 3 · round to Flagsmith
   GrowthBook~partialclaimed5/10

   Pricing docs confirm GrowthBook ships an audit log feature, with 'Exportable audit logs' listed as a Pro-tier capability, implying built-in tracking of changes for governance purposes. However, there is no first-party documentation detailing exactly what is captured (who/when/old-new value) or how the audit trail is presented/queried, and the capability is gated behind a paid plan rather than universally available. Missing for 10: dedicated audit-log documentation describing captured fields, retention, and access/export mechanics, plus confirmation it's available outside paid tiers or independent corroboration of its behavior.

   • [claimed-docs] “Pro, plus: Custom environments Ramp schedules Approval workflows Advanced access control SSO & SCIM provisioning Exportable audit logs”
   • [claimed-docs] “Custom environments * Ramp schedules * Approval workflows * Advanced access control * SSO & SCIM provisioning * Exportable audit l…”
   • [claimed-docs] “SSO & SCIM provisioning * Exportable audit logs”

   proof ↗⚑ flag

   Flagsmith✓fullclaimed9/10

   Flagsmith documents that every action in the admin application (flag, identity, segment changes) is tracked and logged, directly matching the who/what/when/value audit requirement, and offers webhook streaming to external systems for governance workflows. missing for 10: independent/hands-on confirmation of the audit log's exact field-level detail (e.g., explicit before/after value display) beyond vendor docs.

   • [claimed-docs] “Every action taken within the Flagsmith administration application is tracked and logged. This allows you to easily retrace the events and v…”
   • [claimed-docs] “Every action taken within the Flagsmith administration application is tracked and logged.”
   • [claimed-docs] “You can stream your Audit Logs into your own infrastructure using Audit Log Webhooks.”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Flagsmith
   GrowthBook✓fullprobed7/10

   GrowthBook documents a 'full REST API for interacting with the application' (growthbook-docs-12) and a runtime probe confirms it: a self-hosted instance created an org, user, and feature flag purely via API calls and served it through the SDK payload endpoint (growthbook-probe-rt-2), and the official MCP server exposes authenticated read/write meta-tools covering the whole REST API (growthbook-docs-21, growthbook-probe-rt-1). Missing for 10: an explicit enumerated API-vs-UI parity list (e.g. confirmation that experiment approval workflows, visual editor changes, and dashboard configuration are all API-exposed, not just feature flags) and independent third-party corroboration of API completeness.

   • [claimed-docs] “GrowthBook offers a full REST API for interacting with the application.”
   • [claimed-docs] “The official GrowthBook MCP server is a thin bridge: it loads GrowthBook Agent Skills (workflows and guardrails) and makes authenticated cal…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official npm MCP server (`npx -y @growthbook/mcp`) completed a FULL keyless stdio initialize handsh…”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official growthbook/growthbook docker image boot…”
   • [claimed-docs] “The SDK Connection Endpoint provides readonly access to a subset of your feature flag data, just enough for the GrowthBook SDKs to assign va…”

   proof ↗⚑ flag

   Flagsmith✓fullprobed9/10

   Flagsmith explicitly documents that 'Anything you can do in the Flagsmith dashboard, you can also do via the Admin API' (flagsmith-docs-30/40), and this is corroborated by a real probe showing a full create-project/environment/flag/evaluate roundtrip via the API (flagsmith-probe-rt-2). It further exposes this API to AI agents via an official MCP server and CLI (flagsmith-docs-17/18, flagsmith-probe-3/4, flagsmith-probe-rt-1), reinforcing API-UI parity for AI-native use. Missing for 10: a published OpenAPI/swagger spec was probed and returned 404 (flagsmith-probe-2), so machine-readable API completeness is not independently verifiable beyond the vendor's own claim.

   • [claimed-docs] “Anything you can do in the Flagsmith dashboard, you can also do via the Admin API.”
   • [claimed-docs] “This is the API you use to programmatically manage your Flagsmith projects. Anything you can do in the Flagsmith dashboard, you can also do …”
   • [claimed-docs] “The Flagsmith MCP Server gives AI assistants and agents programmatic access to the Flagsmith Admin API through the Model Context Protocol”
   • [claimed-docs] “The Flagsmith CLI lets you manage flags, segments, features, projects and environments from your terminal, and evaluate flags the way an SDK…”
   • [probe] “official MCP server documented at https://docs.flagsmith.com/integrating-with-flagsmith/mcp-server”
   • [probe] “official CLI documented at https://docs.flagsmith.com/integrating-with-flagsmith/CLI”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the documented hosted MCP server https://mcp.flagsmith.com returned…”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official flagsmith/flagsmith unified docker imag…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.flagsmith.com/openapi.json, https://docs.flagsmith.com/swagger.json, https://docs.flags…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round to Flagsmith
   GrowthBook~partialcommunity5/10

   GrowthBook is open-source and self-hostable with its data stored in a standard MongoDB instance, and it exposes a full REST API for programmatic access to feature flags, experiments, and metrics, which together give a technically-inclined user a path to extract all their data without vendor lock-in. However, there is no explicit documented 'export all data' or bulk-export feature/wizard, and 'exportable audit logs' is gated behind an Enterprise tier rather than being a general open-export capability. Missing for 10: an explicit documented full-data export/backup tool or format, confirmation that experiment results/metrics history export in an open format, and evidence this works outside self-hosting for cloud customers.

   • [claimed-docs] “GrowthBook offers a full REST API for interacting with the application.”
   • [claimed-docs] “Run GrowthBook on your own infrastructure with Docker Compose, including MongoDB, the API, and the front-end.”
   • [claimed-docs] “You can use docker compose to get started quickly... run docker compose up -d to start everything and view the app at http://localhost:3000”
   • [claimed-docs] “Custom environments * Ramp schedules * Approval workflows * Advanced access control * SSO & SCIM provisioning * Exportable audit l…”
   • [community] “Re: Eppo - they're similar in setup but Growthbook is open source with an optional cloud SaaS offering. Optimizely is a walled garden, you n…”

   proof ↗⚑ flag

   Flagsmith~partialprobed6/10

   Flagsmith's Admin API exposes full CRUD access to all projects, environments, flags, segments, and users ("anything you can do in the dashboard, you can also do via the Admin API"), and self-hosting via Docker gives full ownership of the underlying Postgres data store, together enabling a user to extract and own all their data outside the SaaS. However there is no documented one-click 'export all data' feature, no bulk export-to-JSON/CSV tool, and probes show no llms.txt or public OpenAPI spec is served, so a turnkey open-format export path is not explicitly evidenced. missing for 10: explicit bulk data-export feature/format, documented data-portability guarantee, and a working OpenAPI/machine-readable spec (docs.flagsmith.com/openapi.json returned 404).

   • [claimed-docs] “Anything you can do in the Flagsmith dashboard, you can also do via the Admin API.”
   • [claimed-docs] “This is the API you use to programmatically manage your Flagsmith projects. Anything you can do in the Flagsmith dashboard, you can also do …”
   • [claimed-docs] “Self-hosting Flagsmith allows you to run the complete Flagsmith platform within your own infrastructure, giving you full control over your d…”
   • [claimed-docs] “You can use Docker to set up an entire Flagsmith Feature Flag environment locally”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official flagsmith/flagsmith unified docker imag…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.flagsmith.com/openapi.json, https://docs.flagsmith.com/swagger.json, https://docs.flags…”

   proof ↗⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round to Flagsmith
   GrowthBook~partialcommunity5/10

   Community evidence explicitly states GrowthBook is open source with an optional cloud SaaS offering, and a public GitHub repository (github.com/growthbook/growthbook) is cited, supporting that source is readable. However, no evidence pack item specifies the actual license type (e.g., MIT/Apache) or licensing terms, so the 'open license' claim isn't directly documented. Missing for 10: explicit license identification/text, first-party docs confirming license terms, independent confirmation of license permissiveness.

   • [community] “Re: Eppo - they're similar in setup but Growthbook is open source with an optional cloud SaaS offering. Optimizely is a walled garden, you n…”
   • [github] “24 SDKs including React, Python, Android, and iOS.”
   • [claimed-docs] “Run GrowthBook on your own infrastructure with Docker Compose, including MongoDB, the API, and the front-end.”

   proof ↗⚑ flag

   Flagsmith✓fullclaimed7/10

   The Flagsmith source code is publicly hosted on GitHub (repeatedly cited) and the project provides self-hosting docs, Docker images, and full platform control, all consistent with an openly licensed codebase. However, the evidence pack never explicitly names or links the license file/type, so the claim of a specific open license is inferred rather than directly documented. Missing for 10: explicit license identifier/file citation (e.g., BSD-3-Clause LICENSE), independent confirmation of license terms.

   • [github] “Make changes remotely: Easily toggle individual features on and off, and make changes without deploying new code”
   • [github] “Just wrap a section of code with a flag, and then use Flagsmith to toggle that feature on or off for different environments, users or user s…”
   • [github] “Easily toggle individual features on and off, and make changes without deploying new code”
   • [claimed-docs] “Self-hosting Flagsmith allows you to run the complete Flagsmith platform within your own infrastructure, giving you full control over your d…”
   • [claimed-docs] “You can use Docker to set up an entire Flagsmith Feature Flag environment locally”

   proof ↗⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round drawn
   GrowthBook✓fullprobed10/10

   GrowthBook is documented and independently confirmed (via runtime probe) to self-host fully via Docker Compose including MongoDB, API, and front-end, with a real keyless create-and-evaluate flag loop working end-to-end; community reviews also corroborate long-term self-hosted usage.

   • [claimed-docs] “Run GrowthBook on your own infrastructure with Docker Compose, including MongoDB, the API, and the front-end.”
   • [claimed-docs] “You can use docker compose to get started quickly... run docker compose up -d to start everything and view the app at http://localhost:3000”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official growthbook/growthbook docker image boot…”
   • [community] “Growthbook is awesome! We have used the self-hosted version a few months in my company now, and are very happy about it. Talented people beh…”

   proof ↗⚑ flag

   Flagsmith✓fullprobed10/10

   Flagsmith documents comprehensive self-hosting via Docker (full platform, own infrastructure, full control over data/security), and a hands-on probe confirms a complete keyless self-host roundtrip — running the official Docker image, creating a project/environment/flag, and successfully evaluating it via the API with no account or license required.

   • [claimed-docs] “Self-hosting Flagsmith allows you to run the complete Flagsmith platform within your own infrastructure, giving you full control over your d…”
   • [claimed-docs] “You can use Docker to set up an entire Flagsmith Feature Flag environment locally”
   • [claimed-docs] “You can use Docker to set up an entire [Flagsmith Feature Flag](https://www.flagsmith.com) environment locally”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official flagsmith/flagsmith unified docker imag…”

   proof ↗⚑ flag

Pricing

1. product managerSee published pricing and understand what drives cost (seats, MAUs, events, requests) before committing

   weight 2 · round to GrowthBook
   GrowthBook~partialclaimed6/10

   GrowthBook publishes a tiered pricing page (Starter/Pro/Enterprise-like tiers) listing feature gates (visual editor, approval workflows, SSO/SCIM, etc.) and calls out unlimited flags/experiments/traffic and per-user caps like 'Up to 3 users', but none of the evidence shows usage-based cost drivers like MAUs, events, or API requests being priced or metered — the plans appear seat/feature-gated rather than usage-metered. missing for 10: explicit cost drivers tied to MAUs/events/requests, a pricing calculator or usage-based cost breakdown, and independent confirmation that published prices match real invoices.

   • [claimed-docs] “Starter, plus: Visual editor Multi-arm bandits Safe rollouts Customizable dashboards Advanced permissioning Power calculator Premium support”
   • [claimed-docs] “Pro, plus: Custom environments Ramp schedules Approval workflows Advanced access control SSO & SCIM provisioning Exportable audit logs”
   • [claimed-docs] “Unlimited feature flags * Unlimited experiments * Unlimited traffic”
   • [claimed-docs] “Starter, plus: * Visual editor * Multi-arm bandits * Safe rollouts * Customizable dashboards”
   • [claimed-docs] “Visual editor * Multi-arm bandits * Safe rollouts * Customizable dashboards * Advanced permissioning * Power calculator”
   • [claimed-docs] “Custom environments * Ramp schedules * Approval workflows * Advanced access control * SSO & SCIM provisioning * Exportable audit l…”
   • [claimed-docs] “Up to 3 users * 1 project * Unlimited feature flags * Unlimited experiments * Unlimited traffic”
   • [claimed-docs] “SSO & SCIM provisioning * Exportable audit logs”

   proof ↗⚑ flag

   Flagsmith—none0/10

   The evidence pack only references the pricing page URL for isolated feature call-outs (SAML/SSO, 2FA) but contains no evidence about actual pricing tiers, seat costs, MAU/event/request-based pricing, or how usage drives cost. missing for 10: published pricing tiers, cost drivers (seats/MAUs/events/requests), any breakdown of what determines plan cost.

   • [claimed-docs] “SAML/SSO Authentication”
   • [claimed-docs] “Two-Factor Authentication (2FA)”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round drawn
   GrowthBook~partialprobed6/10

   GrowthBook can be fully self-hosted via Docker Compose (own MongoDB, API, and front-end), which lets an AI-native user control exactly where all data lives, and a probe confirms a working self-hosted flag create/evaluate loop with no third-party data storage. However, there is no documented cloud-hosted region/residency selection (e.g., EU vs US SaaS options) for users who don't want to self-host. Missing for 10: explicit cloud multi-region hosting options, formal data-residency/compliance certifications, and documentation letting SaaS customers pick a storage region without self-hosting.

   • [claimed-docs] “Run GrowthBook on your own infrastructure with Docker Compose, including MongoDB, the API, and the front-end.”
   • [claimed-docs] “You can use docker compose to get started quickly... run docker compose up -d to start everything and view the app at http://localhost:3000”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official growthbook/growthbook docker image boot…”
   • [claimed-docs] “The actual values of targeting attributes (e.g., user IDs, emails) are never sent to GrowthBook. They are only stored in memory locally with…”

   proof ↗⚑ flag

   Flagsmith~partialprobed6/10

   Flagsmith supports data residency indirectly via self-hosting (full control over data location, deployable in any infrastructure/region) and an Edge Proxy that can be run within a customer's own infrastructure, which lets an AI-native user choose where flag data lives. However, there's no documented SaaS-side region picker (e.g., dedicated EU/US managed cloud regions) — residency control is achieved only by fully self-hosting rather than a first-class 'choose region' setting. Missing for 10: explicit managed-cloud region selection options, documentation of data residency guarantees for the hosted SaaS offering.

   • [claimed-docs] “Self-hosting Flagsmith allows you to run the complete Flagsmith platform within your own infrastructure, giving you full control over your d…”
   • [claimed-docs] “You can use Docker to set up an entire Flagsmith Feature Flag environment locally”
   • [claimed-docs] “The Flagsmith Edge Proxy is a service that you host yourself, that allows you to run an instance of the Flagsmith Engine close to your serve…”
   • [claimed-docs] “You can use Docker to set up an entire [Flagsmith Feature Flag](https://www.flagsmith.com) environment locally”
   • [claimed-docs] “Run the Edge Proxy within in your own infrastructure and connect to it from your server-side SDKs”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official flagsmith/flagsmith unified docker imag…”

   proof ↗⚑ flag

2. ai-native userControl data retention and deletion

   weight 2 · round to Flagsmith
   GrowthBook~partialclaimed4/10

   GrowthBook's privacy-by-design targeting (attribute values never sent to GrowthBook, kept in-memory in the SDK) and the option to self-host on your own infrastructure/MongoDB give users indirect control over what data is retained and how it's deleted, since they own the database. However, there is no explicit documentation of a data retention policy, a deletion API/endpoint, or GDPR-style 'delete my data' tooling in the evidence pack. Missing for 10: explicit retention-period settings, a documented deletion/erasure API or workflow, and any compliance certification (e.g., GDPR/CCPA) statements.

   • [claimed-docs] “The actual values of targeting attributes (e.g., user IDs, emails) are never sent to GrowthBook. They are only stored in memory locally with…”
   • [claimed-docs] “Run GrowthBook on your own infrastructure with Docker Compose, including MongoDB, the API, and the front-end.”
   • [claimed-docs] “You can use docker compose to get started quickly... run docker compose up -d to start everything and view the app at http://localhost:3000”

   proof ↗⚑ flag

   Flagsmith~partialclaimed5/10

   Flagsmith's Admin API and dashboard explicitly support creating, updating, and deleting projects, environments, flags, segments, and users, with a 'protected' flag option to guard against accidental deletion, and self-hosting gives full control over where data lives. However there is no explicit documentation of a data-retention policy, automated data expiry, or a GDPR-style 'delete my data' workflow beyond generic CRUD deletion of resources. Missing for 10: documented retention windows/expiry settings, formal data-deletion/right-to-erasure workflow, and audit-log retention controls.

   • [claimed-docs] “Purpose: Creating, updating, and deleting projects, environments, flags, segments, and users.”
   • [claimed-docs] “add `protected` to prevent them from accidentally being deleted”
   • [claimed-docs] “Self-hosting Flagsmith allows you to run the complete Flagsmith platform within your own infrastructure, giving you full control over your d…”
   • [claimed-docs] “Every action taken within the Flagsmith administration application is tracked and logged. This allows you to easily retrace the events and v…”

   proof ↗⚑ flag

Rollouts

1. platform engineerGuard a rollout with metrics so a regression is detected and the release is rolled back automatically

   weight 2 · round to GrowthBook
   GrowthBook~partialclaimed4/10

   GrowthBook's pricing pages list a named 'Safe rollouts' feature (Pro+ tier) alongside guardrail-metric support in experiment templates, implying automated regression detection and rollback, but the evidence pack contains no docs explaining how the automatic rollback mechanism actually works (thresholds, trigger logic, rollback speed) nor any hands-on confirmation it fires reliably in production. Missing for 10: detailed docs/description of the Safe Rollouts mechanism and guardrail-triggered auto-rollback, plus independent or hands-on evidence it works as claimed.

   • [claimed-docs] “Starter, plus: Visual editor Multi-arm bandits Safe rollouts Customizable dashboards Advanced permissioning Power calculator Premium support”
   • [claimed-docs] “Starter, plus: * Visual editor * Multi-arm bandits * Safe rollouts * Customizable dashboards”
   • [claimed-docs] “Visual editor * Multi-arm bandits * Safe rollouts * Customizable dashboards * Advanced permissioning * Power calculator”
   • [claimed-docs] “Visual editor * Multi-arm bandits * Safe rollouts * Customizable dashboards”
   • [claimed-docs] “Selecting a Template in the creation modal pre-fills settings like the data source, goal/secondary/guardrail metrics, targeting, and traffic”

   proof ↗⚑ flag

   Flagsmith—none0/10

   Flagsmith documents percentage rollouts, scheduled flags, and Release Pipelines with 'triggers and actions,' plus an Experimentation module with Bayesian stats — but none of the evidence describes automatic metric-based regression detection that triggers an automatic rollback of a release. The pipeline features found are about staged promotion and change-request automation, not closed-loop guarded rollouts tied to live health/error metrics.

   • [claimed-docs] “Define stages that your flags progress through automatically, with triggers and actions that control rollouts to selected environments and a…”
   • [claimed-docs] “Release Pipelines can automatically generate change requests when promoting flags to production, eliminating manual effort and streamlining …”
   • [claimed-docs] “Release Pipelines act as your intelligent release assistant, automating your entire feature flag deployment process.”
   • [claimed-docs] “Flagsmith Experimentation lets you run A/B tests end to end on the platform: serve variations of a feature with a multivariate flag, collect…”
   • [claimed-docs] “Add a rule defined with the % Split condition. Specify a value between 1 and 100 to define what percentage of your user base is included wit…”

   proof ↗⚑ flag

2. developerRoll a flag out progressively by percentage with consistent bucketing, ramping from 1% to 100% without redeploying

   weight 3 · round to Flagsmith
   GrowthBook✓fullprobed8/10

   GrowthBook's rules docs explicitly cover 'rolling out gradually' and describe how rules can 'scope, ramp, and schedule' percentage rollouts, with experiment/flag assignment based on a hashing attribute for consistent bucketing, and SDKs evaluate the cached JSON payload client-side with no redeploy needed. Pricing tiers also list dedicated 'Ramp schedules' as a named capability, confirming this is a first-class shipped feature, and probe evidence confirms a live flag create/evaluate loop via the SDK payload endpoint. missing for 10: independent/hands-on confirmation of an actual 1%→100% ramp in practice, and detail on the specific hashing algorithm ensuring bucket stability across ramp changes

   • [claimed-docs] “How rules override a feature's default value — targeting users, rolling out gradually, experimenting — and how to scope, ramp, and schedule …”
   • [claimed-docs] “you can add an experiment rule to a feature that will randomly assign the users based on some hashing attribute into one of your experiment …”
   • [claimed-docs] “When the client loads your application, our SDK will check to see if the user should be part of any experiments, and if so, assign them and …”
   • [claimed-docs] “Pro, plus: Custom environments Ramp schedules Approval workflows Advanced access control SSO & SCIM provisioning Exportable audit logs”
   • [claimed-docs] “Custom environments * Ramp schedules * Approval workflows * Advanced access control * SSO & SCIM provisioning * Exportable audit l…”
   • [claimed-docs] “GrowthBook supports several ways to do this: Feature Flags (most common)... URL Redirects... Our Visual Editor... Your own custom variation …”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official growthbook/growthbook docker image boot…”

   proof ↗⚑ flag

   Flagsmith✓fullprobed9/10

   Flagsmith docs explicitly document percentage-based rollouts via a % Split segment rule (1-100%) that can be adjusted without redeploying, and this is corroborated by a runtime probe showing a live flag create/evaluate loop. Consistent bucketing is implied by the segment-based percentage split mechanism used across SDKs/environments, though the docs don't explicitly describe the hashing/bucketing algorithm in detail. missing for 10: explicit documentation of the deterministic bucketing/hashing algorithm ensuring consistent user assignment across ramp changes.

   • [claimed-docs] “Add a rule defined with the % Split condition. Specify a value between 1 and 100 to define what percentage of your user base is included wit…”
   • [claimed-docs] “Add a rule defined with the **% Split** condition. Specify a value between 1 and 100 to define what percentage of your user base is included…”
   • [claimed-docs] “Rollouts by percentage (also known as staged feature rollouts) allow you to test a new feature with a small subset of your user base.”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official flagsmith/flagsmith unified docker imag…”

   proof ↗⚑ flag

Scheduling

1. developerSchedule flag changes and releases to happen at a specific future time

   weight 1 · round to Flagsmith
   GrowthBook~partialclaimed6/10

   GrowthBook explicitly documents that feature rules can be 'scoped, ramped, and scheduled,' and its pricing page lists 'Ramp schedules' as a distinct capability, confirming a future-time scheduling mechanism for flag rollouts. However, this appears gated to paid plans (Pro tier) and the evidence pack gives no detailed walkthrough of how scheduling is configured or any independent/hands-on confirmation it works as described. Missing for 10: detailed documentation of the scheduling UI/API, hands-on or community verification, and clarity on whether it applies broadly to all rule types.

   • [claimed-docs] “How rules override a feature's default value — targeting users, rolling out gradually, experimenting — and how to scope, ramp, and schedule …”
   • [claimed-docs] “Pro, plus: Custom environments Ramp schedules Approval workflows Advanced access control SSO & SCIM provisioning Exportable audit logs”
   • [claimed-docs] “Custom environments * Ramp schedules * Approval workflows * Advanced access control * SSO & SCIM provisioning * Exportable audit l…”

   proof ↗⚑ flag

   Flagsmith✓fullclaimed9/10

   Flagsmith's docs explicitly describe a Scheduled Flags feature that lets you queue changes to feature flags and have them automatically applied at a specified future time, removing the need for manual intervention at that moment — directly matching the story. This is further complemented by Release Pipelines, which can automatically stage/progress rollouts with triggers, reinforcing scheduled/automated release workflows. missing for 10: independent hands-on confirmation of the scheduling UI/behavior beyond first-party docs.

   • [claimed-docs] “Scheduled flags allow you to queue and automatically apply changes to feature flags at a specified future time, eliminating the need for man…”
   • [claimed-docs] “Scheduled flags allow you to queue and automatically apply changes to feature flags at a specified future time”
   • [claimed-docs] “Define stages that your flags progress through automatically, with triggers and actions that control rollouts to selected environments and a…”
   • [claimed-docs] “Release Pipelines act as your intelligent release assistant, automating your entire feature flag deployment process.”

   proof ↗⚑ flag

Edge

1. platform engineerEvaluate flags at the edge (CDN workers or an edge/relay layer) close to users

   weight 2 · round to GrowthBook
   GrowthBook✓fullclaimed8/10

   GrowthBook documents a dedicated Cloudflare Workers edge SDK that evaluates feature flags and Visual Editor experiments at the edge to cut flicker before HTML ships, plus a self-hostable Proxy layer that caches SDK payloads near users and keeps serving flags if the API is down. Missing for 10: independent hands-on evidence specifically of the edge/Cloudflare SDK in production (community evidence covers general SDK/self-host use but not edge-worker deployments) and broader edge-platform coverage beyond Cloudflare.

   • [claimed-docs] “Evaluate GrowthBook feature flags and Visual Editor experiments at the edge on Cloudflare Workers to cut flicker before HTML ships.”
   • [claimed-docs] “Run the GrowthBook Proxy next to your app to cache SDK payloads, reduce latency, and keep serving flags if the API is down.”
   • [claimed-docs] “The SDK Connection Endpoint provides readonly access to a subset of your feature flag data, just enough for the GrowthBook SDKs to assign va…”

   proof ↗⚑ flag

   Flagsmith~partialcommunity5/10

   Flagsmith ships an official Edge Proxy that can be self-hosted 'close to your servers' and queried by server-side SDKs (flagsmith-docs-14, flagsmith-docs-28), which addresses the general 'evaluate close to users' need, but this is a self-run relay service rather than a native CDN/edge-worker (e.g., Cloudflare Workers, Fastly Compute) integration. A community comment also notes Flagsmith's evaluation is 'primarily server-side' rather than fully local/in-memory, unlike some flagd-style hybrid approaches (flagsmith-comm-1), tempering claims of true edge-local evaluation. Missing for 10: a documented CDN/edge-worker runtime or SDK (e.g., Cloudflare/Fastly integration), multi-region edge network evidence, and independent latency benchmarks confirming edge-proximity gains.

   • [claimed-docs] “The Flagsmith Edge Proxy is a service that you host yourself, that allows you to run an instance of the Flagsmith Engine close to your serve…”
   • [claimed-docs] “Run the Edge Proxy within in your own infrastructure and connect to it from your server-side SDKs”
   • [community] “I would have thought so [flags evaluated in-memory locally]. But flagsmith apparently does primarily server-side eval. And even OpenFeature …”

   proof ↗⚑ flag

Evaluation

1. platform engineerMy server SDKs evaluate flags locally from a cached ruleset — microsecond decisions with no network call per flag check

   weight 3 · round to GrowthBook
   GrowthBook✓fullprobed9/10

   GrowthBook's core architecture is a cacheable JSON ruleset that SDKs evaluate locally, confirmed by docs (SDK Connection Endpoint providing readonly flag data for local assignment, in-memory attribute evaluation) and corroborated by a community founder comment describing exactly this model plus a runtime probe showing the keyless payload endpoint serving flag data for local SDK evaluation. Missing for 10: explicit published latency/microsecond benchmarks or independent third-party performance testing.

   • [claimed-docs] “The SDK Connection Endpoint provides readonly access to a subset of your feature flag data, just enough for the GrowthBook SDKs to assign va…”
   • [claimed-docs] “The actual values of targeting attributes (e.g., user IDs, emails) are never sent to GrowthBook. They are only stored in memory locally with…”
   • [community] “With GrowthBook, there's a single cacheable JSON definition of all of the features and then evaluation happens locally within the SDKs... Th…”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official growthbook/growthbook docker image boot…”
   • [claimed-docs] “Run the GrowthBook Proxy next to your app to cache SDK payloads, reduce latency, and keep serving flags if the API is down.”

   proof ↗⚑ flag

   Flagsmith!disputedcontradicted3/10

   Flagsmith docs describe an Edge Proxy you can self-host to reduce latency (flagsmith-docs-14/28), but this is still a network hop to a proxy, not a local in-process cached rule evaluation. A knowledgeable community source explicitly states Flagsmith 'does primarily server-side eval' rather than in-memory local evaluation (flagsmith-comm-1), and the probed runtime evidence shows flag reads going through a live API call (flagsmith-probe-rt-2), directly contradicting the 'no network call per flag check' claim. missing for 10: first-party SDK docs describing local ruleset caching/polling with microsecond in-process evaluation, and hands-on confirmation that server SDKs avoid per-check network calls.

   • [claimed-docs] “The Flagsmith Edge Proxy is a service that you host yourself, that allows you to run an instance of the Flagsmith Engine close to your serve…”
   • [claimed-docs] “Run the Edge Proxy within in your own infrastructure and connect to it from your server-side SDKs”
   • [community] “I would have thought so [flags evaluated in-memory locally]. But flagsmith apparently does primarily server-side eval. And even OpenFeature …”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official flagsmith/flagsmith unified docker imag…”

   proof ↗⚑ flag

Sdks

1. developerUse official SDKs across my whole stack — backend, web, and mobile — with consistent flag behavior

   weight 2 · round to GrowthBook
   GrowthBook✓fullcommunity7/10

   GrowthBook offers 24 official SDKs spanning backend, web/React, and mobile (Android/iOS) per GitHub README, with docs showing consistent flag-evaluation patterns (useFeatureIsOn, useFeatureValue, targeting attributes evaluated locally) and community reports of quick multi-stack integration (Vue frontend in hours). An older 2021 community comment noted no native mobile SDKs yet, but this is superseded by the more recent GitHub claim explicitly listing Android/iOS SDKs, so it's not a live contradiction. Missing for 10: independent verification of identical targeting/rollout behavior across all 24 SDKs, and more recent hands-on confirmation of mobile SDK maturity/parity with backend/web SDKs.

   • [github] “24 SDKs including React, Python, Android, and iOS.”
   • [claimed-docs] “Get GrowthBook feature flags and experiments running in your app in a few lines of code, then pick a language-specific SDK.”
   • [claimed-docs] “const showNewFeature = useFeatureIsOn("new-feature");”
   • [claimed-docs] “const showNewFeature = useFeatureIsOn("new-feature"); const buttonColor = useFeatureValue("button-color", "blue");”
   • [claimed-docs] “Add targeting conditions to a rule to control which values a user sees.”
   • [community] “At Miggos we have implemented Growth book and we got FF and A/B in our Vue frontend in a few hours. Excited to see the progress of the platf…”
   • [community] “We don't have native mobile SDKs yet, but it's something we want to support in the future. Mobile is a little tricky since you either need t…”

   proof ↗⚑ flag

   Flagsmith~partialcommunity6/10

   Flagsmith explicitly markets support for 15+ languages/frameworks across backend, web, and mobile, and its architecture (environment keys, consistent flag/segment/targeting model) implies consistent behavior across SDKs. However, the evidence pack lacks any per-SDK documentation detail, SDK-to-SDK consistency testing, or independent confirmation of parity across specific SDKs, and community commentary raises questions about server-side vs local evaluation consistency (flagd vs local evaluation) that isn't directly addressed. missing for 10: per-platform SDK documentation snippets, evidence of consistent flag evaluation logic/tests across SDKs, independent/hands-on confirmation of behavior parity across backend/web/mobile.

   • [github] “Choose from 15+ popular languages like Typescript, .NET, Java, and more. Integrate with any framework, including React, Next.js, and more”
   • [claimed-docs] “Manage feature flags and remote config across web, mobile, and server-side apps.”
   • [community] “I would have thought so [flags evaluated in-memory locally]. But flagsmith apparently does primarily server-side eval. And even OpenFeature …”

   proof ↗⚑ flag

Standards

1. platform engineerUse the vendor through OpenFeature providers so my flag code isn't locked to one vendor's SDK API

   weight 1 · round drawn
   GrowthBook—none0/10

   The evidence pack describes GrowthBook's own 24 SDKs, REST API, and proxy, but contains no mention of OpenFeature or an OpenFeature provider integration anywhere. Since this axis (avoiding vendor lock-in via OpenFeature) plausibly applies to a flagging platform, the absence of any supporting evidence yields 'none' rather than 'na'.

   • [github] “24 SDKs including React, Python, Android, and iOS.”
   • [claimed-docs] “Get GrowthBook feature flags and experiments running in your app in a few lines of code, then pick a language-specific SDK.”
   • [claimed-docs] “Get GrowthBook feature flags and experiments running in your app in a few lines of code”

   proof ↗⚑ flag

   Flagsmith—none0/10

   No evidence in the pack mentions OpenFeature support, an OpenFeature provider implementation, or any compatibility layer for the OpenFeature spec — Flagsmith is only shown with its own proprietary SDKs, API, and CLI. Missing for 10: any mention of an OpenFeature provider, spec compliance, or documented integration with the OpenFeature ecosystem.

   ⚑ flag

Streaming

1. developerFlag changes propagate to connected SDKs in seconds via streaming or fast polling — a kill switch actually kills

   weight 2 · round to GrowthBook
   GrowthBook~partialcommunity4/10

   GrowthBook's docs confirm a single cacheable feature-flag JSON payload that SDKs evaluate locally (growthbook-comm-6), a Proxy that caches payloads to reduce latency and keep serving flags if the API is down (growthbook-docs-11), and environment/rule toggles that could function as kill switches (growthbook-docs-3, growthbook-docs-22). However, no evidence in the pack documents actual streaming/SSE or fast-polling update latency in seconds, nor confirms a tested kill-switch effect at runtime. Missing for 10: explicit documentation of push/streaming update mechanism and propagation-time guarantees, and a hands-on/community test showing a kill switch immediately disabling a flag across connected SDKs.

   • [claimed-docs] “Run the GrowthBook Proxy next to your app to cache SDK payloads, reduce latency, and keep serving flags if the API is down.”
   • [community] “With GrowthBook, there's a single cacheable JSON definition of all of the features and then evaluation happens locally within the SDKs... Th…”
   • [claimed-docs] “Use environments such as dev, staging, and production to enable or disable feature flags independently.”
   • [claimed-docs] “How rules override a feature's default value — targeting users, rolling out gradually, experimenting — and how to scope, ramp, and schedule …”
   • [claimed-docs] “The SDK Connection Endpoint provides readonly access to a subset of your feature flag data, just enough for the GrowthBook SDKs to assign va…”

   proof ↗⚑ flag

   Flagsmith—none0/10

   The evidence pack describes flag targeting, rollouts, and toggling but contains no documentation of streaming updates, real-time push, or specific polling intervals that would substantiate seconds-level propagation to SDKs; a community comment even notes Flagsmith does 'primarily server-side eval' rather than confirming fast client propagation. Missing for 10: documented real-time/streaming SDK update mechanism, polling interval specs, and evidence of kill-switch propagation latency.

   • [community] “I would have thought so [flags evaluated in-memory locally]. But flagsmith apparently does primarily server-side eval. And even OpenFeature …”

   proof ↗⚑ flag