Statsig vs Unleash

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round drawn
   Statsig✓fullprobed9/10

   Statsig publishes a working llms.txt (probed live, HTTP 200) and offers a dedicated Docs MCP server that lets AI clients read documentation directly, confirmed via a full keyless initialize handshake at /api/mcp. This directly satisfies pointing an agent at agent-oriented docs. Missing for 10: independent third-party confirmation of agents actually using llms.txt in practice beyond the probe.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.statsig.com/llms.txt # Statsig Documentation > Official documentation for Statsig feature flags, e…”
   • [claimed-docs] “Use the Statsig Docs MCP server to let supported AI clients read public Statsig documentation directly from /api/mcp.”
   • [claimed-docs] “Use the Statsig Docs MCP server to let supported AI clients read public Statsig documentation directly from /api/mcp”
   • [probe] “PROBE runtime (recorded 2026-09-05): the documented no-auth docs MCP server at https://docs.statsig.com/api/mcp completed a FULL keyless ini…”

   proof ↗⚑ flag

   Unleash✓fullprobed9/10

   Unleash publishes a working llms.txt (HTTP 200, with explicit AI-agent instructions for fetching Markdown) plus a documented, MCP-reachable docs endpoint, giving agents structured, agent-oriented documentation entry points. Missing for 10: independent (non-vendor) confirmation that third-party agents actually consume these successfully in the wild.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.getunleash.io/llms.txt # Unleash Documentation ## Instructions for AI Agents - For clean Markdown…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the docs MCP endpoint https://docs.getunleash.io/_mcp/server completed a FULL keyless initialize handsh…”
   • [claimed-docs] “This server enables LLM-powered coding assistants to create and manage feature flags following Unleash best practices.”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round drawn
   Statsig✓fullprobed8/10

   Statsig ships an official CLI (@statsig/siggy) that runs keylessly and is explicitly documented and probe-verified for CI/CD automation of Feature Gates and Experiments, plus a Console API/HTTP API for headless programmatic control. Runtime probes confirm the CLI executes via npx in a scriptable, non-interactive way suitable for automation pipelines. Missing for 10: no independent third-party CI pipeline case study or deeper docs on CLI command coverage beyond gates/experiments.

   • [claimed-docs] “The Statsig CLI is a command-line interface that helps manage Feature Gates, Experiments, and Dynamic Configs within the Statsig platform.”
   • [claimed-docs] “You can integrate the CLI into CI/CD pipelines to automate the management of Feature Gates and Experiments.”
   • [claimed-docs] “The Console API is the CRUD API for performing the actions available on console.statsig.com without using the web UI.”
   • [claimed-docs] “Overview of the Statsig HTTP API for retrieving feature gate, experiment, and dynamic config values and logging events directly without an S…”
   • [probe] “official CLI documented at https://docs.statsig.com/statsigcli/introduction”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Statsig CLI (npm @statsig/siggy) ran keylessly via `npx -y @statsig/siggy --version`, prin…”

   proof ↗⚑ flag

   Unleash✓fullprobed8/10

   Unleash ships a self-hostable Docker image, a full Admin/Client API, and scoped API tokens designed for automation/SDKs rather than human UI use, and a hands-on probe confirms a complete keyless headless workflow (docker boot, flag creation via Admin API, evaluation via Client API) with no manual UI steps — exactly the CI/automation pattern the story asks for. missing for 10: no independent third-party CI pipeline example or official CI/CD docs page walking through pipeline integration.

   • [claimed-docs] “This guide helps you set up Unleash Open Source or Unleash Enterprise in your own environment using Docker.”
   • [claimed-docs] “The Admin API provides comprehensive programmatic access to all Unleash resources.”
   • [claimed-docs] “Unleash uses API keys to facilitate communication between consuming clients such as SDKs, Unleash Edge, or other tools and automation.”
   • [claimed-docs] “Backend tokens are scoped to one or more projects and a single environment.”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official unleashorg/unleash-server docker image …”

   proof ↗⚑ flag

3. ai-native userConnect an agent via an official MCP server

   weight 3 · round to Statsig
   Statsig✓fullprobed9/10

   Statsig ships an official MCP server integrating with agent tools like Codex, Cursor, and Claude Code for querying Statsig data, plus a separate docs MCP server; runtime probes confirm both endpoints are live, with the main MCP server enforcing OAuth and the docs server completing a full handshake. missing for 10: independent third-party (non-vendor) hands-on report of using the MCP server in a real agent workflow beyond docs/probes.

   • [claimed-docs] “The Statsig MCP (Model Context Protocol) server brings the power of Statsig into tools like Codex, Cursor, and Claude Code. With this setup,…”
   • [claimed-docs] “The Statsig MCP (Model Context Protocol) server brings the power of Statsig into tools like Codex, Cursor, and Claude Code.”
   • [claimed-docs] “Use the Statsig Docs MCP server to let supported AI clients read public Statsig documentation directly from /api/mcp.”
   • [probe] “official MCP server documented at https://docs.statsig.com/integrations/mcp/overview”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.statsig.com/v1/mcp returned HTTP …”
   • [probe] “PROBE runtime (recorded 2026-09-05): the documented no-auth docs MCP server at https://docs.statsig.com/api/mcp completed a FULL keyless ini…”

   proof ↗⚑ flag

   Unleash✓fullprobed8/10

   Unleash documents an official MCP server (@unleash/mcp) that lets LLM-powered coding assistants create and manage feature flags following best practices, and a probe confirms the docs MCP endpoint completes a full handshake. missing for 10: independent/hands-on third-party validation of the flag-management MCP server itself (only the docs-MCP handshake was runtime-verified, not the @unleash/mcp flag server), and no evidence of broader community adoption reports.

   • [claimed-docs] “This server enables LLM-powered coding assistants to create and manage feature flags following Unleash best practices.”
   • [probe] “official MCP server documented at https://docs.getunleash.io/integrate/mcp”
   • [probe] “PROBE runtime (recorded 2026-09-05): the docs MCP endpoint https://docs.getunleash.io/_mcp/server completed a FULL keyless initialize handsh…”

   proof ↗⚑ flag

4. ai-native userUse an official CLI

   weight 2 · round to Statsig
   Statsig✓fullprobed9/10

   Statsig ships an official CLI (Statsig CLI / @statsig/siggy) for managing Feature Gates, Experiments, and Dynamic Configs, documented for CI/CD automation, and a runtime probe confirms it executes keylessly via npx and is scriptable. missing for 10: no independent (non-Statsig) hands-on review of the CLI's full feature set beyond the version-check probe.

   • [claimed-docs] “The Statsig CLI is a command-line interface that helps manage Feature Gates, Experiments, and Dynamic Configs within the Statsig platform.”
   • [claimed-docs] “You can integrate the CLI into CI/CD pipelines to automate the management of Feature Gates and Experiments.”
   • [probe] “official CLI documented at https://docs.statsig.com/statsigcli/introduction”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Statsig CLI (npm @statsig/siggy) ran keylessly via `npx -y @statsig/siggy --version`, prin…”

   proof ↗⚑ flag

   Unleash—none0/10

   The evidence pack details SDKs, Admin API, MCP server, and Docker deployment, but no official CLI tool for Unleash is mentioned anywhere in the docs or GitHub materials.

   ⚑ flag

5. ai-native userDrive the product through a documented public API

   weight 3 · round drawn
   Statsig✓fullprobed9/10

   Statsig documents multiple public API surfaces (HTTP API for gate/experiment retrieval and event logging, full CRUD Console API, and CLI) that let developers/agents drive the product programmatically, and runtime probes confirm the CLI works keylessly and the MCP servers respond live. missing for 10: a discoverable OpenAPI/Swagger spec (probe found 404s on standard OpenAPI paths), which would give full machine-readable API discoverability.

   • [claimed-docs] “Overview of the Statsig HTTP API for retrieving feature gate, experiment, and dynamic config values and logging events directly without an S…”
   • [claimed-docs] “The Console API is the CRUD API for performing the actions available on console.statsig.com without using the web UI.”
   • [claimed-docs] “The Statsig CLI is a command-line interface that helps manage Feature Gates, Experiments, and Dynamic Configs within the Statsig platform.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Statsig CLI (npm @statsig/siggy) ran keylessly via `npx -y @statsig/siggy --version`, prin…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.statsig.com/openapi.json, https://docs.statsig.com/swagger.json, https://docs.statsig.c…”

   proof ↗⚑ flag

   Unleash✓fullprobed9/10

   Unleash's Admin API is explicitly documented as providing comprehensive programmatic access to all resources, backed by scoped API tokens/keys, and this is corroborated by a hands-on probe showing a full keyless create-and-evaluate flag roundtrip via the Admin and Client APIs. This directly demonstrates AI-native drivability through a documented public API. Missing for 10: no independent third-party API client showcase beyond Unleash's own SDK/probe evidence.

   • [claimed-docs] “The Admin API provides comprehensive programmatic access to all Unleash resources.”
   • [claimed-docs] “Backend tokens are scoped to one or more projects and a single environment.”
   • [claimed-docs] “Unleash uses API keys to facilitate communication between consuming clients such as SDKs, Unleash Edge, or other tools and automation.”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official unleashorg/unleash-server docker image …”

   proof ↗⚑ flag

6. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round to Unleash
   Statsig~partialprobed3/10

   Statsig documents general access-management/SSO/SCIM controls and shows its hosted MCP server requires an OAuth bearer token rather than being fully open (statsig-probe-rt-2), implying some authenticated, permissioned access surface an agent could use. However there is no explicit documentation of issuing scoped or least-privilege API keys/tokens specifically for agent use (e.g., read-only or project-scoped Console API keys). Missing for 10: explicit docs on creating scoped/limited-permission API keys, role-based key scoping for agents, and any guidance on least-privilege credential issuance for AI agents.

   • [claimed-docs] “In many enterprise environments, organizations use SSO and SCIM together to improve security and usability.”
   • [claimed-docs] “Basic settings include automatically adding new users with the same email domain to your project. Statsig also supports SSO to simplify invi…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.statsig.com/v1/mcp returned HTTP …”

   proof ↗⚑ flag

   Unleash~partialprobed6/10

   Unleash supports scoped credentials generally: backend API tokens are scoped to specific projects and a single environment, frontend tokens are explicitly non-secret/least-privilege by design, and RBAC governs permissions; the official MCP flag-management server requires a PAT (a scoped credential) at startup, which is directly relevant to agent access. However, there's no explicit documentation of fine-grained, agent-specific credential scoping (e.g., read-only vs write scopes tailored for an AI agent) beyond project/environment granularity. Missing for 10: dedicated agent-credential scoping guide, granular permission scopes beyond project+environment, and independent confirmation of least-privilege PAT usage in agent workflows.

   • [claimed-docs] “Backend tokens are scoped to one or more projects and a single environment.”
   • [claimed-docs] “Unleash uses API keys to facilitate communication between consuming clients such as SDKs, Unleash Edge, or other tools and automation.”
   • [claimed-docs] “Frontend tokens are not considered secret and are safe to expose client-side.”
   • [claimed-docs] “Unleash implements role-based access control on two levels”
   • [probe] “PROBE runtime (recorded 2026-09-05): the docs MCP endpoint https://docs.getunleash.io/_mcp/server completed a FULL keyless initialize handsh…”
   • [claimed-docs] “This server enables LLM-powered coding assistants to create and manage feature flags following Unleash best practices.”

   proof ↗⚑ flag

7. ai-native userBuild against official SDKs

   weight 2 · round to Unleash
   Statsig✓fullclaimed7/10

   Statsig's docs explicitly describe SDK architecture (client vs. server evaluation, on-device eval SDKs, deterministic evaluation across platforms) and even offer an HTTP API for cases without an SDK, confirming a first-party SDK ecosystem that AI-native developers can build against. Missing for 10: a concrete list/matrix of supported languages/platforms and independent hands-on confirmation of SDK usage beyond docs.

   • [claimed-docs] “Evaluation in Statsig is deterministic. Given the same user object and the same state of the experiment or feature gate, Statsig always retu…”
   • [claimed-docs] “Statsig's client-side On-Device Eval SDKs provide an alternate client-side architecture. The device keeps the definition of each experiment …”
   • [claimed-docs] “Evaluation in Statsig is deterministic. Given the same user object and the same state of the experiment or feature gate, Statsig always retu…”
   • [claimed-docs] “Statsig's client-side On-Device Eval SDKs provide an alternate client-side architecture. The device keeps the definition of each experiment …”
   • [claimed-docs] “Overview of the Statsig HTTP API for retrieving feature gate, experiment, and dynamic config values and logging events directly without an S…”

   proof ↗⚑ flag

   Unleash✓fullclaimed8/10

   Unleash documents official client/backend SDKs as the primary integration path (quickstart guide, API token docs distinguishing frontend/backend SDK scoping), which AI-native developers can build against directly. missing for 10: no independent/hands-on corroboration of SDK quality or breadth across languages, and no explicit SDK versioning/changelog evidence.

   • [claimed-docs] “Create your first feature flag** using the Unleash Admin UI. **Connect Unleash to your application** by integrating an SDK into your codebas…”
   • [claimed-docs] “use one of the client or backend SDKs to connect Unleash with your application”
   • [claimed-docs] “Create your first feature flag using the Unleash Admin UI. Connect Unleash to your application by integrating an SDK into your codebase.”
   • [claimed-docs] “Backend tokens are scoped to one or more projects and a single environment.”
   • [claimed-docs] “Unleash uses API keys to facilitate communication between consuming clients such as SDKs, Unleash Edge, or other tools and automation.”
   • [claimed-docs] “Frontend tokens are not considered secret and are safe to expose client-side.”

   proof ↗⚑ flag

8. ai-native userSubscribe to events via webhooks

   weight 2 · round drawn
   Statsig—none0/10

   The evidence pack contains no mention of webhooks or event subscription mechanisms anywhere in Statsig's docs, community posts, or probes — only feature gates, experiments, CLI, MCP, and access management are covered. Webhooks are a plausible axis for a flagging/analytics platform, so absence of evidence yields none rather than na.

   ⚑ flag

   Unleash—none0/10

   The evidence pack contains no mention of webhooks, event subscriptions, or push-based notification mechanisms anywhere in Unleash's docs, API overview, or integrations list — only SDKs, Admin/Client APIs, Edge caching, and MCP flag-management tools are documented.

   ⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round to Statsig
   Statsig~partialprobed5/10

   Statsig's MCP server explicitly lets AI clients (Claude Code, Cursor, Codex) 'ask questions, explore experiments, and access your Statsig data using AI,' which is a form of AI-generated insight retrieval, and this is confirmed live via a working OAuth-gated MCP endpoint. However this is an external-agent integration rather than a native in-product AI insights/suggestions feature (e.g., no evidence of built-in AI-generated experiment summaries, anomaly explanations, or suggestion widgets inside the Statsig console itself). Missing for 10: native in-app AI-generated insights/copilot UI, evidence of proactive AI suggestions surfaced directly in the console rather than via external MCP client.

   • [claimed-docs] “The Statsig MCP (Model Context Protocol) server brings the power of Statsig into tools like Codex, Cursor, and Claude Code. With this setup,…”
   • [claimed-docs] “The Statsig MCP (Model Context Protocol) server brings the power of Statsig into tools like Codex, Cursor, and Claude Code.”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.statsig.com/v1/mcp returned HTTP …”
   • [probe] “official MCP server documented at https://docs.statsig.com/integrations/mcp/overview”

   proof ↗⚑ flag

   Unleash—none0/10

   Unleash's evidence covers feature flag management, impact metrics thresholds, and an MCP server that lets external LLM coding assistants create/manage flags — but nothing shows Unleash itself generating AI insights or suggestions from data inside the product (e.g., AI-driven rollout recommendations, anomaly analysis, or narrative summaries). The impact metrics feature is rule-based automation, not AI-generated insight, and the MCP server is for external agents to act on Unleash, not Unleash surfacing AI-generated analysis to users.

   • [claimed-docs] “Progress from 25% to 50% only if error rates stay below a threshold.”
   • [claimed-docs] “Automate milestone progression or trigger safeguards that pause rollouts or disable environments when metrics cross a threshold.”
   • [claimed-docs] “Pause automatically when request latency increases during a rollout.”
   • [claimed-docs] “This server enables LLM-powered coding assistants to create and manage feature flags following Unleash best practices.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the docs MCP endpoint https://docs.getunleash.io/_mcp/server completed a FULL keyless initialize handsh…”

   proof ↗⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Statsig
   Statsig~partialprobed6/10

   Statsig supports background-running automations like Scheduled Rollouts (gradual gate rollout without manual intervention) and Autotune (autonomously shifts traffic toward winning variants), plus CLI integration into CI/CD pipelines for automated gate/experiment management. These are real autonomous background mechanisms, but they are experimentation/rollout automations rather than general-purpose AI-agent automations, and there's no evidence of scheduling/triggering arbitrary agentic workflows. Missing for 10: evidence of general-purpose scheduled/triggered agent workflows beyond rollout/autotune, and independent verification that these run reliably unattended in production.

   • [claimed-docs] “Gradually deploy a feature over time by setting up a Feature Gate as a Scheduled Rollout.”
   • [claimed-docs] “Unlike a standard A/B test, Autotune shifts traffic toward the winning variant to maximize a target metric.”
   • [claimed-docs] “Autotune shifts traffic toward the winning variant to maximize a target metric.”
   • [claimed-docs] “You can integrate the CLI into CI/CD pipelines to automate the management of Feature Gates and Experiments.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Statsig CLI (npm @statsig/siggy) ran keylessly via `npx -y @statsig/siggy --version`, prin…”

   proof ↗⚑ flag

   Unleash~partialprobed5/10

   Unleash documents metrics-driven automation — progressing rollout milestones or triggering safeguards (pause/disable) automatically when thresholds are crossed — which is a real background automation running without manual intervention, and its MCP integration lets AI agents create/manage flags. However this automation is narrowly scoped to release/rollout progression rather than a general-purpose agentic automation framework, and there's no evidence of scheduling arbitrary autonomous agent tasks beyond flag rollout safeguards. Missing for 10: evidence of broader autonomous task scheduling beyond rollout thresholds, independent corroboration of the impact-metrics automation working in production, and confirmation the MCP server itself can trigger these automations autonomously rather than just via API calls.

   • [claimed-docs] “Release management in Unleash helps you standardize, automate and safeguard your feature rollouts.”
   • [claimed-docs] “Progress from 25% to 50% only if error rates stay below a threshold.”
   • [claimed-docs] “Automate milestone progression or trigger safeguards that pause rollouts or disable environments when metrics cross a threshold.”
   • [claimed-docs] “Pause automatically when request latency increases during a rollout.”
   • [claimed-docs] “This server enables LLM-powered coding assistants to create and manage feature flags following Unleash best practices.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the docs MCP endpoint https://docs.getunleash.io/_mcp/server completed a FULL keyless initialize handsh…”

   proof ↗⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round drawn
   Statsig—none0/10

   Evidence covers Statsig's MCP server, which lets external AI tools (Cursor, Claude Code, Codex) query Statsig data — the opposite direction of an in-product assistant that users delegate tasks to. No evidence of a built-in AI assistant embedded in the Statsig console itself.

   ⚑ flag

   Unleash—none0/10

   Unleash exposes an MCP server so external LLM coding assistants can manage flags (unleash-docs-7, unleash-probe-2, unleash-probe-rt-2), but this is the reverse of the story — it makes Unleash a tool for outside agents, not a built-in assistant living inside the Unleash product that users delegate tasks to. No evidence of an in-product AI assistant/chat feature for creating rollouts, analyzing flags, etc.

   • [claimed-docs] “This server enables LLM-powered coding assistants to create and manage feature flags following Unleash best practices.”
   • [probe] “official MCP server documented at https://docs.getunleash.io/integrate/mcp”
   • [probe] “PROBE runtime (recorded 2026-09-05): the docs MCP endpoint https://docs.getunleash.io/_mcp/server completed a FULL keyless initialize handsh…”

   proof ↗⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round to Unleash
   Statsig~partialprobed6/10

   Statsig ships a first-party MCP server (verified live via runtime probe) that lets AI clients like Claude Code and Cursor query experiments, gates, and data using natural language, which is the core mechanism for natural-language operation. However, this is an agentic/AI-integration layer bolted onto a console/CLI/API-driven product rather than a native NL command interface across the whole product surface. missing for 10: evidence of natural-language command support within the core Statsig console UI itself (not just via external MCP-connected agents), independent hands-on validation of NL command accuracy/reliability, and coverage of write-actions (creating/modifying gates or experiments) via natural language rather than read-only querying.

   • [claimed-docs] “The Statsig MCP (Model Context Protocol) server brings the power of Statsig into tools like Codex, Cursor, and Claude Code. With this setup,…”
   • [claimed-docs] “The Statsig MCP (Model Context Protocol) server brings the power of Statsig into tools like Codex, Cursor, and Claude Code.”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.statsig.com/v1/mcp returned HTTP …”
   • [probe] “official MCP server documented at https://docs.statsig.com/integrations/mcp/overview”

   proof ↗⚑ flag

   Unleash✓fullprobed7/10

   Unleash ships an official MCP server (@unleash/mcp) documented to let LLM-powered coding assistants create and manage feature flags following best practices, and a probe confirms the docs' MCP endpoint completes a handshake, corroborating the integration is real and agent-reachable. Missing for 10: no independent/hands-on demonstration of the flag-management MCP server itself in use (only the docs MCP endpoint was probed), and it requires local setup with a PAT rather than being zero-config.

   • [claimed-docs] “This server enables LLM-powered coding assistants to create and manage feature flags following Unleash best practices.”
   • [probe] “official MCP server documented at https://docs.getunleash.io/integrate/mcp”
   • [probe] “PROBE runtime (recorded 2026-09-05): the docs MCP endpoint https://docs.getunleash.io/_mcp/server completed a FULL keyless initialize handsh…”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   Statsig—none0/10

   Statsig documents an HTTP API and Console API but there is no evidence of an interactive API reference with runnable/try-it examples; explicit probes for OpenAPI/Swagger endpoints returned 404s, indicating no such interactive explorer exists.

   • [claimed-docs] “Overview of the Statsig HTTP API for retrieving feature gate, experiment, and dynamic config values and logging events directly without an S…”
   • [claimed-docs] “The Console API is the CRUD API for performing the actions available on console.statsig.com without using the web UI.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.statsig.com/openapi.json, https://docs.statsig.com/swagger.json, https://docs.statsig.c…”

   proof ↗⚑ flag

   Unleash—none0/10

   The evidence pack shows Admin API docs and an MCP server for AI agents to manage flags, but nothing describes an interactive API reference (e.g., Swagger/OpenAPI explorer) with runnable/try-it-out examples. This is a fair capability to expect from an API-driven product, but no evidence supports it.

   ⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   Statsig—none0/10

   Statsig documents a Console API and HTTP API but a direct probe for OpenAPI/swagger specs at common paths (openapi.json, swagger.json, api/openapi.json, .well-known/openapi.json) all returned 404, and no docs reference a downloadable machine-readable spec.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.statsig.com/openapi.json, https://docs.statsig.com/swagger.json, https://docs.statsig.c…”
   • [claimed-docs] “The Console API is the CRUD API for performing the actions available on console.statsig.com without using the web UI.”
   • [claimed-docs] “Overview of the Statsig HTTP API for retrieving feature gate, experiment, and dynamic config values and logging events directly without an S…”

   proof ↗⚑ flag

   Unleash—none0/10

   The evidence pack describes Unleash's Admin API as providing 'comprehensive programmatic access to all Unleash resources' but never mentions an OpenAPI/Swagger spec, schema file, or any downloadable machine-readable API definition. Missing for 10: explicit mention of an OpenAPI/Swagger document, a spec download endpoint, or documentation confirming machine-readable API schema availability.

   • [claimed-docs] “The Admin API provides comprehensive programmatic access to all Unleash resources.”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round to Unleash
   Statsig—none0/10

   The evidence pack documents feature gates, experiments, CLI, and MCP integrations, but nothing describes a distinct sandbox/staging environment or project isolated from production data for testing purposes — only production-oriented tools like emergency disable switches, gate overrides, and scheduled rollouts are mentioned. This is a fair question for a feature-flag/experimentation platform, but no evidence confirms a sandbox capability exists.

   • [claimed-docs] “Feature Gates, commonly known as feature flags, allow you to toggle the behavior of your product in real time without deploying new code.”
   • [claimed-docs] “Have an emergency disable switch that lets you immediately turn off a particular code branch for users in production”
   • [claimed-docs] “After your Feature Gate is live, view Feature Gate exposures in the Statsig console to monitor who encounters your gate.”
   • [claimed-docs] “set up Feature Gate overrides to allow specific users to bypass the gate.”
   • [claimed-docs] “Basic settings include automatically adding new users with the same email domain to your project. Statsig also supports SSO to simplify invi…”

   proof ↗⚑ flag

   Unleash✓fullprobed7/10

   Unleash's environments concept explicitly supports enabling flags in development/test without affecting production (unleash-docs-21, unleash-docs-10), and a hands-on probe confirms a full self-hosted Docker instance can be spun up locally with its own Postgres, tokens, and flag create/evaluate loop entirely separate from any production account (unleash-probe-rt-1). This gives an AI-native user a genuine sandbox path via self-hosting plus environment isolation. Missing for 10: no dedicated 'sandbox mode' or ephemeral test-environment feature purpose-built for AI agents, and no independent (non-vendor) corroboration of the environment-isolation guarantees in production use.

   • [claimed-docs] “This allows you to enable a flag in development or test without enabling it in production.”
   • [claimed-docs] “Environments represent different stages in your development lifecycle. They allow you to manage your product releases from local development…”
   • [claimed-docs] “This guide helps you set up Unleash Open Source or Unleash Enterprise in your own environment using Docker.”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official unleashorg/unleash-server docker image …”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Statsig—none0/10

   Statsig documents an HTTP API and Console API but the evidence pack contains no mention of API versioning scheme or a documented deprecation policy; a probe for OpenAPI spec files (which would typically reveal version info) returned 404s across all candidate paths.

   • [claimed-docs] “Overview of the Statsig HTTP API for retrieving feature gate, experiment, and dynamic config values and logging events directly without an S…”
   • [claimed-docs] “The Console API is the CRUD API for performing the actions available on console.statsig.com without using the web UI.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.statsig.com/openapi.json, https://docs.statsig.com/swagger.json, https://docs.statsig.c…”

   proof ↗⚑ flag

   Unleash—none0/10

   The evidence pack documents the Admin API, tokens, and SDKs but contains no mention of API versioning scheme or a documented deprecation policy for Unleash's APIs. Missing for 10: any documentation of API version numbers, changelogs, or deprecation/sunset policy for endpoints or SDKs.

   • [claimed-docs] “The Admin API provides comprehensive programmatic access to all Unleash resources.”
   • [claimed-docs] “Backend tokens are scoped to one or more projects and a single environment.”
   • [claimed-docs] “Unleash uses API keys to facilitate communication between consuming clients such as SDKs, Unleash Edge, or other tools and automation.”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Statsig
   Statsig~partialprobed6/10

   Statsig's Console API provides a CRUD API for managing gates, experiments, and configs programmatically, and the CLI (confirmed runnable via npx) can be scripted/integrated into CI/CD, enabling automation across many items rather than one-by-one UI clicks. However, no evidence explicitly documents a bulk/batch endpoint or bulk-edit feature, so bulk operations must be assembled by looping API/CLI calls rather than a native bulk primitive; missing for 10: explicit bulk/batch API endpoints, documented multi-item transactional updates, and independent confirmation of large-scale automation use.

   • [claimed-docs] “The Console API is the CRUD API for performing the actions available on console.statsig.com without using the web UI.”
   • [claimed-docs] “The Statsig CLI is a command-line interface that helps manage Feature Gates, Experiments, and Dynamic Configs within the Statsig platform.”
   • [claimed-docs] “You can integrate the CLI into CI/CD pipelines to automate the management of Feature Gates and Experiments.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Statsig CLI (npm @statsig/siggy) ran keylessly via `npx -y @statsig/siggy --version`, prin…”

   proof ↗⚑ flag

   Unleash~partialclaimed3/10

   The Admin API is described as providing 'comprehensive programmatic access to all Unleash resources,' which implies scripting could achieve bulk create/update flows, but no evidence documents an explicit bulk-operations feature (e.g., batch enable/disable, bulk import/export, multi-select actions in UI) beyond a vague pricing-page mention of 'export/import feature flags for easy migration.' missing for 10: explicit bulk-update/bulk-tagging API endpoints, UI multi-select bulk actions, and any hands-on demonstration of operating on many items in one call.

   • [claimed-docs] “The Admin API provides comprehensive programmatic access to all Unleash resources.”
   • [claimed-docs] “Export/import feature flags for easy migration”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to Unleash
   Statsig~partialclaimed4/10

   Statsig supports rule-based automation such as targeting rules on Feature Gates, Scheduled Rollouts, and Autotune (which automatically shifts traffic toward a winning variant based on live metrics) — these are forms of automatic rule-triggered behavior, but they are evaluation/targeting rules and traffic-shift algorithms rather than a general 'on event X, trigger action Y' automation engine. Missing for 10: documented event-triggered workflow/webhook automation (e.g., alert-to-action pipelines), explicit event-based trigger definitions distinct from targeting/eval rules, and independent hands-on evidence of such automation working end-to-end.

   • [claimed-docs] “Gradually deploy a feature over time by setting up a Feature Gate as a Scheduled Rollout.”
   • [claimed-docs] “Unlike a standard A/B test, Autotune shifts traffic toward the winning variant to maximize a target metric.”
   • [claimed-docs] “Schedule gradual feature rollouts to safely deploy new code”
   • [claimed-docs] “Autotune shifts traffic toward the winning variant to maximize a target metric.”
   • [claimed-docs] “set up Feature Gate overrides to allow specific users to bypass the gate.”
   • [claimed-docs] “Infra Analytics: Monitor and debug service health alongside product outcomes. Ingest metrics and traces with OpenTelemetry, search and group…”

   proof ↗⚑ flag

   Unleash~partialclaimed6/10

   Unleash's impact-metrics feature lets users define threshold-based rules that automatically trigger actions — progressing rollouts, pausing on latency/error-rate breaches, or disabling environments — which is genuine event-triggered automation ([unleash-docs-14],[unleash-docs-22],[unleash-docs-29]). However this is scoped narrowly to rollout safeguards rather than a general-purpose rule/event engine for arbitrary actions across the system. Missing for 10: evidence of a broader rule-definition system covering non-rollout events/actions, and independent confirmation the automation works reliably in practice.

   • [claimed-docs] “Progress from 25% to 50% only if error rates stay below a threshold.”
   • [claimed-docs] “Automate milestone progression or trigger safeguards that pause rollouts or disable environments when metrics cross a threshold.”
   • [claimed-docs] “Pause automatically when request latency increases during a rollout.”
   • [claimed-docs] “Release management in Unleash helps you standardize, automate and safeguard your feature rollouts.”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round drawn
   Statsig—none0/10

   Statsig only documents one-time 'Scheduled Rollouts' for feature gates and CLI/CI-CD scripting of flag management (statsig-docs-3, statsig-docs-25, statsig-docs-31); there is no evidence of a recurring job/workflow scheduler (e.g., cron-like recurring automation) that an AI-native user could set up. missing for 10: any documented recurring/cron-style job scheduling, workflow orchestration, or automated recurring task execution beyond a single scheduled feature rollout.

   • [claimed-docs] “Gradually deploy a feature over time by setting up a Feature Gate as a Scheduled Rollout.”
   • [claimed-docs] “Schedule gradual feature rollouts to safely deploy new code”
   • [claimed-docs] “You can integrate the CLI into CI/CD pipelines to automate the management of Feature Gates and Experiments.”

   proof ↗⚑ flag

   Unleash—none0/10

   Unleash supports one-time scheduled feature releases (specific date/time) and conditional automation of rollout milestones/pauses based on metrics, but there is no evidence of recurring/cron-style job scheduling or workflow automation that repeats on an interval.

   • [claimed-docs] “to release a feature at a specific date and time** (for a product launch, for instance)”
   • [claimed-docs] “to release a feature at a specific date and time (for a product launch, for instance)”
   • [claimed-docs] “Automate milestone progression or trigger safeguards that pause rollouts or disable environments when metrics cross a threshold.”
   • [claimed-docs] “Pause automatically when request latency increases during a rollout.”

   proof ↗⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round to Unleash
   Statsig~partialclaimed4/10

   Statsig's feature gates support rollback-like controls (emergency disable switch, scheduled rollouts) and CLI/Console API can be scripted into CI/CD for programmatic management, implying some versioned control over flag state, but the evidence never documents an explicit audit/version history, diffing, or approval-review workflow for these 'automations'. missing for 10: explicit version history/audit log for gate or experiment changes, a review/approval workflow before changes go live, and a documented one-click rollback to a prior version (only an on/off disable switch is shown).

   • [claimed-docs] “Have an emergency disable switch that lets you immediately turn off a particular code branch for users in production”
   • [claimed-docs] “Gradually deploy a feature over time by setting up a Feature Gate as a Scheduled Rollout.”
   • [claimed-docs] “The Statsig CLI is a command-line interface that helps manage Feature Gates, Experiments, and Dynamic Configs within the Statsig platform.”
   • [claimed-docs] “You can integrate the CLI into CI/CD pipelines to automate the management of Feature Gates and Experiments.”
   • [claimed-docs] “The Console API is the CRUD API for performing the actions available on console.statsig.com without using the web UI.”
   • [claimed-docs] “After your Feature Gate is live, view Feature Gate exposures in the Statsig console to monitor who encounters your gate.”

   proof ↗⚑ flag

   Unleash~partialclaimed5/10

   Unleash's 'automations' are feature-flag rollouts: change requests provide a review/approval step before changes go live (docs-11), and flags can be toggled or reverted instantly without redeploying (docs-18), giving a rollback-like mechanism, plus RBAC constrains who can change what (docs-12). However there is no evidence of an explicit version-history/audit trail or diff view for flag configuration changes, only approval gating and on/off toggling. Missing for 10: documented changelog/audit-log of flag config versions, explicit 'rollback to previous version' feature, and independent confirmation of change-request review workflow in practice.

   • [claimed-docs] “Change requests allow you to require an additional approval step before any changes can be made in an environment.”
   • [claimed-docs] “They allow you to enable and disable features for certain users without having to redeploy your application.”
   • [claimed-docs] “Unleash implements role-based access control on two levels”
   • [claimed-docs] “Release management in Unleash helps you standardize, automate and safeguard your feature rollouts.”

   proof ↗⚑ flag

Proxy

1. platform engineerRun a relay/edge proxy so flags stay served when the vendor is unreachable and SDK traffic stays inside my network

   weight 1 · round to Unleash
   Statsig—none0/10

   No evidence in the pack describes a self-hostable relay/edge proxy for offline or in-network flag serving; documentation covers SDKs, HTTP API, CLI, and MCP but nothing about a proxy component that caches flags locally when Statsig's servers are unreachable.

   ⚑ flag

   Unleash✓fullclaimed8/10

   Unleash Edge (open source) and Unleash Enterprise Edge are documented as a lightweight caching/relay layer that sits between SDKs and the Unleash server, acting as a read replica that can serve thousands of SDKs and improve resilience — keeping SDK traffic local and flags served even if the vendor/backend is unreachable. Self-hosting via Docker is also documented, reinforcing the on-network deployment model. Missing for 10: independent/hands-on validation of Edge's offline-resilience behavior and more detail on failover semantics beyond docs claims.

   • [claimed-docs] “functioning as a read replica that can handle thousands of connected SDKs without increasing the read load on your primary Unleash instance”
   • [claimed-docs] “Unleash Enterprise Edge is a lightweight caching layer designed to improve scalability, performance, and resilience.”
   • [claimed-docs] “This guide helps you set up Unleash Open Source or Unleash Enterprise in your own environment using Docker.”
   • [claimed-docs] “Unleash uses API keys to facilitate communication between consuming clients such as SDKs, Unleash Edge, or other tools and automation.”

   proof ↗⚑ flag

Self host

1. platform engineerSelf-host the full flag platform from an open-source distribution, keeping evaluation data on my infrastructure

   weight 3 · round to Unleash
   Statsig—none0/10

   No evidence in the pack mentions an open-source self-hosted distribution of the full Statsig platform; all material describes the hosted SaaS console, SDKs, CLI, and MCP integrations that connect to Statsig's cloud API. Absence of evidence for this applicable axis means it is not delivered.

   ⚑ flag

   Unleash✓fullprobed9/10

   Docs explicitly describe deploying Unleash Open Source via Docker in your own environment, and a hands-on probe confirms a full keyless self-hosted flag create-and-evaluate loop using the official unleash-server image against Postgres with no external account, demonstrating evaluation data stays on the operator's infrastructure. missing for 10: no independent third-party production case study of long-term self-hosted operation at scale.

   • [claimed-docs] “This guide helps you set up Unleash Open Source or Unleash Enterprise in your own environment using Docker.”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official unleashorg/unleash-server docker image …”
   • [github] “Feature flags in Unleash let you test your code with real production data, reducing the risk of negatively impacting your users' experience.”

   proof ↗⚑ flag

Agent ops

1. ai agentRead experiment configurations and results programmatically to summarize outcomes and recommend ship/rollback decisions

   weight 1 · round to Statsig
   Statsig✓fullprobed8/10

   Statsig provides a Console API (CRUD for experiments/gates), an HTTP API for retrieving experiment/gate values, a CLI scriptable in CI, and an official MCP server explicitly documented to let AI tools like Claude Code/Cursor/Codex 'ask questions, explore experiments, and access your Statsig data' — directly enabling an AI agent to programmatically read experiment configs/results. Runtime probes confirm the MCP server and CLI are live and reachable. Missing for 10: explicit documented example of an agent using these APIs to produce a ship/rollback recommendation (only general data-access capability is evidenced, not the recommendation workflow itself).

   • [claimed-docs] “The Console API is the CRUD API for performing the actions available on console.statsig.com without using the web UI.”
   • [claimed-docs] “Overview of the Statsig HTTP API for retrieving feature gate, experiment, and dynamic config values and logging events directly without an S…”
   • [claimed-docs] “The Statsig MCP (Model Context Protocol) server brings the power of Statsig into tools like Codex, Cursor, and Claude Code. With this setup,…”
   • [claimed-docs] “The Statsig CLI is a command-line interface that helps manage Feature Gates, Experiments, and Dynamic Configs within the Statsig platform.”
   • [claimed-docs] “You can integrate the CLI into CI/CD pipelines to automate the management of Feature Gates and Experiments.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Statsig CLI (npm @statsig/siggy) ran keylessly via `npx -y @statsig/siggy --version`, prin…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.statsig.com/v1/mcp returned HTTP …”

   proof ↗⚑ flag

   Unleash~partialprobed5/10

   Unleash exposes a comprehensive Admin API for programmatic access to flags, variants, and impact metrics (thresholds, error-rate pausing) that an agent could poll to gauge rollout health, and an official MCP server lets LLM agents manage flags. However, evidence shows the MCP server is scoped to 'create and manage feature flags' rather than reading experiment results/outcomes or generating ship/rollback recommendations, and no docs describe an agent-facing results-summary or decision-recommendation workflow. missing for 10: explicit API/MCP support for reading experiment results/outcome data, and any documented agent workflow that summarizes outcomes or recommends ship/rollback decisions.

   • [claimed-docs] “The Admin API provides comprehensive programmatic access to all Unleash resources.”
   • [claimed-docs] “Progress from 25% to 50% only if error rates stay below a threshold.”
   • [claimed-docs] “Automate milestone progression or trigger safeguards that pause rollouts or disable environments when metrics cross a threshold.”
   • [claimed-docs] “Pause automatically when request latency increases during a rollout.”
   • [claimed-docs] “This server enables LLM-powered coding assistants to create and manage feature flags following Unleash best practices.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the docs MCP endpoint https://docs.getunleash.io/_mcp/server completed a FULL keyless initialize handsh…”
   • [claimed-docs] “Variants let you expose different versions of a feature to users, for example, as part of A/B and multivariate testing.”

   proof ↗⚑ flag

Analysis

1. product managerTrust a documented statistics engine (Bayesian or frequentist, with variance-reduction options) behind experiment results

   weight 2 · round to Statsig
   Statsig~partialcommunity6/10

   Statsig documents a full experimentation suite with variance-reduction techniques (CUPED, stratified sampling, switchback tests) and hypothesis/metric scorecards, and community sentiment praises its 'stat rigor,' but the evidence never names or details whether the underlying engine is Bayesian, frequentist, or a documented hybrid methodology. missing for 10: explicit documentation of the statistical model (Bayesian vs frequentist), confidence-interval/p-value methodology details, and independent validation of statistical correctness beyond anecdotal praise.

   • [claimed-docs] “Statsig offers a full suite of experiment measurement tools, including CUPED, Stratified Sampling, Switchback Tests, and more.”
   • [claimed-docs] “The Scorecard feature makes this easy by letting you enter your hypothesis and select both primary and secondary metrics.”
   • [claimed-docs] “Statsig Warehouse Native is an experimentation platform that runs analysis directly in your data warehouse.”
   • [community] “Hats off to Statsig. They built a stellar product. Superior to many of their industry competitors like Optimizely. Back when I was on an int…”

   proof ↗⚑ flag

   Unleash—none0/10

   Unleash's docs mention 'impact metrics' for pausing/progressing rollouts based on thresholds, but there is no evidence of a documented statistics engine (Bayesian/frequentist), significance testing, confidence intervals, or variance-reduction techniques (e.g., CUPED) behind experiment/variant results. A Hacker News comment even notes Unleash's rule system is more limited than competitors, but this doesn't address the statistics engine specifically. missing for 10: any mention of a named statistical methodology, significance/power calculations, confidence intervals, or variance-reduction options for A/B test analysis.

   • [claimed-docs] “Progress from 25% to 50% only if error rates stay below a threshold.”
   • [claimed-docs] “Automate milestone progression or trigger safeguards that pause rollouts or disable environments when metrics cross a threshold.”
   • [claimed-docs] “Pause automatically when request latency increases during a rollout.”
   • [claimed-docs] “Variants let you expose different versions of a feature to users, for example, as part of A/B and multivariate testing.”
   • [community] “Many open source solutions (Unleash, last I checked, some time ago) are more limited: some of th[e]... [compared to LaunchDarkly's more flex…”

   proof ↗⚑ flag

Experiments

1. product managerRun A/B and multivariate experiments on flags and see which variation wins on my metrics

   weight 3 · round to Statsig
   Statsig✓fullcommunity9/10

   Statsig's docs directly describe running A/B/n and multivariate experiments with primary/secondary metrics via the Scorecard, plus Layers for managing multiple experiments and Autotune for identifying/shifting toward winning variants, and community feedback corroborates its statistical rigor versus competitors. Missing for 10: independent hands-on validation of the results/analysis UI itself (e.g., screenshots or third-party review of the 'winning variation' scorecard output) rather than only docs.

   • [claimed-docs] “Statsig experimentation runs randomized controlled trials (A/B or A/B/n tests) that measure how product changes affect your key metrics.”
   • [claimed-docs] “The Scorecard feature makes this easy by letting you enter your hypothesis and select both primary and secondary metrics.”
   • [claimed-docs] “Unlike a standard A/B test, Autotune shifts traffic toward the winning variant to maximize a target metric.”
   • [claimed-docs] “Autotune shifts traffic toward the winning variant to maximize a target metric.”
   • [claimed-docs] “A Layer allows you to manage multiple experiments and feature flags together.”
   • [community] “Hats off to Statsig. They built a stellar product. Superior to many of their industry competitors like Optimizely. Back when I was on an int…”
   • [community] “Have implemented Statsig in two companies as alternatives to LaunchDarkly and loved it both times. Going to be interesting how their other b…”

   proof ↗⚑ flag

   Unleash~partialclaimed5/10

   Unleash documents 'variants' explicitly for A/B and multivariate testing (unleash-docs-8) and has 'impact metrics' that can gate/automate rollout progression based on threshold metrics (unleash-docs-14, unleash-docs-22, unleash-docs-29), giving PMs some data-driven feedback loop on flag variations. However, there is no evidence of a dedicated experimentation dashboard, statistical significance/winner-declaration reporting, or goal-metric comparison across variants akin to a full A/B testing suite. Missing for 10: statistical significance engine, experiment results/winner dashboard, and metric-goal configuration UI tied directly to variant performance.

   • [claimed-docs] “Variants let you expose different versions of a feature to users, for example, as part of A/B and multivariate testing.”
   • [claimed-docs] “Progress from 25% to 50% only if error rates stay below a threshold.”
   • [claimed-docs] “Automate milestone progression or trigger safeguards that pause rollouts or disable environments when metrics cross a threshold.”
   • [claimed-docs] “Pause automatically when request latency increases during a rollout.”

   proof ↗⚑ flag

Metrics

1. product managerDefine experiment metrics from my own data — warehouse tables or ingested events — instead of a black-box metric store

   weight 2 · round to Statsig
   Statsig✓fullclaimed7/10

   Statsig Warehouse Native explicitly runs experiment analysis directly on the customer's own data warehouse rather than a black-box metric store, and Statsig's core event-ingestion pipeline lets PMs define metrics from ingested events with a full analysis toolkit (CUPED, stratified sampling, switchback tests). missing for 10: independent/hands-on corroboration of Warehouse Native in production and detail on the custom-metric-definition UI/workflow itself.

   • [claimed-docs] “Statsig Warehouse Native is an experimentation platform that runs analysis directly in your data warehouse.”
   • [claimed-docs] “Statsig offers a full suite of experiment measurement tools, including CUPED, Stratified Sampling, Switchback Tests, and more.”
   • [claimed-docs] “Statsig experimentation runs randomized controlled trials (A/B or A/B/n tests) that measure how product changes affect your key metrics.”
   • [claimed-docs] “Experiments enable you to run randomized controlled trials (A/B or A/B/n tests) to measure the impact of product changes on key metrics.”

   proof ↗⚑ flag

   Unleash—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

Environments

1. developerManage separate environments (dev/staging/prod) with independent flag states and scoped SDK keys

   weight 2 · round to Unleash
   Statsig—none0/10

   The evidence pack covers feature gates, experiments, CLI, console API, and access management, but nowhere mentions environment tags (dev/staging/prod) or environment-scoped SDK keys — a core Statsig platform capability that isn't documented here. Missing for 10: explicit environment/project configuration docs, environment-scoped SDK key generation, and any mention of promoting flag state across environments.

   ⚑ flag

   Unleash✓fullprobed9/10

   Unleash's docs explicitly describe environments as distinct stages (dev/staging/prod) with independent flag states per environment, and API tokens/keys are scoped to specific project(s) and a single environment, satisfying both independent state and scoped SDK keys. Missing for 10: independent third-party corroboration of multi-environment isolation beyond docs/probe.

   • [claimed-docs] “Environments represent different stages in your development lifecycle. They allow you to manage your product releases from local development…”
   • [claimed-docs] “This allows you to enable a flag in development or test without enabling it in production.”
   • [claimed-docs] “Backend tokens are scoped to one or more projects and a single environment.”
   • [claimed-docs] “Unleash uses API keys to facilitate communication between consuming clients such as SDKs, Unleash Edge, or other tools and automation.”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official unleashorg/unleash-server docker image …”

   proof ↗⚑ flag

Flags

1. developerCreate a feature flag and toggle it live in production within minutes of signing up

   weight 3 · round to Unleash
   Statsig✓fullcommunity8/10

   Docs provide a first-feature tutorial creating a gate, targeting users, and checking it live via SDK within a browser console (statsig-docs-24), plus a free-tier signup with feature flags included (statsig-docs-17), console-based exposure monitoring and overrides (statsig-docs-19, statsig-docs-29), and community praise confirming Statsig ships fast, developer-friendly tooling (statsig-comm-1, statsig-comm-2). Missing for 10: independent hands-on account of actual time-to-first-flag from signup and no third-party benchmark of onboarding speed.

   • [claimed-docs] “This tutorial creates a feature gate, targets mobile platforms and internal testers, then checks it live with the JavaScript SDK in your bro…”
   • [claimed-docs] “You get feature flags and 2 million metered events for free.”
   • [claimed-docs] “After your Feature Gate is live, view Feature Gate exposures in the Statsig console to monitor who encounters your gate.”
   • [claimed-docs] “set up Feature Gate overrides to allow specific users to bypass the gate.”
   • [community] “Hats off to Statsig. They built a stellar product. Superior to many of their industry competitors like Optimizely. Back when I was on an int…”
   • [community] “Have implemented Statsig in two companies as alternatives to LaunchDarkly and loved it both times. Going to be interesting how their other b…”

   proof ↗⚑ flag

   Unleash✓fullprobed9/10

   Docs show a clear quickstart flow: create a feature flag in the Admin UI, connect an SDK, and toggle it via activation strategies without redeploying — and a runtime probe confirms an actual create-flag-and-evaluate loop working end-to-end within minutes on a fresh self-hosted instance. Missing for 10: independent third-party account of a brand-new signup-to-production toggle timing (only a self-hosted probe, not hosted SaaS signup, is verified).

   • [claimed-docs] “Create your first feature flag** using the Unleash Admin UI. **Connect Unleash to your application** by integrating an SDK into your codebas…”
   • [claimed-docs] “They allow you to enable and disable features for certain users without having to redeploy your application.”
   • [claimed-docs] “use one of the client or backend SDKs to connect Unleash with your application”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official unleashorg/unleash-server docker image …”

   proof ↗⚑ flag

2. developerServe multivariate flags and dynamic configuration values (strings, numbers, JSON), not just booleans

   weight 2 · round drawn
   Statsig~partialclaimed6/10

   Statsig docs reference Dynamic Configs alongside Feature Gates and Experiments (e.g., statsig-docs-9, statsig-docs-10, statsig-docs-11, statsig-docs-31 mention 'Dynamic Configs' as a managed entity distinct from booleans), and experiments/layers imply multivariate parameter delivery (statsig-docs-4, statsig-docs-26, statsig-docs-30). However, no evidence explicitly confirms support for strings/numbers/JSON value types or shows a dynamic config JSON payload example. missing for 10: explicit documentation or example showing dynamic config value types (string/number/JSON), SDK code sample retrieving a non-boolean config value.

   • [claimed-docs] “The Statsig CLI is a command-line interface that helps manage Feature Gates, Experiments, and Dynamic Configs within the Statsig platform.”
   • [claimed-docs] “Overview of the Statsig HTTP API for retrieving feature gate, experiment, and dynamic config values and logging events directly without an S…”
   • [claimed-docs] “The Console API is the CRUD API for performing the actions available on console.statsig.com without using the web UI.”
   • [claimed-docs] “You can integrate the CLI into CI/CD pipelines to automate the management of Feature Gates and Experiments.”
   • [claimed-docs] “A Layer allows you to manage multiple experiments and feature flags together.”
   • [claimed-docs] “Statsig experimentation runs randomized controlled trials (A/B or A/B/n tests) that measure how product changes affect your key metrics.”

   proof ↗⚑ flag

   Unleash~partialprobed6/10

   Unleash's docs confirm a dedicated 'variants' concept explicitly used for multivariate/A-B testing, which supports serving multiple values per flag, but the evidence never explicitly documents the variant payload types (string, number, JSON) or shows an SDK example reading a non-boolean value at runtime. missing for 10: explicit documentation of variant payload types (string/number/json) and hands-on/SDK evidence of retrieving a non-boolean variant value.

   • [claimed-docs] “Variants let you expose different versions of a feature to users, for example, as part of A/B and multivariate testing.”
   • [claimed-docs] “Release management in Unleash helps you standardize, automate and safeguard your feature rollouts.”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official unleashorg/unleash-server docker image …”

   proof ↗⚑ flag

Lifecycle

1. platform engineerFind stale flags and code references so temporary flags actually get removed from the codebase

   weight 2 · round drawn
   Statsig—none0/10

   The evidence pack covers feature gates, experiments, rollout scheduling, overrides, CLI, and MCP integrations, but nothing documents stale-flag detection, flag usage/code-reference scanning, or cleanup workflows. This is a reasonable axis for a feature-flag platform, but no capability is evidenced.

   ⚑ flag

   Unleash—none0/10

   The evidence pack covers flag creation, rollout strategies, environments, RBAC, change requests, and impact metrics, but contains no mention of stale-flag detection, flag lifecycle status, or code-reference scanning tools that would help a platform engineer find and remove unused flags from source code.

   ⚑ flag

Targeting

1. developerTarget or exclude specific individual users for a flag (allowlists, beta testers, internal accounts)

   weight 1 · round to Statsig
   Statsig✓fullclaimed8/10

   Docs explicitly cover overrides for targeting/excluding specific users and a guided tutorial that targets internal testers/mobile platforms, directly matching the allowlist/beta-tester/internal-account use case. missing for 10: no independent hands-on confirmation of override UI behavior beyond first-party docs.

   • [claimed-docs] “set up Feature Gate overrides to allow specific users to bypass the gate.”
   • [claimed-docs] “This tutorial creates a feature gate, targets mobile platforms and internal testers, then checks it live with the JavaScript SDK in your bro…”
   • [claimed-docs] “After your Feature Gate is live, view Feature Gate exposures in the Statsig console to monitor who encounters your gate.”

   proof ↗⚑ flag

   Unleash~partialclaimed6/10

   Unleash supports activation strategies and constraints (segments, custom activation strategies and targeting) that can target or exclude specific users/groups, and tokens/environments allow scoping test rollouts to internal accounts, but no evidence pack item explicitly documents a simple 'individual user allowlist/exclude by userId' UI feature or shows a hands-on example of listing specific beta tester IDs. missing for 10: explicit documentation/example of per-user allowlist or exclude-by-userId targeting UI, independent/hands-on confirmation of this exact workflow beyond generic 'custom activation strategies and targeting' marketing copy.

   • [claimed-docs] “A segment is a reusable collection of strategy constraints.”
   • [claimed-docs] “Custom activation strategies and targeting”
   • [claimed-docs] “They allow you to enable and disable features for certain users without having to redeploy your application.”
   • [claimed-docs] “Backend tokens are scoped to one or more projects and a single environment.”
   • [claimed-docs] “This allows you to enable a flag in development or test without enabling it in production.”

   proof ↗⚑ flag

2. developerTarget flags with attribute-based rules and reusable segments so the right users see the right variation

   weight 3 · round to Unleash
   Statsig~partialclaimed5/10

   Docs confirm Feature Gates support attribute-based targeting (e.g., targeting mobile platforms and internal testers) and per-user overrides, indicating rule-based flag targeting is supported. However, the evidence pack never mentions a dedicated 'segments' or reusable audience-list feature, so reusable, cross-flag targeting groups are unevidenced. missing for 10: explicit documentation of reusable segment/audience definitions, evidence of segment reuse across multiple flags/experiments, and independent/hands-on confirmation of complex rule builders.

   • [claimed-docs] “This tutorial creates a feature gate, targets mobile platforms and internal testers, then checks it live with the JavaScript SDK in your bro…”
   • [claimed-docs] “set up Feature Gate overrides to allow specific users to bypass the gate.”
   • [claimed-docs] “Feature Gates, commonly known as feature flags, allow you to toggle the behavior of your product in real time without deploying new code.”

   proof ↗⚑ flag

   Unleash✓fullcommunity7/10

   Unleash docs directly describe segments as reusable collections of strategy constraints, plus custom activation strategies/targeting and attribute-based rules for flag rollout (unleash-docs-9, unleash-docs-17, unleash-docs-18). This directly matches the story's request for attribute-based rules and reusable segments to target specific user variations. Missing for 10: independent/hands-on validation of complex multi-attribute rule evaluation and the HN comment noting relative rule-system limitations versus competitors introduces some doubt but is not a concrete contradiction.

   • [claimed-docs] “A segment is a reusable collection of strategy constraints.”
   • [claimed-docs] “Custom activation strategies and targeting”
   • [claimed-docs] “They allow you to enable and disable features for certain users without having to redeploy your application.”
   • [claimed-docs] “Variants let you expose different versions of a feature to users, for example, as part of A/B and multivariate testing.”
   • [community] “Many open source solutions (Unleash, last I checked, some time ago) are more limited: some of th[e]... [compared to LaunchDarkly's more flex…”

   proof ↗⚑ flag

Access

1. platform engineerRestrict who can change which flags with roles, permissions, and scoped API tokens

   weight 2 · round to Unleash
   Statsig~partialclaimed4/10

   Statsig docs reference enterprise access management with SSO/SCIM and a Console API for programmatic control, implying some access-control layer exists, but no evidence details role definitions, granular permissions, or scoped/limited API tokens for restricting who can change specific flags. Missing for 10: explicit RBAC documentation, permission granularity for flag edits, and scoped/read-vs-write API token support.

   • [claimed-docs] “In many enterprise environments, organizations use SSO and SCIM together to improve security and usability.”
   • [claimed-docs] “Basic settings include automatically adding new users with the same email domain to your project. Statsig also supports SSO to simplify invi…”
   • [claimed-docs] “The Console API is the CRUD API for performing the actions available on console.statsig.com without using the web UI.”

   proof ↗⚑ flag

   Unleash✓fullclaimed8/10

   Unleash documents RBAC at two levels (project and instance), scoped API tokens restricted to specific projects/environments (backend tokens) with non-secret frontend tokens, and change requests requiring approval before changes apply in an environment — together enabling platform engineers to restrict who can change which flags. GitHub evidence also confirms RBAC, SSO, and SCIM as enterprise features. missing for 10: independent/hands-on verification of granular permission enforcement and no first-party detail on custom fine-grained permission roles beyond RBAC docs.

   • [claimed-docs] “Unleash implements role-based access control on two levels”
   • [claimed-docs] “Backend tokens are scoped to one or more projects and a single environment.”
   • [claimed-docs] “Frontend tokens are not considered secret and are safe to expose client-side.”
   • [claimed-docs] “Change requests allow you to require an additional approval step before any changes can be made in an environment.”
   • [github] “This gives you access to a hosted instance with unlimited projects and environments and features such as role-based access control, change r…”
   • [claimed-docs] “Unleash uses API keys to facilitate communication between consuming clients such as SDKs, Unleash Edge, or other tools and automation.”

   proof ↗⚑ flag

Agent ops

1. ai agentCreate and toggle flags through documented APIs, CLIs, or MCP — and the platform can force my changes through approval workflows instead of letting me write to production unreviewed

   weight 3 · round to Unleash
   Statsig~partialprobed4/10

   Statsig documents multiple agent-friendly interfaces for creating/toggling flags — a CLI (also usable in CI/CD), a Console CRUD API, and an official MCP server reachable via OAuth — giving strong coverage of the 'documented APIs, CLIs, or MCP' half of the story. However, there is no evidence of an approval/review workflow that intercepts or blocks agent-initiated changes before they hit production; access-management docs mention SSO/SCIM for identity but not change-approval gating. Missing for 10: documented approval-workflow/change-review gating for flag mutations, evidence that CLI/API/MCP writes can be routed through a review step rather than applied directly.

   • [claimed-docs] “The Statsig CLI is a command-line interface that helps manage Feature Gates, Experiments, and Dynamic Configs within the Statsig platform.”
   • [claimed-docs] “The Console API is the CRUD API for performing the actions available on console.statsig.com without using the web UI.”
   • [claimed-docs] “The Statsig MCP (Model Context Protocol) server brings the power of Statsig into tools like Codex, Cursor, and Claude Code. With this setup,…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Statsig CLI (npm @statsig/siggy) ran keylessly via `npx -y @statsig/siggy --version`, prin…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the hosted MCP server https://api.statsig.com/v1/mcp returned HTTP …”
   • [claimed-docs] “In many enterprise environments, organizations use SSO and SCIM together to improve security and usability.”

   proof ↗⚑ flag

   Unleash~partialprobed7/10

   Unleash documents an Admin API and MCP server for creating/managing flags (unleash-docs-6, unleash-docs-7, unleash-probe-2) and the runtime probe confirms real flag creation via the Admin API (unleash-probe-rt-1). Change requests (unleash-docs-11) and RBAC (unleash-docs-12) provide an approval-gating mechanism for environment changes, but this is an Enterprise-tier feature and evidence does not explicitly confirm it enforces approval on MCP/API-driven writes specifically (only that it applies to 'changes in an environment') nor is a documented CLI shown. Missing for 10: explicit CLI support, confirmation that change-request approval gates API/MCP-originated writes specifically, and independent (non-vendor) verification of the approval workflow blocking agent writes.

   • [claimed-docs] “The Admin API provides comprehensive programmatic access to all Unleash resources.”
   • [claimed-docs] “This server enables LLM-powered coding assistants to create and manage feature flags following Unleash best practices.”
   • [probe] “official MCP server documented at https://docs.getunleash.io/integrate/mcp”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official unleashorg/unleash-server docker image …”
   • [claimed-docs] “Change requests allow you to require an additional approval step before any changes can be made in an environment.”
   • [claimed-docs] “Unleash implements role-based access control on two levels”
   • [github] “This gives you access to a hosted instance with unlimited projects and environments and features such as role-based access control, change r…”

   proof ↗⚑ flag

Approvals

1. platform engineerRequire approvals or change requests before production flag changes go live

   weight 3 · round to Unleash
   Statsig—none0/10

   No evidence describes an approval workflow, change request, or review gate before flag changes go live in production; docs mention scheduled rollouts, overrides, and audit-adjacent features like exposure logging but not a governance approval mechanism.

   ⚑ flag

   Unleash✓fullclaimed8/10

   Unleash docs explicitly describe Change Requests as an approval workflow that requires an additional approval step before changes go live in an environment, and this is corroborated by GitHub listing change requests as a hosted/enterprise feature alongside RBAC and SSO. Missing for 10: independent hands-on verification of the approval workflow in practice and details on multi-stage/production-specific enforcement.

   • [claimed-docs] “Change requests allow you to require an additional approval step before any changes can be made in an environment.”
   • [github] “This gives you access to a hosted instance with unlimited projects and environments and features such as role-based access control, change r…”
   • [claimed-docs] “Environments represent different stages in your development lifecycle. They allow you to manage your product releases from local development…”
   • [claimed-docs] “Unleash implements role-based access control on two levels”

   proof ↗⚑ flag

Audit

1. platform engineerEvery flag change is recorded in an audit log — who changed what, when, and to which value

   weight 3 · round drawn
   Statsig—none0/10

   The evidence pack covers feature gates, experiments, SSO/SCIM, CLI, Console API, and MCP integrations, but nowhere mentions an audit log or change history capturing who changed a flag, when, or to what value. This is a standard governance capability for feature-flag platforms, so its complete absence from the docs pack means the axis applies but is unsupported.

   ⚑ flag

   Unleash—none0/10

   The evidence pack covers RBAC, change requests, environments, and API tokens, but contains no mention of an audit log recording who changed what flag, when, or to what value — the specific capability the story asks about is unevidenced.

   ⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Statsig
   Statsig✓fullprobed8/10

   Statsig documents a full Console API that is explicitly the CRUD API for 'performing the actions available on console.statsig.com without using the web UI,' plus a separate HTTP API for gate/experiment evaluation and event logging, and an official CLI wrapping these for CI/CD scripting — together covering feature gates, experiments, dynamic configs, and access management via API/CLI rather than only the UI. MCP servers and CLI were confirmed live/reachable in runtime probes, reinforcing that programmatic parity is real and operational, not just documented aspiration. Missing for 10: no independent third-party audit confirming 1:1 UI/API feature parity (e.g., some newer UI features like Autotune or Warehouse Native scorecards aren't explicitly confirmed as API-exposed), and no public OpenAPI spec was found (404s on standard paths).

   • [claimed-docs] “The Console API is the CRUD API for performing the actions available on console.statsig.com without using the web UI.”
   • [claimed-docs] “Overview of the Statsig HTTP API for retrieving feature gate, experiment, and dynamic config values and logging events directly without an S…”
   • [claimed-docs] “The Statsig CLI is a command-line interface that helps manage Feature Gates, Experiments, and Dynamic Configs within the Statsig platform.”
   • [claimed-docs] “You can integrate the CLI into CI/CD pipelines to automate the management of Feature Gates and Experiments.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Statsig CLI (npm @statsig/siggy) ran keylessly via `npx -y @statsig/siggy --version`, prin…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.statsig.com/openapi.json, https://docs.statsig.com/swagger.json, https://docs.statsig.c…”

   proof ↗⚑ flag

   Unleash~partialprobed7/10

   The Admin API is documented as providing 'comprehensive programmatic access to all Unleash resources', and a runtime probe confirms a real create-and-evaluate flag loop via the Admin/Client APIs with no UI involved, showing core flag management is fully API-driven. However, some UI-centric workflows (e.g., change request approval flows, RBAC/SSO configuration, impact-metrics dashboards) are not shown to have full API parity in the evidence. missing for 10: explicit documentation/proof that change-request approvals, RBAC/SSO setup, and impact-metrics automation can be fully performed via API rather than just referenced conceptually.

   • [claimed-docs] “The Admin API provides comprehensive programmatic access to all Unleash resources.”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official unleashorg/unleash-server docker image …”
   • [claimed-docs] “Change requests allow you to require an additional approval step before any changes can be made in an environment.”
   • [claimed-docs] “Unleash implements role-based access control on two levels”
   • [claimed-docs] “Automate milestone progression or trigger safeguards that pause rollouts or disable environments when metrics cross a threshold.”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round to Unleash
   Statsig~partialclaimed4/10

   Statsig provides a Console API and CLI for programmatic access to configuration data (feature gates, experiments, dynamic configs), which supports exporting configuration/setup data in a scriptable way, but there is no evidence of a bulk data export feature for user-level event/analytics data in open/standard formats, nor documentation of data portability or account deletion/export workflows for users wanting to leave the platform. missing for 10: documented bulk export of analytics/event data in open formats (CSV/JSON/Parquet), explicit data portability or 'leave the platform' export tooling, evidence of exporting experiment results/metrics data, and any community confirmation of successful full data export.

   • [claimed-docs] “The Console API is the CRUD API for performing the actions available on console.statsig.com without using the web UI.”
   • [claimed-docs] “The Statsig CLI is a command-line interface that helps manage Feature Gates, Experiments, and Dynamic Configs within the Statsig platform.”
   • [claimed-docs] “You can integrate the CLI into CI/CD pipelines to automate the management of Feature Gates and Experiments.”

   proof ↗⚑ flag

   Unleash~partialclaimed5/10

   Unleash is open source and self-hostable, and pricing docs mention 'export/import feature flags for easy migration,' plus the Admin API gives programmatic access to all resources, supporting data portability. However, there's no explicit documented full-data-export format spec, no evidence of exporting audit logs, metrics history, or user/RBAC data, and no dedicated 'leave and take everything' migration guide. Missing for 10: a documented comprehensive data-export/backup format covering all entities (not just flags), evidence of full account data portability, and independent confirmation that exports are truly complete and open-format.

   • [claimed-docs] “Export/import feature flags for easy migration”
   • [claimed-docs] “The Admin API provides comprehensive programmatic access to all Unleash resources.”
   • [claimed-docs] “This guide helps you set up Unleash Open Source or Unleash Enterprise in your own environment using Docker.”

   proof ↗⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round to Unleash
   Statsig—none0/10

   No evidence in the pack indicates Statsig's core platform or source code is released under an open license; only docs, CLI, MCP servers, and SaaS features are mentioned, none of which speak to open-source licensing of the product's source.

   ⚑ flag

   Unleash~partialclaimed5/10

   The GitHub repo (github.com/Unleash/unleash) is cited multiple times and docs explicitly distinguish 'Unleash Open Source' from 'Unleash Enterprise', indicating the core product's source is publicly available. However, the evidence pack never states the specific open-source license (e.g. Apache-2.0) or confirms the full extent of what's open vs. enterprise-gated. Missing for 10: explicit license name/file citation, clarity on which modules are open vs. proprietary enterprise-only code.

   • [github] “Feature flags in Unleash let you test your code with real production data, reducing the risk of negatively impacting your users' experience.”
   • [github] “It also enables your team to work on multiple features simultaneously without the need for separate feature branches.”
   • [github] “This gives you access to a hosted instance with unlimited projects and environments and features such as role-based access control, change r…”
   • [claimed-docs] “This guide helps you set up Unleash Open Source or Unleash Enterprise in your own environment using Docker.”

   proof ↗⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round to Unleash
   Statsig—none0/10

   No evidence anywhere in the pack mentions self-hosting or an on-prem/open-source deployment of the core Statsig platform; all documentation describes a hosted SaaS console, hosted MCP/API endpoints, and cloud-based warehouse-native analysis. Self-hosting is a fair ask for this category (competitors offer it), so absence of evidence means 'none' rather than 'na'.

   ⚑ flag

   Unleash✓fullprobed9/10

   Unleash Open Source is explicitly documented as self-hostable via Docker (unleash-docs-4), and a runtime probe confirms a full keyless self-host roundtrip creating and evaluating a flag with no account or license (unleash-probe-rt-1), directly demonstrating self-hosting of the core product. missing for 10: independent third-party (non-vendor) write-up confirming production-grade self-hosted deployment at scale.

   • [claimed-docs] “This guide helps you set up Unleash Open Source or Unleash Enterprise in your own environment using Docker.”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official unleashorg/unleash-server docker image …”
   • [github] “Feature flags in Unleash let you test your code with real production data, reducing the risk of negatively impacting your users' experience.”

   proof ↗⚑ flag

Pricing

1. product managerSee published pricing and understand what drives cost (seats, MAUs, events, requests) before committing

   weight 2 · round drawn
   Statsig~partialclaimed3/10

   Docs mention a free tier tied to 'events' as a metered unit, giving a partial glimpse of cost drivers, but there is no evidence of a full published pricing page detailing seats, MAUs, or request-based costs across paid tiers. missing for 10: published pricing tiers/page, seat-based pricing, MAU-based pricing, request-based pricing, enterprise pricing details.

   • [claimed-docs] “You get feature flags and 2 million metered events for free.”

   proof ↗⚑ flag

   Unleash~partialclaimed3/10

   The pricing page is referenced and shows some plan-gated features (export/import, custom strategies, SSO) but the evidence never shows actual published prices or explicit cost drivers like per-seat, MAU, event, or request-based pricing that a PM could use to estimate cost before committing. Missing for 10: explicit price figures, cost-driver metrics (seats/MAUs/events/requests), and any tier comparison table showing how usage translates to price.

   • [claimed-docs] “Export/import feature flags for easy migration”
   • [claimed-docs] “Custom activation strategies and targeting”
   • [claimed-docs] “Single Sign-On (SSO) with SAML 2.0 or OpenID Connect (Azure AD, Okta)”
   • [github] “This gives you access to a hosted instance with unlimited projects and environments and features such as role-based access control, change r…”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round to Unleash
   Statsig—none0/10

   No evidence in the pack mentions data residency, regional storage options, or geographic data controls for Statsig's platform; the documentation excerpts cover feature flags, experiments, MCP, CLI, and access management but never address where data is stored or user choice of region.

   ⚑ flag

   Unleash~partialprobed5/10

   Unleash's self-hosted deployment (Docker, own environment) inherently lets users control where infrastructure and thus data resides, and this was verified in a runtime probe of a full self-host flag create/evaluate loop. However, there is no explicit documentation of region/residency options for the hosted Enterprise/Cloud offering (e.g., choosing EU/US data centers), so residency is only achievable via self-hosting rather than a first-class SaaS feature. Missing for 10: explicit region-selection or data-residency documentation for the managed/cloud product, and any compliance certifications tied to specific regions.

   • [claimed-docs] “This guide helps you set up Unleash Open Source or Unleash Enterprise in your own environment using Docker.”
   • [probe] “PROBE runtime (recorded 2026-09-05): full KEYLESS self-host roundtrip on this machine — the official unleashorg/unleash-server docker image …”
   • [github] “This gives you access to a hosted instance with unlimited projects and environments and features such as role-based access control, change r…”

   proof ↗⚑ flag

2. ai-native userControl data retention and deletion

   weight 2 · round drawn
   Statsig—none0/10

   The evidence pack covers feature flags, experimentation, MCP integrations, and access management (SSO/SCIM), but contains no mention of data retention policies, deletion controls, or GDPR/CCPA-style data lifecycle management. This is a fair axis for a SaaS analytics/experimentation platform handling user event data, so absence of evidence yields 'none' rather than 'na'.

   ⚑ flag

   Unleash—none0/10

   The evidence pack covers feature-flag mechanics, deployment, RBAC, and API tokens, but contains no documentation of data retention policies, deletion workflows, or GDPR-style data controls for flags, audit logs, or user data. Self-hosting is mentioned but that alone doesn't constitute a documented retention/deletion control feature.

   ⚑ flag

3. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Statsig—none0/10

   No evidence in the pack addresses whether Statsig's own tools (CLI, SDKs, MCP server, docs site) collect telemetry/usage data or provide an opt-out mechanism; the pack only covers Statsig's core flagging/experimentation features, not its own instrumentation privacy controls.

   ⚑ flag

   Unleash—none0/10

   No evidence in the pack addresses telemetry or usage-tracking opt-out settings for Unleash itself (self-hosted or SaaS); nothing documents what data Unleash collects about its own usage or how to disable it. missing for 10: any mention of telemetry collection, opt-out flags/env vars, or privacy documentation regarding usage tracking.

   ⚑ flag

Rollouts

1. platform engineerGuard a rollout with metrics so a regression is detected and the release is rolled back automatically

   weight 2 · round to Unleash
   Statsig~partialclaimed5/10

   Statsig supports gradual/scheduled rollouts, feature-gate emergency kill switches, and metric-based experimentation with alerting for regressions (Infra Analytics 'set alerts to catch regressions'), giving the pieces needed to guard a rollout with metrics. However, none of the evidence shows an automated rollback action triggered by a detected metric regression — alerts and kill switches appear to require manual intervention rather than an automatic release rollback loop. Missing for 10: documented automatic rollback/kill-switch triggered by metric threshold breach, closed-loop guardrail-to-rollback automation, and independent confirmation this works in production.

   • [claimed-docs] “Have an emergency disable switch that lets you immediately turn off a particular code branch for users in production”
   • [claimed-docs] “Gradually deploy a feature over time by setting up a Feature Gate as a Scheduled Rollout.”
   • [claimed-docs] “Infra Analytics: Monitor and debug service health alongside product outcomes. Ingest metrics and traces with OpenTelemetry, search and group…”
   • [claimed-docs] “Monitor and debug service health alongside product outcomes. Ingest metrics and traces with OpenTelemetry, search and group logs, and set al…”
   • [claimed-docs] “Schedule gradual feature rollouts to safely deploy new code”
   • [claimed-docs] “The Scorecard feature makes this easy by letting you enter your hypothesis and select both primary and secondary metrics.”

   proof ↗⚑ flag

   Unleash✓fullclaimed7/10

   Unleash's impact-metrics feature explicitly supports guarding rollouts with metrics, including automatic pause/rollback when error rates or latency cross thresholds (e.g., 'Automate milestone progression or trigger safeguards that pause rollouts or disable environments when metrics cross a threshold' and 'Pause automatically when request latency increases during a rollout'). This directly matches the story's requirement of metric-guarded rollout with automatic rollback. missing for 10: independent/hands-on verification of the automatic rollback behavior in production, and more detail on how 'rollback' (vs. pause/disable) is technically executed.

   • [claimed-docs] “Progress from 25% to 50% only if error rates stay below a threshold.”
   • [claimed-docs] “Automate milestone progression or trigger safeguards that pause rollouts or disable environments when metrics cross a threshold.”
   • [claimed-docs] “Pause automatically when request latency increases during a rollout.”
   • [claimed-docs] “Release management in Unleash helps you standardize, automate and safeguard your feature rollouts.”

   proof ↗⚑ flag

2. developerRoll a flag out progressively by percentage with consistent bucketing, ramping from 1% to 100% without redeploying

   weight 3 · round to Unleash
   Statsig✓fullclaimed8/10

   Statsig docs explicitly describe scheduled/gradual rollouts to safely deploy new code without redeploying, and deterministic evaluation ensures consistent bucketing across evaluations for the same user given the same gate state. Percentage-based ramping (1%→100%) is a core documented Feature Gate/Scheduled Rollout capability, backed by consistent hashing via deterministic evaluation. missing for 10: explicit mention of percentage granularity/UI slider mechanics and independent hands-on confirmation of ramp behavior.

   • [claimed-docs] “Gradually deploy a feature over time by setting up a Feature Gate as a Scheduled Rollout.”
   • [claimed-docs] “Schedule gradual feature rollouts to safely deploy new code”
   • [claimed-docs] “Evaluation in Statsig is deterministic. Given the same user object and the same state of the experiment or feature gate, Statsig always retu…”
   • [claimed-docs] “Evaluation in Statsig is deterministic. Given the same user object and the same state of the experiment or feature gate, Statsig always retu…”
   • [claimed-docs] “Feature Gates, commonly known as feature flags, allow you to toggle the behavior of your product in real time without deploying new code.”

   proof ↗⚑ flag

   Unleash✓fullclaimed9/10

   Unleash's gradual rollout activation strategy is a documented first-class concept, explicitly supporting progressive percentage rollout with consistent bucketing (via stickiness) without redeploying, and impact-metrics docs show automated progression between percentage thresholds (e.g., 25% to 50%) based on health checks. Missing for 10: no independent hands-on evidence of the specific 1%→100% ramp UI/automation flow in practice beyond docs.

   • [claimed-docs] “A **gradual rollout** is a controlled release strategy where a new feature is first released to a small subset of users.”
   • [claimed-docs] “They allow you to enable and disable features for certain users without having to redeploy your application.”
   • [claimed-docs] “Progress from 25% to 50% only if error rates stay below a threshold.”
   • [claimed-docs] “Automate milestone progression or trigger safeguards that pause rollouts or disable environments when metrics cross a threshold.”
   • [claimed-docs] “Pause automatically when request latency increases during a rollout.”

   proof ↗⚑ flag

Scheduling

1. developerSchedule flag changes and releases to happen at a specific future time

   weight 1 · round to Unleash
   Statsig~partialclaimed6/10

   Docs confirm Statsig supports 'Scheduled Rollout' for Feature Gates, letting you gradually deploy a feature over time rather than instantly ('Gradually deploy a feature over time by setting up a Feature Gate as a Scheduled Rollout', 'Schedule gradual feature rollouts to safely deploy new code'), which implies future-time scheduling of flag state changes. However, the pack lacks detail on setting an exact future timestamp for a release event, calendar/date-picker UI specifics, or independent confirmation of this exact workflow. Missing for 10: explicit documentation of specifying an exact date/time for a rollout step, screenshots or hands-on walkthrough, and third-party confirmation that scheduled changes fire reliably.

   • [claimed-docs] “Gradually deploy a feature over time by setting up a Feature Gate as a Scheduled Rollout.”
   • [claimed-docs] “Schedule gradual feature rollouts to safely deploy new code”

   proof ↗⚑ flag

   Unleash✓fullclaimed8/10

   Unleash documents a dedicated how-to guide for scheduling feature releases at a specific date/time, framed explicitly for use cases like product launches, and this sits alongside broader release management, gradual rollout, and environment concepts that support scheduled changes. Missing for 10: independent/hands-on corroboration of the scheduling feature working in practice, and detail on granularity/timezone handling or recurring schedules.

   • [claimed-docs] “to release a feature at a specific date and time** (for a product launch, for instance)”
   • [claimed-docs] “to release a feature at a specific date and time (for a product launch, for instance)”
   • [claimed-docs] “Release management in Unleash helps you standardize, automate and safeguard your feature rollouts.”

   proof ↗⚑ flag

Edge

1. platform engineerEvaluate flags at the edge (CDN workers or an edge/relay layer) close to users

   weight 2 · round to Unleash
   Statsig—none0/10

   The evidence pack covers Statsig's feature gates, on-device eval SDKs for client-side, server SDKs, and CLI/API surfaces, but nothing addresses evaluating flags specifically at CDN/edge workers or an edge/relay layer close to users. Missing for 10: any mention of Cloudflare Workers, edge runtime SDKs, or a proxy/relay layer for edge evaluation.

   • [claimed-docs] “Statsig's client-side On-Device Eval SDKs provide an alternate client-side architecture. The device keeps the definition of each experiment …”
   • [claimed-docs] “Statsig's client-side On-Device Eval SDKs provide an alternate client-side architecture. The device keeps the definition of each experiment …”
   • [claimed-docs] “Evaluation in Statsig is deterministic. Given the same user object and the same state of the experiment or feature gate, Statsig always retu…”

   proof ↗⚑ flag

   Unleash✓fullclaimed8/10

   Unleash Edge is a documented purpose-built component (Rust-based caching/relay layer) that sits between SDKs and the primary server, described explicitly as reducing read load and improving scalability/performance/resilience for connected SDKs, functioning as a low-latency edge/relay evaluation layer close to users. This directly matches the story, though evidence doesn't explicitly show deployment as a CDN/edge-worker runtime (e.g., Cloudflare Workers) itself. missing for 10: explicit CDN-worker (e.g., Cloudflare Workers) deployment guide, independent performance/latency benchmarks for Edge in production.

   • [claimed-docs] “functioning as a read replica that can handle thousands of connected SDKs without increasing the read load on your primary Unleash instance”
   • [claimed-docs] “Unleash Enterprise Edge is a lightweight caching layer designed to improve scalability, performance, and resilience.”
   • [claimed-docs] “Unleash uses API keys to facilitate communication between consuming clients such as SDKs, Unleash Edge, or other tools and automation.”

   proof ↗⚑ flag

Evaluation

1. platform engineerMy server SDKs evaluate flags locally from a cached ruleset — microsecond decisions with no network call per flag check

   weight 3 · round to Statsig
   Statsig~partialclaimed5/10

   Docs confirm deterministic evaluation 'even across different platforms (client or server)' implying server SDKs replicate rule logic locally, and explicitly describe on-device eval SDKs keeping definitions in memory for fast evaluation — but this on-device architecture is documented only for client SDKs, not explicitly server SDKs. Missing for 10: explicit server-SDK docs on background polling/streaming to a local cache, no-network-call guarantee per check, and microsecond-latency benchmarks specific to server SDKs.

   • [claimed-docs] “Evaluation in Statsig is deterministic. Given the same user object and the same state of the experiment or feature gate, Statsig always retu…”
   • [claimed-docs] “Statsig's client-side On-Device Eval SDKs provide an alternate client-side architecture. The device keeps the definition of each experiment …”
   • [claimed-docs] “Evaluation in Statsig is deterministic. Given the same user object and the same state of the experiment or feature gate, Statsig always retu…”
   • [claimed-docs] “Statsig's client-side On-Device Eval SDKs provide an alternate client-side architecture. The device keeps the definition of each experiment …”

   proof ↗⚑ flag

   Unleash~partialclaimed3/10

   Evidence shows Unleash SDKs communicate via API keys and that Edge exists as a caching layer to reduce read load from 'thousands of connected SDKs,' implying a poll-and-cache model, but no evidence explicitly describes local, in-process flag evaluation from a cached ruleset or confirms 'no network call per flag check' / microsecond latency. missing for 10: explicit SDK architecture docs describing local evaluation engine, cache refresh intervals, and no-network-per-check guarantee, plus independent benchmarks.

   • [claimed-docs] “functioning as a read replica that can handle thousands of connected SDKs without increasing the read load on your primary Unleash instance”
   • [claimed-docs] “Unleash Enterprise Edge is a lightweight caching layer designed to improve scalability, performance, and resilience.”
   • [claimed-docs] “Unleash uses API keys to facilitate communication between consuming clients such as SDKs, Unleash Edge, or other tools and automation.”

   proof ↗⚑ flag

Sdks

1. developerUse official SDKs across my whole stack — backend, web, and mobile — with consistent flag behavior

   weight 2 · round drawn
   Statsig~partialclaimed5/10

   Statsig documents deterministic evaluation guaranteeing identical results 'across different platforms (client or server)' and offers On-Device Eval client SDKs plus a JS SDK tutorial that also targets mobile platforms, implying a consistent cross-stack SDK model. However, the pack never enumerates specific backend/web/mobile SDK names or shows hands-on parity testing across a real multi-platform stack. Missing for 10: an explicit list of officially supported language/platform SDKs, and independent/hands-on evidence confirming identical flag behavior when used together across backend, web, and mobile in a real project.

   • [claimed-docs] “Evaluation in Statsig is deterministic. Given the same user object and the same state of the experiment or feature gate, Statsig always retu…”
   • [claimed-docs] “Evaluation in Statsig is deterministic. Given the same user object and the same state of the experiment or feature gate, Statsig always retu…”
   • [claimed-docs] “Statsig's client-side On-Device Eval SDKs provide an alternate client-side architecture. The device keeps the definition of each experiment …”
   • [claimed-docs] “Statsig's client-side On-Device Eval SDKs provide an alternate client-side architecture. The device keeps the definition of each experiment …”
   • [claimed-docs] “This tutorial creates a feature gate, targets mobile platforms and internal testers, then checks it live with the JavaScript SDK in your bro…”

   proof ↗⚑ flag

   Unleash~partialclaimed5/10

   Docs confirm Unleash's SDK model — distinguishing backend/server SDKs (secret tokens) from client-side SDKs for web/mobile (safe-to-expose frontend tokens) and instruct developers to 'use one of the client or backend SDKs to connect Unleash with your application.' However, the pack never lists concrete SDK names/languages (e.g., specific mobile or web SDKs) or documents consistent flag-evaluation behavior across them. Missing for 10: explicit enumeration of officially supported backend/web/mobile SDKs, evidence of consistent evaluation semantics across SDKs, and independent developer corroboration of cross-stack consistency.

   • [claimed-docs] “use one of the client or backend SDKs to connect Unleash with your application”
   • [claimed-docs] “Frontend tokens are not considered secret and are safe to expose client-side.”
   • [claimed-docs] “Unleash uses API keys to facilitate communication between consuming clients such as SDKs, Unleash Edge, or other tools and automation.”
   • [claimed-docs] “Backend tokens are scoped to one or more projects and a single environment.”

   proof ↗⚑ flag

Standards

1. platform engineerUse the vendor through OpenFeature providers so my flag code isn't locked to one vendor's SDK API

   weight 1 · round drawn
   Statsig—none0/10

   No evidence pack item mentions OpenFeature or a provider implementation for it; all SDK references are to Statsig's own proprietary SDKs, CLI, HTTP API, and Console API. This is a fair axis for a feature-flag/experimentation platform, so absence of evidence yields 'none'.

   ⚑ flag

   Unleash—none0/10

   The evidence pack discusses Unleash's own SDKs, Admin API, Edge, and MCP integration but never mentions OpenFeature or any OpenFeature provider, so there is no evidence supporting vendor-neutral SDK usage via OpenFeature.

   • [claimed-docs] “Create your first feature flag** using the Unleash Admin UI. **Connect Unleash to your application** by integrating an SDK into your codebas…”
   • [claimed-docs] “use one of the client or backend SDKs to connect Unleash with your application”
   • [claimed-docs] “functioning as a read replica that can handle thousands of connected SDKs without increasing the read load on your primary Unleash instance”
   • [claimed-docs] “The Admin API provides comprehensive programmatic access to all Unleash resources.”

   proof ↗⚑ flag

Streaming

1. developerFlag changes propagate to connected SDKs in seconds via streaming or fast polling — a kill switch actually kills

   weight 2 · round to Statsig
   Statsig~partialclaimed7/10

   Statsig's docs explicitly describe real-time toggling and an 'emergency disable switch' to immediately turn off a code branch in production, plus deterministic evaluation and on-device SDKs for fast client updates — directly supporting the kill-switch and fast-propagation story. However, there is no explicit documentation of streaming vs polling mechanics, propagation latency in seconds, or independent/hands-on verification of SDK update speed. Missing for 10: documented streaming/polling interval specifics, third-party benchmark or hands-on confirmation of sub-second/seconds-level propagation.

   • [claimed-docs] “Feature Gates, commonly known as feature flags, allow you to toggle the behavior of your product in real time without deploying new code.”
   • [claimed-docs] “Have an emergency disable switch that lets you immediately turn off a particular code branch for users in production”
   • [claimed-docs] “Evaluation in Statsig is deterministic. Given the same user object and the same state of the experiment or feature gate, Statsig always retu…”
   • [claimed-docs] “Statsig's client-side On-Device Eval SDKs provide an alternate client-side architecture. The device keeps the definition of each experiment …”

   proof ↗⚑ flag

   Unleash~partialclaimed4/10

   Docs confirm the core mechanic — flags can be enabled/disabled without redeploy (unleash-docs-18) and SDKs/Edge communicate via API keys with Edge scaling to thousands of connected SDKs (unleash-docs-5, unleash-docs-23, unleash-docs-28) — but there is no explicit documentation of streaming vs. polling propagation times, no stated 'seconds' latency figure, and no hands-on/independent test confirming a kill switch instantly disables a live SDK. missing for 10: explicit streaming/polling interval docs, measured propagation latency, independent or hands-on confirmation of kill-switch behavior in production SDKs.

   • [claimed-docs] “They allow you to enable and disable features for certain users without having to redeploy your application.”
   • [claimed-docs] “functioning as a read replica that can handle thousands of connected SDKs without increasing the read load on your primary Unleash instance”
   • [claimed-docs] “Unleash uses API keys to facilitate communication between consuming clients such as SDKs, Unleash Edge, or other tools and automation.”
   • [claimed-docs] “Unleash Enterprise Edge is a lightweight caching layer designed to improve scalability, performance, and resilience.”

   proof ↗⚑ flag